Natasja Sheriff, Project Leader of the WorldFish KSinR Pilot Project, together with Tonya Schuetz (IWMI)-who helped facilitate the WorldFish Pilot’s training workshop, wrote and submitted a paper for the workshop ‘Rethinking impact: Understanding the Complexity of Poverty and Change‘ which was convened in Cali, Colombia 26-29 March 2008.
The paper, entitled “Monitoring for change, assessing for impact: the WorldFish center experience” can be viewed from the following link- paper. This paper was based on the experience gained from the initial introduction to and use by the Project of Outcome Mapping and Most Significnat Change-the workshop which was described in a previous bog post.
According to the paper “like many CG centers, a traditional emphasis on the development and dissemination of new technolgies has shaped impact assessment within the WorldFish center” and ” assessing the impact of projects undertaken…has largely been quantitative in nature, applying economic models to assess productivity, welfare and technological efficiency for example”.
This paper contends that “in comparison to ex post impact assessment activities, less attention has been given to monitoring and evaluation, and to the process of learning and adaptation, during project implementation”.
This paper outlines the new trend of research towards a broader approach to addressing poverty alleviation and the move towards development and application of methods which increase the impact of agricultural research on poverty and which facilitate learning and change.
The paper posits that “there is a lack of appropriate, effective tools for participatory monitoring and evaluation for application in a natural resource management context” and “simultaneously there has been a trend towards increased partner collaboration and impact-oriented research which requires a more responsive and adaptive approach to impact assessment and M&E than has been previously applied”.
This was the driver behind the proposal of the CP35 project at the WorldFish Centre to the Knowledge Sharing in Research call for proposals, to pilot new M&E methods to compliment existing quantitative M&E tools, and to support a more open and responsive approach to change occurring in communities involved in the project. This paper outlines the initial experiences of the project in piloting new approaches to M&E and impact assessment mainly in the form of Outcome Mapping and Most Significant Change methods.
The paper provides a rationale for the choice of particular approaches, examining what potential fit and benefit they would have with the research project into which they would be integrated.
Next a clear description of the activities undertaken to introduce and initiate such methods was provided.
The real ‘meat’ of the paper comes in the strong review and analysis of the piloting of these kind of methods which is presented in the form of annotated lists of ‘benefits’ and ‘issues hindering effectiveness’ of each of the methods being employed. While there were some initial positive signs of benefits that would/could be derived from using such methods, the challenges, issues and concerns raised were more of a highlight.
Some benefits include:
- Creating a longer term vision for sustainability and impact
- Identifying unanticipated problems and constraints to project success
- Revealing outcome and impact priorities held by project participants and stakeholders
- Creating a sense of ownership and responsibility for project success
- Clarifying roles and responsibilities
- Articulating where change is needed and monitoring progress towards required change
Some disadvantages identifed were:
- The potential for unequal power relationships amongst stakeholders (and even team) to influence the process/method
- Relative complexity of the approach
- Difficulty in communicating terminologies and processes in various languages
- Substantial time investment of project team and stakeholders to work through OM
- Potential for misinterpretation and inappropriate application of the concept of ‘behavioural change’
Although some negative consequences were described, these were proposed to be valuable learning experiences from which specific attention could be paid to relevant modifications and adaptations which could be made for future use in the project-in its other country sites.
The authors concluded, therefore that “there is a need to carefully evaluate alternative methodologies available to research scientists and to put forward appropriate tools for impact assessment and M&E that can be readily taken up and applied in R4D, particularly in the natural resource management context”.