Classic evaluation thinking, as distinct from monitoring or verification, requires the evaluator to make a systematic and objective assessment of the design, implementation and results of a programme.
This thinking adds value to prizes for development by focusing attention on the logic before and beyond the prize award. It challenges assumptions behind a prize process, considering the wider context in which prizes are awarded (such as other policy initiatives), and it raises deeper questions about why it might or might not work. All of this becomes even more important where prizes are seeking to ultimately have a societal and environmental impact.
An evaluation tool that brings a lot of benefits to prizes
In terms of specific evaluation tools, we have found that the development of a Theory of Change is particularly valuable for prizes for development:
“Theory of Change is essentially a comprehensive description and illustration of how and why a desired change is expected to happen in a particular context” (Center for Theory of Change)
Theory of Change enables prize teams to articulate assumptions and risks, to navigate the complexity that accompanies innovation prizes, and to inform decisions during launch and implementation.
A Theory of Change helps innovation prizes focus on meeting development outcomes over simply awarding prizes. However, in our experience, it is not something that the evaluator of a prize can develop on its own.
Developing a Theory of Change requires, and benefits, from involvement from across the prize team. In several cases, we have found that the process of questioning assumptions about how the prize connects to intended outcomes has contributed to refinements being made to the design.
A good example of this occurred during the design of the Adaptation at Scale prize (one of the five Ideas to Impact prizes). The Theory of Change was developed iteratively, with a first draft developed by the lead evaluator for the prize, based on existing designs and discussions with the wider design team. When the prize team reviewed the prize logic, as captured in the draft Theory of Change, they identified a gap in the design. This led to the first stage prize being refocused and redesigned and adapted – an iterative process between the evaluators and implementation team.
We’ve also seen with the Adaptation at Scale prize how the Theory of Change can be used as a tool to help prize teams communicate with other stakeholders about the prize logic and expected outcomes, through participatory workshops.
What do you need in place before you start planning the evaluation?
As we’ve observed previously , innovation prizes for development bring together three different fields that need to understand each other and work collaboratively.
When planning evaluation of innovation prizes, we think it is key for the evaluator to fully understand the other two domains (innovation prizes, and the development sector).It is also important to be involved while there is still an opportunity to refine the Theory of Change and contribute to the planning. This helps ensure prize decisions have a sound foundation that can be tested in the future and hopefully improve the data available when the evaluation takes place.
Innovation prize designers bring the necessary expertise in how innovation prizes work and a rich understanding of how typical solvers behave. It is important the evaluator the evaluator accommodates innovation prize conventions in evaluation plans, to embed robust data on solvers (and especially non-winners).
This is our final post in this series of reflections on planning evaluations of prizes for development. Look out for more posts from the Evaluation Team as we start analysing our evaluation data to see what we’re learning about innovation prizes.
Cheryl Brown is the Evaluation & Learning Coordinator on Ideas to Impact and Itad Ltd, an innovative monitoring and evaluation consultancy.