In this step you will determine if your intervention was effective and why or why not. Evaluate the effectiveness of your measure(s) according to the detailed evaluation plan developed in step 4. Consider both short- and long-term evaluations.
Any evaluation should meet the following criteria: utility (useful to some audience), feasibility (in political, practical and cost-effectiveness terms), propriety (carried out fairly and ethically), and technical adequacy (carried out with technical skill and sensitivity). These four criteria are culturally bound and therefore not easy to be made operational and measured. However, the toolbox gives some useful hints to overcome this problem.
| Actions | Questions to be answered | Issues to be considered |
|---|---|---|
| Evaluation of effects | What was the result of your measure(s)? | -* displaced: Did the problem move to a different location? -* reduced: Was the problem reduced? -* unchanged: Did the problem remain? -* eliminated: Was the problem ended? |
| Evaluation of costs and encountered problems | Were the effects as expected? Did each measure achieve its aim? | -* quantitative or qualitative evaluation (quantitative preferred) -* reasons for unexpected results |
| Was any part of the plan not implemented? If so, why? | -* quantitative or qualitative evaluation (quantitative preferred) -* reasons for unexpected results | |
| What was the cost-effectiveness of the measure(s)? | -* CBA analysis report -* CEA analysis report | |
| Evaluation of the whole intervention process and problem approach | Were the underlying causes of the problem properly identified? | -* qualitative evaluation -* need to review the current intervention programme? -* need to review the current evaluation design? -* need to further analyse the problem? |
| Were the key stakeholders and resources identified and included? | -* qualitative evaluation -* need to review the current intervention programme? -* need to review the current evaluation design? -* need to further analyse the problem? | |
| Is it necessary to return to the analysis step to re-consider the problem? | -* qualitative evaluation -* need to review the current intervention programme? -* need to review the current evaluation design? -* need to further analyse the problem? | |
| Evaluation of implications | What are the conclusions and recommendations? | -* implications for policy, practice and innovation -* implications for the future of the intervention (Who will need to be further involved? What could happen if the measures are left in place? What could happen if they are taken away?) -* implications for future evaluation and research |
| Will the intervention require a long-term commitment and monitoring? | -* where? -* for how long? | |
| Publication of results | Will the results be available to all those who could potentially benefit from them? | -* scientific articles -* research reports -* conference papers -* website -* media announcements |
It is important to note that every set of measures may ask for a different approach in the evaluation process. It is important to think about the evaluation process beforehand and perform it during the 6 steps of the process. The first outline for the evaluation can be drawn in the steps that focus on the analysis of the problem (steps 1 and 2). The evaluation can then be performed through preliminary CBA /CEA as part of step 3 (selection of measures) and elaborated in the implementation plan (step 4), together with the solutions chosen. The evaluation is to be concluded in step 6.
Process and analyse the evaluation data. One possible way is to compare the before-implementation data with that collected after implementation. Draw clear conclusions about the measure and consider the possible shortcomings of the study. The analysis should not only focus on the success of the measures but also investigate the impact and meaning of the specific variables (regarding location and other relevant factors) for the problem. In turn, this leads to a better understanding of the nature of the problem. Provide recommendations and guidelines for the future set-up of these measures. Communicate your intervention and share your results.
RSS 2.0