To be able to design an appropriate evaluation methodology, you must be noticeably clear about its purpose. Why is the evaluation carried out and who is it supposed to be? Does your donor request you to do the evaluation? Do you want to evaluate your projects internally to see the possible progress potential? Is it both?

The purpose of the evaluation mostly sets the bar for its scope and form. Many times, the donor already has specific expectations that need to be met and specific regulations that need to be fulfilled. Sometimes even legal requirements come into play. The clearer you are about the purpose of your evaluation, the easier it is to define its form and the appropriate way to go about it.

The second thing you should consider while designing your evaluation methodology is the scope of the evaluation. Deciding the scope means deciding which interventions will be evaluated, which geographical area will be covered, and which period will be covered.

If you are working on an exceedingly small project, these questions are normally easy to answer. If your project just comprises a few interventions, a defined geographical area, and a limited period, your evaluation should cover the entire project. If you already know that evaluating certain aspects will be particularly challenging, it might be a clever idea to exclude them from the final evaluation to adjust the donor's expectations. This might apply if your project just aims to kick-start a process that will only show its impact in the long term (after your evaluation takes place) or if you already know that several measures out of your control will probably make it difficult to evaluate your own project activities. Be clear about it, though, so that the donors know what to expect or could object if they do not agree with your approach.

In this step, you should be able to describe the interventions planned, their potential impact, and their interactions during your project phase. You should also consider external actions that might influence the implementation of your project, being positive or negative.

Writing down or making a diagram of the intervention logic makes sure that you clearly understand how the project is supposed to work and what was expected in the beginning. The intervention logic is dynamic and might change during the project, but these changes can be documented and give insight into areas where plans and expectations need to be adjusted to reality.

Once you detail how your project is planned to work and how the expected impact is, you can formulate good evaluation questions. Evaluation questions are those questions that are supposed to be answered by your evaluation. It gives you the chance to specify what you want to analyze in your evaluation.

If you word your questions carefully, you can make sure a critical analysis can take place. Be careful not to end up with simple Yes/No questions, which seem easy to answer, but will give almost no insight in the end and thus will have extraordinarily little additional value for the donor or your organization.

At the same time, you should try to find questions you can answer. While project applications are full of promised impact, it is difficult to measure impact. To be able to assess the impact of your project, you would need to take a lot of data also out of your project to be sure no external events influenced the outcome of your project. Even with a huge dataset, it is almost impossible to be 100% sure of the impact your interventions had because things like policy changes, general opinion, or other events might play a role that you are not even aware of.

Sometimes that means breaking one issue down into several questions. These questions can be quantitative (answered by hard data, numbers, etc.) or qualitative (opinions, perceptions, etc.). 

It is also important not to forget to allocate resources to your evaluation methodology to be able to carry it out. Often, the evaluation does not get enough attention regarding resources, and people do not have enough designated time to do it. Particularly in smaller projects, sometimes the project manager must do the evaluation “on the side” of his or her normal tasks. This poses several risks, as not enough time is designated for the important task and the project manager might be biased.

Setting aside labor and resources for the evaluation from the first project phase shows responsible behavior on the organization’s side and guarantees that the evaluation will be carried out professionally.

Once you have carried out the above-mentioned steps (ideally in a team), you have gathered enough information to design your evaluation methodology. You have decided which methods you will need and which data you will have to collect for that, and ideally already allotted the corresponding responsibilities to the assigned staff so that everybody knows what her or his role is in the process.

If you put this information together in a document, it is also a good opportunity to share this with your donors or potential donors. A thought-through evaluation methodology shows that you and your organization are remarkably familiar with the working area of your project, have put a lot of thought into the design, and are able and willing to critically analyze your project interventions. It creates transparency and thus more reason for the donors to trust you and your organization. It also makes common ground with respect to expectations towards the evaluation report in the end and gives all stakeholders the opportunity to add input if needed and desired.