Measuring and Evaluating Strategies for Philanthropies

The last posting provided three low cost suggestions you can potentially employ if you find yourself writing your final report and lacking information on outcomes. Alternatively, if your a philanthropy and have foresight to plan your program evaluation prior to the program’s commencement (which is ideal), then here are several suggests that you can employ:

1.  Latent trait analysis: A latent trait is something that is hidden or invisible. Knowledge, for example is a latent trait. The goal of HIV/AIDS prevention education programs is to alter behavior through changing people’s knowledge and attitudes. Latent trait analysis, known also as Item Response Theory, can be utilized to do this. Read this basic overview about how IRT is used in creating measurement tools. The benefit of this approach is that it provides the possibility to measure things which people typically believe cannot be measured. Creating a measurement tool using IRT, however, can take considerable resources.

2.  Simple random sample: Create questions that can be asked of your beneficiary population that will help assess if your programmatic goals will be met. Next, select your random sample. Ask the questions of this sample before the program begins, and again after it has been completed. For more details on how to perform a simple random sample survey, read The Right and Wrong Way to Survey. The strength of this approach is that it is a widely accepted framework for evaluation. If there is more than one intervention occurring on the population, however, the random sample survey may not distinguish what is attributed to the impact of your intervention.

3.  SMS texting: In the coming years, we will see texting increasing used in evaluation. One reason is the spread of cell phones, seemingly to every corner of the globe; it was recently reported that Africa has surpassed the United States in cell phone users. The useful thing about SMS is that it provides both the opportunity to gauge the impact of a program as well as to improve it. An example of this is a tuberculosis treatment program in Pakistan, where patients send a text message after they have taken the medicine. If patients do not respond within a prescribed time period, a series of reminder text messages are sent. A strength of this approach is the real-time, dynamic nature of the tool. One downside is that, while cell phones are widely available, not everyone has them. More on this soon.