Measuring Outcomes and Evaluating Programs

As mentioned in a previous posting, many non-profit organizations spend significant time outlining project goals, while devoting relatively few resources to the task of evaluating whether these goals are met. The lack of output data often leads to a crisis in writing the final report. Many organizations compensate by overemphasizing stories of transformation at the expense of verifiable facts. Unfortunately, this “solution” to the problem is often misleading to donors and detrimental to the long-term health of the organization.

So what can you do if you are writing the final report for your program and you are faced with a lack of hard information about outcomes? Here are three low cost solutions to make the most of the information at your disposal and hopefully get you out of this predicament:

1. Cost per beneficiary: Dividing the total program cost by the number of beneficiaries gives donors and organizational leaders an understanding of the breadth of your program’s impact. Simply put, the lower the cost per beneficiary, the more people your program can reach. The principal benefit of looking at the actual vs. expected cost per beneficiary is that it yields a precise picture of the economic efficiency of your program’s execution. Armed with this information, decision-makers in your organization will be able to refine the program to increase its overall efficiency.

2. Sector indicators and proxy measures: Another simple way to evaluate your program is to enlist the use of existing sector indicators. For example, changes in literacy rates, standardized test scores or grade-level retention rates can assess educational programs. Similarly, proxy measures can be employed when direct indicators do not exist. One example of a proxy measure is the use of teen pregnancy rates to assess the impact of school-based HIV/AIDS prevention education programs. Additionally, school attendance rates can be used as an indirect measure of the impact of public health programs on a community because healthy children attend school. The benefit of this approach in evaluation is that the measures used already exist and the data is more easily collected and processes. The weakness of this approach is that it cannot take into account how other, similar interventions implemented at the same time might be impacting your beneficiaries. In this sense, the approach lacks the control offered by a more thorough study; thus its results are less certain. As an example, when attempting to measure the impact of a malaria prevention program occurring at the same time as a deworming program, it is impossible to separate the effects of the deworming program from the malaria prevention program when using existing indicators and proxy measures because the data is common to the population as a whole, not to the beneficiaries of your program.

3. Social return on investment (SROI): Given the difficult financial climate of the present recession and the shrinking of foundation endowments, social investors are increasingly seeking confirmation of the return they are getting on their philanthropic investments. SROI is helpful because it provides a metric to evaluate programs across sectors (education vs. health vs. microfinance) by monetizing the outcome. It takes into account the number beneficiaries served and the type of program intervention. Similar to the cost-per-beneficiary approach, an actual vs. expected SROI can assess a program’s productivity against the program’s design, that is, how it actually performed vis-a-vis how it was designed to perform. This approach is not for everyone, as many have difficulty assigning a dollar value to how program beneficiaries are impacted.

So what if you have foresight and are want to implement an evaluation plan for your program prior to its commencement?  Thoughts on this next week.

1 comment to Measuring Outcomes and Evaluating Program, Part 1