By Mara J. Roberts
As we mentioned back in June, we recently were contracted to measure the impact of a Neglected Tropical Disease program in Burundi. The client had measured the medical benefits of the de-worming program, but wanted to know the social benefits that resulted from the de-worming. Were more children going to school? Were communities being empowered? The difficult part of accomplishing this task was that the three year program had already concluded.
Because of the retrospective nature of the assessment, we decided to employ a mixed mode approach that blended both quantitative and qualitative methods. We utilized quantitative methods to measure things like body mass index, literacy and cognitive development. From our literature review, this was likely the first assessment of literacy and cognitive development on primary school children (ages 10-19) undertaken there. We also used a cluster sample in two stages to 1) randomly select which schools would participate in the survey, and 2) to randomly select which students at these schools would take the survey. The sample was stratified by both geography and gender.
The qualitative methods included an arts based approach through student drawings. Students were asked to create two drawings: 1) what the life was like before the medications and 2) what life was like after the medications. Then, through the iterative process known in qualitative research circles as Action Research, we determined how the NTD program impacted students’ access to education. This Action Research process is comprised of three steps: look, think, and act. These iterative stages formed an “adaptive” or “dynamic” process where the results and conclusions from one stage was the basis for how the questions and design for the next stage was determined.
Given the results from the drawings, we looked at how the school enrollment and standardized test scores at the primary schools in the survey had changed over the course of the de-worming program by obtaining this information from the school headmasters. The last step in this iterative process was to survey a sample of teachers to assess their opinions on the connection between the improved health of the students as a result of the NTD program and the change in enrollment and test scores.
While the preferred method for evaluating the social impact of the NTD program in Burundi would have been a randomized control trial, the use of a mixed mode survey, given the retrospective nature of the project, proved to be a valuable tool in the assessment process.