AWATS Contest

Due to the overwhelming show of support for A Walk Across the Sun from today’s email, we will be limiting the free book give away to the first fifty.

Thanks for everyone’s support, and please keep voting!

A Walk Across the Sun: Finals

March Madness ends tonight with the men’s NCAA basketball finals. Harper Collin’s March Madness finals continue this week. Thanks to many of you, Addison’s book, ‘A Walk Across the Sun’ is in!

For those of you on my mailing list who vote this week, I will send you a free Kindle copy of his book if he wins! Simply email info [at] ndpmetrics {dot} com with your information after you vote. You can vote every hour.

Please vote for it this week when you can:

A Walk Across the Sun

Addison’s book is in the Final Four in Harper’s March Madness: ‘A Walk Across The Sun.’ He is the only living author on the list!

As I’ve mentioned before, this is a great book I highly recommend buying. It is on the tough topic of child sex trafficking, written in a beautifully redemptive way. Please click and vote for it and ‘Where the Wild Things Are.’ You can vote every hour:


This is a great book I highly recommend buying. Addison does a wonderful job of writing about the tough topic of child sex trafficking. It is a true thriller, written in a beautifully redemptive way. Addison says he wrote it so that “grandmothers and 17 year old girls could read it.”

Vote for A Walk Across The Sun in in Harper’s March Madness. Please click and vote for it. You can vote every hour!

HIV Educational Impact Assessment: Objectively Measuring Education with IRT

This video shows how New Dominion Philanthropy Metrics created a measurement scale to assess the impact of HIV educational interventions carried in Narobi Kenya. Item Response Theory was used to create and refine this scale to measure HIV knowledge, attitudes and practices of beneificaries. This methodology can be applied to various sectors to create reproducible, objective measurement of philanthropic projects.

Crowdsourcing vs Experts: Assessing Technological Approaches to Conflict Monitoring

By Mara J. Roberts

This piece was originally published on the Peacebuilder blog

Not too long ago, I met up with a friend of mine who used to be an analyst for the State Department. One of the hot topics of our dialogue included the subject of peacebuilding efforts in Africa. During the conversation, he recounted a story about a project he was involved with where he was tasked with helping evaluate a peacebuilding project in Northern Uganda. The project involved building a peace center and culminated in hosting two peace conferences where the opposing parties were both brought to the table. The recipients of the money were very proud of their achievement, however when my friend asked them this question: “What difference have these conferences made to decrease conflict in the area,” they didn’t know the answer. This vignette highlights the difference between outputs (hosting a peace conference) and outcomes (a decrease in violence) and the particular complexities of trying to monitor and evaluate conflict.

My last post was about Ushahidi and its novel approach to monitoring conflict through the combination of SMS, Twitter and geo-mapping. At the center of the Ushahidi methodology for crisis monitoring is crowdsourcing: the use of the general public’s knowledge or opinion to provide information. Most of us benefit from crowdsourcing every day. When we shop on Amazon, we look at an item’s reviews. When we rent a movie on iTunes and Netflix, we look at its ratings. These both use crowdsourcing methods.

Weaknesses of a Ushahidi: Limits to Access
While there is great innovation in harnessing the general public’s information through crowdsourcing via SMS and Twitter, these methods have their drawbacks. First, Ushahidi utilizes a small army of volunteer computer programmers to implement a specific version of the map when a new crisis arises. The code is “open source” meaning that anyone can have access to it, but unless you are a programmer or know programmers, this is really of no use. Having the code open sourced also allows anyone to create their own customized version of the crisis map. In the recent uprising in Egypt, there were five different versions of crisis maps created by several different factions.

Another significant weakness to the method is that you do not know who is providing the information. Because of this, there could be disinformation sent. Ushahidi attempts to guard against this through validating both users and content. Perhaps Ushahidi’s greatest weakness is its dependence upon the general public’s access to cell phones and internet. Countries in Africa with the highest cell phone penetration include South Africa (85%), Egypt (70%), and Kenya (50%). The people who live in poverty typically are most susceptible to violent acts and marginalization and can rarely afford a cell phone.

An alternative to Ushahidi: Experts and SMS
A less expensive, lower-tech alternative to Ushahidi is using experts to relay information using SMS texting. This is what my company did in the 2010 Burundi presidential election to monitor the post-election violence. In Burundi, primary schools function as community centers where information is exchanged and where networking occurs. Because of this, primary school teachers were used as “experts” and were sent questions through SMS, asking about any election-related violence they saw or heard about from credible sources in their networks. The results of this surveillance indicated a low level of violence related to the election, which was validated through various media reports coming out of the country. This approach avoided many of the weaknesses of Ushahidi, especially given Burundi’s low cell phone penetration rate of only 10%.

Despite the challenges of monitoring and evaluating peacebuilding efforts, practitioners in the field have increasing access to a variety of tools that can strengthen the capacity to better evaluate their work and leverage their impact. As these monitoring and evaluation tools begin to be used more effectively, stories like the one in Uganda will hopefully occur less frequently.

Acknowledgements: David Roberts, who oversaw the technical details of the 2010 Burundi election violence surveillance, contributed to this article.

The Next Generation Conflict Monitoring System: People and Technology

By Mara J. Roberts

This piece was originally published on the Peacebuilder blog

Reports were trickling in from our friend and coworker living in Kenya that the situation was rapidly deteriorating. After what appeared to be a stolen presidential election on December 27, 2007, the resulting violence seemed to be drawn along tribal lines. The 100 year old mission compound where Esau and his family lived seemed to always be insulated from national crises like this. But on January 9, my husband got the email. A flyer, causing mass fear as it flew over the base declared the following threats: “We swear by the sacred Mugumo tree that when we descend upon Kijabe, we shall not leave any Luo alive …” Afraid for their safety, Esau (a Luo) and his family moved to a friend’s house away from Kijabe to live in hiding for several weeks. At the end of the 59 day political crisis, the sobering death totals throughout the country were approximately 1,500 dead, 3,000 innocent women raped, and 300,000 people internally displaced.

Patrick Meier had grown up in Kenya and happened to be back visiting his parents during the crisis. Articles with headlines like: “Kenya Is Not Burning,” were surfacing from the Kenyan government media outlets, who were trying to manage the national and international opinion through disinformation practices. He read these and wanted to do something to improve the ability to report human rights abuses resulting from the conflict. He enlisted the help of some Kenyan nationals to assist in realizing his idea. They called it Ushahidi (pronounced OO sha hēdē), which means “testimony” in Swahili. The system was simple, but novel in that it integrated Google Maps with Short Message Service (SMS) otherwise known as text messaging. It employed the idea of crowdsourcing, which harnesses eyewitness accounts of acts of violence from the general public. Ushahidi allowed people to send an SMS to a free, local number with their location and a description of what they saw.

Kenya post-election violence crisis map

The results of the experiment were extremely successful: several hundred texts were received with first hand reports of riots, death, property loss, sexual assaults as well as peace efforts throughout the country. Ushahidi had provided a method for creating real-time mapping that documented acts of violence and peacbuilding using common people. The system also provided a warning system, where people could automatically receive a text message if an alert was posted geographically close to a self-selected location.

This prototype system was modified to monitor election violence during the presidential election in Burundi in July of 2010, as well as Kenya’s constitutional referendum in August, 2010. In both of these cases, the system produced reports of little to no violence resulting from the elections.

Ushahidi was used most recently during the Arab Spring protests in Egypt and is currently monitoring the conflict in Libya. With the high penetration of the internet in cities like Cairo and Alexandria, the deployment in Egypt provided the opportunity to also incorporate Twitter tweets (in addition to SMS) into the conflict map.

Overall, this system, which simply combined multiple existing technologies, yielded a powerful conflict monitoring tool. This tool, however, is not void of weaknesses. In my next entry, I will discuss some of these weaknesses as well as a simple system alternative for monitoring election violence.

Most Popular Philanthropy Metrics Stories of 2010

6. Our Philanthropy Metrics team traveled to Burundi in June between elections there to perform a country wide survey examining the secondary impact of a three year neglected tropical disease project.

5. In December, Transpanency International published it 2010 Global Corruption Barometer Report which indicated that ninety two percent of Kenyans believe that the police force is the most corrupt institution in Kenya. We looked into this survey methodology to report some weaknesses in the study that likely bias the results towards those living in urban areas.

4. In April and May, we had a two part article on the importance of monitoring and evaluation. Philanthropies often focus on setting goals, but do little to measure if goals are met, leading to a lack of data and a crisis in writing the final report. In the first article, we provided options for those who find themselves without sufficient data upon completion of their project. The second article provides both basic and more advanced options for those who want to do M&E correctly.

3. In January, the world was drawn to a small island off the coast of Florida. The earthquake that rocked Haiti led to a herculean search and rescue (SAR) effort from the international community. Unfortunately, the benefit of human lives saved was small while the cost was massive, leading to an article questioning the rationale for large SAR operations.

2. The article on The Right and Wrong Way to Survey highlighted the fact that surveys are harder to do well than most people think and are often designed poorly, leading to faulty data that can lead to erroneous decisions being made. We provided a step-by-step guide for you to get started in doing your own survey.

1. In February, we were on a conference call with Sean Stannard-Stockton of Tactical Philanthropy hosted by Philanthropy Action looking at the role of qualitative metrics in good philanthropic practices. The highlight of the call was the need for a more blended approach by using both quantitative and qualitative tools for decision-making.

Photos from the field: Kenya Survey

Philanthropy Metrics is focused on creating a tool that measures the impact of HIV educational programs.  This is one focal area for us because of the lack of efficacious questionnaires out there.   Over the past year, we have performed several pilot studies where our field team has implemented several iterations of a questionnaire.  In 2011, our plan is to role out a beta version in Swahili which can run on both desktop and mobile devices.

Kenya Survey Finds Police Corruption

A recent survey in Kenya reported in Transparency International’s 2010 Global Corruption Barometer Report yesterday indicated that ninety two percent of Kenyans believe that the police force is the most corrupt institution in Kenya.  The index which has similar ratings from around the globe, put the police at 4.6 on a scale of one to five – with five being most corrupt.  It also reported that 45 percent of Kenyans had given a bribe to a public official in order to access services.

I found the results of the survey interesting, and performing surveys myself, I looked at their survey methodology.  The survey, which was performed by Synovate, had a sample size of  1,000 Kenyans, which is substantially enough to obtain a significant result assuming that the population was randomly selected and representative of the population.  The sample, however, was performed by computer-assisted telephone interviewing (CATI).  While an excellent tool in developed countries where most people have telephones, its use in Kenya puts into question the sample’s ability to represent the population since many Kenyans do not personally have a phone.

So instead of saying the sample is representative of the general population, Transparency International should say that it is representative of the population in Kenya with telephones.