Skip to content

More Than Good Intentions (Karlan and Appel): “Back to School” Book Report by Carol McLaughlin

October 20, 2011

Carol McLaughlin

This is the sixth in a series of “Back to School” book reports by members of our team. We selected books based on their potential to help us in our own work to identify high-impact philanthropic opportunities and help donors improve their philanthropic impact. We hope this series helps make some of our own learning transparent so that others may benefit. This week’s “Back to School” book report is by Carol McLaughlin, research director for global public health.

“How can we act with more than good intentions?

How can we find the best solutions?”

Karlan and Appel’s More Than Good Intentions is written for donors—particularly donors who care about alleviating global poverty and maximizing their impact doing so. The book’s evidence-based approach to answering the question of “what works” provides donors with the confidence they need to know their money is actually doing good.

To start, authors Karlan and Appel make the case (and we whole-heartedly agree) that individual donors are critically important: “contributing over $200 billion to charity each year…exceeding those of corporations, foundations, and bequests by three to one.” As a result, it is critical that individual donors give to efforts that can break the cycle of poverty for poor families. The authors provide a two-pronged approach to this challenge:

  1. Donors should first understand the problems that poor people face.
  2. Then, rigorously evaluate competing solutions.

But how can donors figure out what actually works? For Karlan and Appel the answer is randomized controlled trials (RCTs) that can measure a program’s impact. Simply said, this method measures how people’s lives changed with the program, compared to how they would have fared without it. By using real world case examples, the book makes abstract concepts such as control groups and randomization easily understandable for audiences unfamiliar with this methodology. Through captivating chapters with vivid on-the-ground stories from Ghana, the Philippines, and Peru, the authors successfully highlight how RCT methods can be applied to a broad set of questions in international development. These include everything from improving microcredit to raising school attendance or increasing use of bednets to prevent malaria.

Beyond providing answers to the question of what works—and what doesn’t—the authors take an additional step by linking considerations of cost and impact. They answer the question that we believe to be critical in donor decision making: “For each dollar donated, how much good can be done?” For example, if the goal is to decrease absenteeism in rural schools, their results found that the cost of one year of increased school attendance was $1000 using conditional cash transfers (paying families to keep children in school), compared to $100 using uniform give-away programs compared to $3.50 using deworming programs.

The final chapter nicely summarizes the lessons learned and how to apply these key take aways to donor decision making. From the RCTs conducted by Innovations for Poverty Action and J-PAL (MIT’s Poverty Action Lab), they highlight seven interventions with a strong evidence base, which should give donors confidence to take informed action.

What is missing from the book is a discussion of the limitations of RCTs and the complementary methods that can be used. Donors reading the book may walk away believing that RCTs are the only acceptable evaluation method—which is not the case—depending on the question at hand. For example, even in the field of medicine where RCTs are considered the gold standard to test new drugs and interventions, the research community acknowledges that qualitative methods (e.g., focus groups), quasi-experimental designs, and other mixed methods play a critical role. In our work at the Center for High Impact Philanthropy, we have found these other methods of evaluation to be quite helpful particularly in assessing adaptation and delivery of proven interventions to different local contexts or when RCTs are not feasible due to practicality, expense, or organizational capacity. Placing RCTs in the broader context of measuring and managing to impact is especially important as donors and implementing organizations need to prioritize what questions are best addressed by RCTs versus when to put precious time and resources into other evaluation methods that better match the situation at hand.

Overall, More Than Good Intentions rightly emphasizes the key questions that impact-focused donors should be asking:

  • What works?
  • What doesn’t work?
  • How much does change cost?

Hopefully there will be many sequels to follow packed with similar evidence based solutions for donors who want to address global poverty.

“How much more good could we do in the world if impact-informed giving came to be seen as the coolest kind of all?

4 Comments leave one →
  1. October 20, 2011 4:04 pm

    We can all agree that aid can be more effective and that well-formed questions and well-executed, applied research can offer many relevant clues about this. We all want to see deeper thinking behind the doing. Where I differ with Karlan & Appel is on some fundamental beliefs about what prevents this and what ails philanthropy and the aid industry overall. Is it a lack of information about “what works”? Or is it a lack of respect for local initiatives and understanding about complex power dynamics that impede authentic relationships among development partners? And if it’s the latter, are RCTs just a band-aid on a deeper issue?
    You can read my review of the book at:
    http://www.how-matters.org/2011/05/24/rcts-band-aid-on-deeper-issue/
    and my guidelines for donors to consider about RCTs:
    http://www.how-matters.org/2011/05/25/rcts-how-matters-advice-for-donors/

  2. February 21, 2012 3:23 am

    “How much does change cost?” – sometimes I see charity programs which cost too much. This manual helped out team find efficient means of financing our group in helping the less fortunate.

Trackbacks

  1. News and Events: Links to what we’re reading « High Impact Philanthropy
  2. It Ain’t What You Give, It’s the Way That You Give It: Q & A with Caroline Fiennes « High Impact Philanthropy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: