Learning From Failure

AMERICANS love success stories. Go to the Web sites of the United States Agency for International Development, the Bill and Melinda Gates Foundation or a plethora of global health and development organizations, and you’ll find articles, charts and videos documenting their triumphs and innovations, with the promise of more on the way.

Beyond simply doing good, there’s an impetus to show success: nongovernmental organizations, contractors and researchers want a good track record, funding officials must show they are spending wisely, and journal editors want to highlight breakthroughs.

But “success stories” are rarely the whole story. Global health and development projects frequently go off course, and it’s not unusual for them to fail outright. What is unusual is for researchers to openly discuss their failures. That’s a shame, because it’s a basic principle of science that you get things right by analyzing what went wrong.

So it was a pleasant surprise when, last summer, researchers at Mumbai’s City Initiative for Newborn Health published, in the journal PLoS Medicine, the disappointing results of their three-year effort to implement a community-based maternal- and infant-health initiative in the city’s slums.

It was a textbook global health project: community-based, empowering the beneficiaries, focusing on women and children. And the model was successful elsewhere, reducing newborn mortality rates in other parts of India, as well as Bolivia and Nepal, and by 30 to 40 percent.

But those programs were in rural areas. This was the first time the effort was tried in an urban setting.

It turned out to be a major hurdle. While urban areas have the advantage of more infrastructure and health services, there are also unexpected variables and unanticipated dynamics. Much of the global health boom since the 1990s has focused on rural areas; far less work has been done on urban programs, in part because those situations are difficult.

And Mumbai is not just any big city. More than half its nearly 13 million people are jammed into the concrete and corrugated-metal mazes of the city’s slums. In 2005, the researchers began gathering data in 48 slum communities. They set up 244 women’s groups, which met biweekly with social workers to discuss maternal and child health problems, counsel one another and work to develop solutions.

While that worked in rural villages, in the city slums, mothers were hesitant to depend on neighbors. At some meetings, residents from one street would not sit with hosts from a block away.

The researchers tried using innovative approaches to engage the women. In one case, they had them play a game, using cards that presented various social and health problems. The women then devised their own solutions within the game. But such prompting didn’t lead to substantial action back in the real world.

Measuring improvement in rural infant health is also easier than in urban areas. In the former, the situation is often so dire that almost any well-devised intervention will bring substantial improvements. In cities, people have access to urgent care through public hospitals. It is more in the pre-emergency period, before it becomes simply a matter of life or death, where change is needed.

Yet mothers in slums see issues like stunting and chronic disease as part of life and think “that’s how children will grow up in this environment,” said Neena Shah More, a researcher with the Society for Nutrition, Education and Health Action, a member of the consortium behind the project. Instead, water and sanitation were top priorities. But those long-term issues were beyond the project’s scope.

Last year they rebooted. They set up small centers that offer basic health services like immunization, feeding, family planning and help navigating the city’s convoluted health and social service systems. So far, providing concrete services, rather than just advice on collective organizing, seems to be more in tune with the needs of people in the slums.

The travails of the Newborn Health project aren’t unique. What is noteworthy is that when the project did not work as planned, the team reported it openly and in detail, providing potentially valuable information for other researchers.

The risk is that too few people will follow. Especially in tough economic times, the pressure is on to show that they are getting bang for their buck. Last year an Obama administration official called on the aid community to adopt a “permanent campaign mind-set,” in which fund-raising and promotion are on the front burner. This creates an incentive to go for easy victories, highlight successes and bury failures. Even with the new fad in the aid world for metrics and impact assessments, their public reports are rarely forthcoming about missteps.

That’s bad science. While aid organizations must be accountable for outcomes, that pressure for positive results should not be an encouragement to skimp on the truth. Making a difference in the world is hard, often messy work. Pretending otherwise is no help at all.

Sam Loewenberg was a 2012 Nieman Foundation global health reporting fellow at Harvard. Financial support for this article was provided by the Pulitzer Center on Crisis Reporting.

A version of this op-ed appeared in print on February 3, 2013, on page SR12 of the New York edition with the headline: Learning From Failure.