When you shop for a new refrigerator or pair of shoes, where do you look for information about products? Do you log onto Amazon and read reviews? Check Consumer Reports for lab-verified results? Consult your neighbor or mom?

Now imagine you live on less than $2 a day as nearly 2.7 billion people around the world do. You’ve heard that filtering your water is important for your family’s health, but there are over 100 models of water filters on the market and you have little or no access to information about what works and what doesn’t. Which product would you choose?

That’s reality for many of the families interviewed by Massachusetts Institute of Technology (MIT) researchers as part of a product evaluation in Ahmedabad, India, last summer. Last week, my colleagues and I released a new report evaluating household water filters on the market in Ahmedabad, where they have become ubiquitous in households of all income levels but aren’t properly meeting the needs of the poor.

The report details the second experimental evaluation designed and implemented by the Comprehensive Initiative on Technology Evaluation (CITE), a program supported by the U.S. Agency for International Development and led by a multidisciplinary team of faculty, staff, and students at MIT.

‘Consumer Reports’ for Global Development

Technologies like water filters, solar lanterns, and grain silos are increasingly heralded as simple, cost-effective solutions to development challenges like unsafe drinking water, energy access, and crop loss. But technology alone won’t be enough to bring equity and access to impoverished communities around the globe.

In 2013 Dennis Whittle, the co-founder and now director of Feedback Labs, a member organization dedicated to “closing the feedback loop” in aid, posed a question: What if there was a Consumer Reports for global development?

“What if governments – and even regular citizens – had a similar tool to use in development projects? Or better yet – why don’t they?” wrote Dennis in an article for the Center for Global Development. “When I went on my first mission to Indonesia in 1987 as a young economist at the World Bank, I was struck by the fact that there was almost no way for farmers to tell us ‘experts’ what types of agriculture projects they wanted.”

That’s the driving idea behind CITE, the first-ever program dedicated to developing methods for product evaluation in global development. CITE researchers don’t only evaluate the technical performance of products, we also evaluate a product’s “scalability,” or how well its supply chain reaches consumers, as well as its “sustainability,” if a product can be used correctly, consistently, and continuously over time.

While many of CITE’s evaluation efforts focus on technologies and products, we are also looking at the wider ecosystem around those products. A recent CITE evaluation focused on understanding the effect of a World Food Program initiative to bring low-cost post-harvest storage silos and bins to farmers in Uganda. Another examines the effectiveness of training local water committees to use hydrogen sulfide test kits to assess household water quality in India.

Ineffective at the Bottom, Wasteful at the Top

CITE’s new report looks closely at the booming household water filter market in Ahmedabad, India.

According to the United Nations, about 90 percent of the world’s population now has access to “improved” drinking water. But improved water isn’t always safe water. Water quality is often compromised by storage in contaminated vessels or other unhygienic handling practices in the home, and safe piped water can be disrupted by repairs, leaks, or pump failures and electrical outages. In such cases, families concerned about quality often use household filters as a stopgap solution.

In Ahmedabad, we found the poor using low-cost ($0.50-$1.00), widely available conventional particle filters – usually a piece of sari cloth or jali mesh. Unfortunately, lab tests show these filters are largely ineffective at removing harmful bacteria like E.coli from drinking water.

This poses a major challenge. Ahmedabad’s poor are acutely aware of the need to filter their water, but lack a suitable, affordable option to do so. CITE’s research shows that despite the flood of filters on the market, there is a major gap in the market between low-cost but largely ineffective cloth and mesh filters, and more effective but higher-priced gravity and reverse osmosis filters.

“It is critical that products recommended and marketed to this population are actually efficacious at removing the organisms that cause diarrheal disease,” says Daniele Lantagne, assistant professor at Tufts University Department of Civil and Environmental Engineering.

The World Health Organization has developed a household water treatment certification scheme to confirm the efficacy of such products, but it’s not always used.

“The work completed by CITE is an important complement to this international scheme,” says Lantagne, “expanding it to account for local circumstances in India, to look at all products in the market, and to also investigate sustainability and scalability concerns.”

Our results beg the question, however: Is a culturally appropriate, low-cost, and effective water filter possible for this poorer population? The current market outlook is not good.

There are also issues at the high end of the market. The more expensive reverse osmosis filters CITE studied are growing in popularity among middle-income consumers, but waste about three times as much water as they produce. This should be cause for concern in a region already struggling with water scarcity.