Nonprofits Make Gains in Measuring Their Impact

by Hank Sartin

In December, for the 13th year in a row, The Rotary Foundation received a four-star rating — the highest possible — from Charity Navigator, an independent evaluator of charities in the United States. Charity Navigator determines its ratings by examining how nonprofits use their funds, how sustainable their programs are, and how well they follow industry standards of transparency and accountability.

160,000

Nonprofits rated by Charity Navigator
$449 billion

Amount Americans gave to U.S. charities in 2019
47 million

Number of hours Rotary members volunteer annually

In a Matter of Measurement

In recent years, many nonprofits — including Rotary — have begun to increase their focus on measuring the impact of their work, and Charity Navigator has responded to that shift. In 2020, it acquired ImpactMatters, a ratings organization dedicated to assessing the impact of nonprofits. Based on the existing ratings of ImpactMatters, Charity Navigator will create impact ratings for thousands of charities.

In an effort to measure the impact of Foundation grants and other Rotary projects, Rotary is now also emphasizing the importance of incorporating monitoring and evaluation into the projects that Rotary clubs and districts carry out in their own communities and around the world. But measuring impact can be challenging, because there is no one-size-fits-all methodology. For some projects, large-scale surveys are useful. For others, impact might be assessed using economic indicators.

“What we are asking is this: Is this a good use of resources?” explains Elijah Goldberg, vice president of impact ratings at Charity Navigator. “Nonprofits are trying to solve a problem. The question is, is this particular type of activity efficient when you solve that problem?”

Nonprofits are trying to solve a problem. The question is, is this particular type of activity efficient when you solve that problem?”

Elijah Goldberg
vice president of impact ratings at Charity Navigator

The intensified focus on measuring impact is an attempt to apply the principles of academic research to the work of charitable organizations. “In the past 15 years, we have seen a huge expansion of the number of careful, rigorous evaluations being done in academia on impact,” says Dean Karlan, co-founder of ImpactMatters and co-director of the Global Poverty Research Lab at Northwestern University. “These days it is much more practical to do fieldwork collecting data; the internet has radically lowered the cost of doing that. And a lot of the exciting evaluations now come from the nonprofit sector.”

More Robust Planning for Smaller Projects

Assessing impact through carefully constructed surveys and statistical analysis makes sense for large-scale initiatives, but it is often impractical for smaller projects. The good news for clubs is that there are ways to use existing research to demonstrate that a planned project is in line with proven methods for solving a given problem, and then to use those results to guide your program’s impact measurement.

Money can be well spent asking basic questions. How are things going? Are we delivering what we said we would deliver? How do folks feel about it? Would they recommend we make any improvements? Those types of questions can really go a long way.

Randall Blair
associate director at Mathematica, a data-focused policy research firm

“In the formative stages of project planning, you can get surprisingly good information using an internet search,” says Randall Blair, who, as an associate director at Mathematica, a data-focused policy research firm, helps foundations and federal agencies figure out the best types of programs to invest in. “You just have to pick keywords. You can search for the big concept, like ‘educational approaches to adolescent sexual and reproductive health.’ To make the search more effective, add words like ‘recommendations for policymakers.’” It’s important to filter the results you find, he notes, but there is a great deal of valuable data at your fingertips. Research in the early phases can pay off when it is time to assess impact, because you will have more information on benchmarks for your type of project.

And even though he works in a statistics-driven field, Blair emphasizes that for smaller projects, simple surveys and check-in interviews are a valid tool for measuring impact. “Money can be well spent asking basic questions. How are things going? Are we delivering what we said we would deliver? How do folks feel about it? Would they recommend we make any improvements? Those types of questions can really go a long way.”

By documenting the metrics of our good works, we get a clear picture of results. We learn how to replicate and scale up our successes. And we can tell more compelling stories about the good we do.

John Hewko
general secretary of Rotary International

For Rotary, the new emphasis on measuring impact must be part of what we do. “For many years, The Rotary Foundation has attained the highest rating from Charity Navigator. But the gold standard of social impact is changing,” notes John Hewko, general secretary of Rotary International. “The benchmarks of our progress are not just financial health, accountability, and transparency. They are also our ability to measure what we do and prove that our efforts translate into concrete impact, based on clear evidence. To increase our impact and get the credit that we deserve for our hard work, we have to be more data-driven. By documenting the metrics of our good works, we get a clear picture of results. We learn how to replicate and scale up our successes. And we can tell more compelling stories about the good we do.”

• This story originally appeared in the May 2021 issue of Rotary magazine.