Original version written by Sophie Culpepper and published on the Nieman Lab website on June 11, 2025
« What is the right metric for measuring a news outlet's success?» Several local media managers, mostly from small newsrooms, debated this question during a panel discussion organized as part of the annual conference of theInstitute for Nonprofit News. The discussion covered both nonprofit organization communications and journalistic missions, and specifically targeted small media outlets that don't make audience size their primary success metric.
The speakers detailed their approaches to conceiving, measuring, and presenting their successes to their communities and funders. While their methods differ, they share a common approach: rather than measuring the reach or performance of an article (page views, awards won, etc.), they evaluate their success based on the concrete impact on the populations they aim to serve. One perspective among many in a broad and complex debate around evaluating the success and impact of nonprofit media.
"Ultimately, our goal isn't to save journalism," explains Angilee Shah, editor-in-chief and executive director of Charlottesville Tomorrow. "Our goal is to create something useful and essential for our community."
"We think too much emphasis has been placed on growth indicators and quantitative measures," adds Amy Kovac-Ashley, executive director of Tiny News Collective, "and not enough on what we call impact indicators." While this roundtable focused on the latter, she hoped participants would leave seeing these two types of indicators as complementary, "like two inseparable flavors in the United States: peanut butter and jelly."
Charlottesville Tomorrow
The territory covered by Charlottesville Tomorrow, in central Virginia, has approximately 400,000 residents, notes Angilee Shah. A figure that helps put the media's quantitative data in perspective. Four hundred people participating in a survey about local candidates may seem modest, but it's actually remarkable when the electoral district has only 700 registered voters.
Charlottesville Tomorrow prioritizes monitoring feedback from its community. The news outlet directs readers toward a contact form and regularly organizes individual interviews. The newsroom receives approximately 130 messages per day from its community and has developed a Slack system that facilitates sharing feedback among journalists to feed its impact measurement tool. A reporter specializing in community relations then assigns each piece of feedback a rating on a five-point scale:
- 1 point: "Glad you're here" (general thank you message to Charlottesville Tomorrow or for a specific article).
- 2 points: Reported impact on a person.
- 3 points: The person took action as a result of the article.
- 4 points: The person took action and mobilized others.
- 5 points: A concrete change occurred in the community (new law, modification of a real estate project, revision of municipal budget).
Charlottesville Tomorrow is currently developing an internal dashboard that will highlight impact scores rather than simple metrics like page views.
Spending your time on Google Analytics, does it measure the value your community places on your work or simply your ability to generate web traffic?
Understanding what metrics truly represent is crucial, Angilee Shah emphasizes. Spending time on Google Analytics—does it measure the value your community places on your work, or simply your ability to generate web traffic? One can influence the other, but
When a funder asks for data that doesn't align with the news organization's mission, Angilee Shah provides it while making clear upfront that this isn't how Charlottesville Tomorrow measures success.
Baltimore Beat
The Baltimore Beat, a nonprofit news outlet that carries on "the tradition of Black press and the spirit of alternative weekly journalism", operates with even tighter constraints than Charlottesville Tomorrow. The newsroom has four full-time staff members, a new Report For America fellow (a national program that places young journalists in local newsrooms to cover underreported topics), and will host an RFA journalist this summer, according to editor-in-chief and co-founder Lisa Snowden.
Partnerships have always been at the heart of Beat's strategy. The most fruitful one connects the media with Wide Angle Youth Media, which introduces 14-24 year-olds to careers in the arts. Young people passionate about design become the "de facto designers" of the Beat for its twice-weekly print edition. These same young people also contributed to coverage of the Freddie Gray case (a young African American killed by police in 2015) and collaborated on two electoral guides aimed at young voters last year.
These collaborations extend beyond the media world. A local brewery recently launched a beer called "The Beat Goes On" ("with notes of blackberry and basil"); one dollar per sale goes to the Beat. Earlier this year, as the 10th anniversary of Freddie Gray's death approached, the Beat partnered with the Baltimore Museum of Art to organize citizen forums and a photography exhibition.
In 2020, new partnerships and additional funding enabled the Beat to create its "Beat boxes," a nod to the iconic bright yellow metal boxes of the Baltimore City Paper. These wooden boxes (cheaper than metal) do more than distribute the print edition of the Baltimore Beat: they also serve as community exchange points for practical resources like snacks, Covid tests, and drug screening tests. Residents have "adopted" Beat boxes to keep them stocked, and the Beat is considering how to expand this initiative.
The media fully embraces its positioning: "We clearly state who we are and what we do, without trying to be all things to all people," sums up Lisa Snowden.
Invisible Institute
Based in Chicago, the Invisible Institute (twice Pulitzer Prize winner last year) is primarily dedicated to investigating police violence. Its roots go back to the 1990s, when its founder Jamie Kalven covered the end of large public housing complexes and worked to support residents, whether filing complaints about broken elevators or reporting police abuse.
"In our reports to foundations, we often explain that we measure our impact by evaluating our influence on public policy and practices," details Maira Khwaja, director of public strategy at the Invisible Institute.
For the structure, creating lasting impact in Chicago requires "long-term support work," helping residents obtain justice, particularly by directing them toward the legal resources of other organizations.
The institute designs its action as "an integral part of a broader network of lawyers, activists and people directly affected," clarifies Maira Khwaja.
She highlights two particularly impactful investigations by the institute. Since the publication in 2016 of an investigation into drug trafficking and police corruption, over 212 convictions have been overturned, thanks to lasting relationships built with the investigation's participants and legal support from organizations like The Exoneration Project. A 2020 investigation into a wrongful conviction brought new evidence in the Robert Johnson case, who was freed a few months ago.
Conversely, an investigation into the death of Harith Augustus, killed by police in 2018, certainly sparked debate and media attention and inspired an Oscar-nominated documentary, but had no
She acknowledges that growth indicators, debates, and events can matter, but the institute measures its impact differently: "Does this change how police or oversight bodies behave? And concretely, do the people most affected actually feel a difference? It's not just about changing the text on paper."
However, Maira Khwaja acknowledges that
Open Campus
Open Campus is a national media outlet that partners with 16 local newsrooms to provide them with reporting specialized in higher education. A real need: most people study within an 80-kilometer radius of their home, yet few local newsrooms have a journalist dedicated to this sector, notes Colleen Murphy, editorial director of the local network.
In early 2025, the media outlet completely redesigned its impact measurement system. Previously, a Slack channel was dedicated to this tracking, but it filled up with press mentions and compliments, "which doesn't equate to impact," observes Colleen Murphy. As part of this overhaul, Open Campus renamed that channel "kudos" and created a separate "formal" impact tracking system.
Now, Open Campus uses Airtable to measure concrete impact at individual, community, and institutional levels, drawing on Impact Architects' methodology.
Journalists fill out brief evaluation forms at least once a month (ideally as soon as a significant event occurs). Colleen Murphy reviews these data, often adds complementary information, and transforms them into narratives. Beyond the annual impact report, she compiles quarterly summaries for the team, funders, partner newsrooms, and the public, and produces localized impact reports for each partner.
Although Open Campus continues to track press mentions and article pickups, these indicators are no longer a priority. They count "for visibility," Colleen Murphy explains, but "don't really reflect engagement or our reach in the communities".
Colleen Murphy cites three examples of the kind of impact Open Campus is proud of. A journalist at WBEZ Chicago told the story of a young woman who was struggling to repay her student loans; listeners and readers of that story spontaneously paid the remaining $10,000. At Mirror Indy, after a journalist investigated how the construction of Indiana University's campus in Indianapolis displaced generations of Black families, former neighbors reconnected on social media. And when a Texas Tribune journalist profiled a young mother juggling childcare, night shifts at a laundromat, and full-time university studies, readers asked how they could help her financially. "How do your stories strengthen empathy and build social connection in your communities?" Colleen Murphy asks.
