Participatory Evaluation for Researchers -- Super Team Guide | Issue 3 of 12
Who's This For
You have heard that stories matter for evaluation. Maybe you have even collected a few -- a compelling quote from a participant, an anecdote shared at a meeting, a moment someone described in a final report. But you are not sure how stories become data. You worry that reviewers will dismiss narrative evidence as "soft" or "anecdotal." You want to use stories in your evaluation, but you need a systematic way to capture them -- something structured enough to produce defensible findings, yet open enough to let participants tell you what actually mattered. You need a protocol, not a wish. This issue gives you one.
The Partnership Moment
You are at a cross-institutional convening. About fifty participants from different universities, community colleges, tribal education offices, and nonprofit organizations are presenting their broader impacts work in a poster session. The gallery walk format is familiar -- presenters stand by their posters, visitors circulate, conversations happen.
During the session, you notice something. A community college biology instructor is building her poster -- a visual map of her institution's assets. She is listing not just lab equipment and funding, but community connections, student populations, local employer relationships, unique regional contexts. As she works, something shifts. She pauses. She realizes that her college's location adjacent to a major agricultural processing corridor gives her students access to environmental monitoring questions that no R1 university could easily study. A researcher from a partner institution walks by, stops at her poster, and says, "We have been looking for exactly this kind of field site."
That moment -- the pause, the recognition, the connection -- is the kind of thing that transforms programs. It is also the kind of thing that evaporates if no one captures it.
A satisfaction survey sent three weeks later will not find this moment. A checkbox on a registration form will not record it. But if you have a story capture protocol built into the activity itself, this moment becomes data. Powerful, specific, defensible data that shows how your program creates impact.
Under the Surface
The reason stories matter for evaluation is not sentimental. It is methodological.
Surveys capture what participants can report retrospectively through the filter of predetermined categories. If you did not think to ask about "moments of asset recognition during poster-making activities," that data does not exist. Stories capture what participants experienced -- including things you did not know to look for.
More importantly, stories capture mechanism. A survey can tell you that 85 percent of participants reported increased professional networks after a convening. A story can tell you how that networking actually happened -- through a specific conversation prompted by a specific poster about a specific regional asset. The survey gives you the "what." The story gives you the "how" and the "why."
This matters enormously for NSF reporting and for program improvement. Funders increasingly want to understand not just whether programs produce outcomes, but through what mechanisms. When you can show that a poster-making activity designed around institutional asset mapping led to a cross-institutional research collaboration that produced environmental monitoring data used in a county zoning decision, you are demonstrating a causal pathway that no satisfaction score can match.
But stories only function as data when they are captured systematically. Random anecdotes are not evidence. A structured story capture protocol -- applied consistently across participants, across events, across time -- produces a dataset that can be coded, analyzed, and reported with the same rigor you would apply to any qualitative research.
