Participatory Evaluation for Researchers -- Super Team Guide | Issue 4 of 12
Who's This For
You have been collecting stories. Maybe five, maybe twenty, maybe more. Each one is rich, specific, and compelling on its own. But now you are sitting with a folder full of individual narratives and a reporting deadline approaching. Your program officer expects patterns. Your annual report needs findings, not just anecdotes. You know there is something in these stories -- recurring dynamics, shared experiences, common turning points -- but you need a systematic way to surface those patterns and present them as credible evidence. You need to move from "here are some stories" to "here is what our stories tell us." This issue shows you how.
The Partnership Moment
You are leading a multi-institution STEM partnership program. Over the course of Year 1, you collected stories from participants at three points: the fall kickoff convening, a mid-year virtual check-in, and the spring showcase. At each point, you used the same simple prompt: "Tell us about a moment in this program that mattered to you."
By year's end, you have about 120 stories. Not survey responses. Actual narrative accounts of specific moments -- the kind of data you learned to capture in Issue 3.
Now it is January. The annual report is due in six weeks. And you are staring at 120 stories, each one unique, each one valuable, and you have no idea how to turn them into the kind of findings NSF expects.
You could cherry-pick three or four of the most compelling stories and include them as "illustrative examples." That is what most PIs do. But you know it is not enough. Cherry-picked stories raise the obvious question: are these representative, or are they the exceptions? Reviewers want to know whether your program consistently produces impact, not whether it occasionally produces a good anecdote.
What you need is a method for moving from individual stories to collective evidence -- a way to honor the specificity of each narrative while identifying the patterns that emerge across all of them.
Under the Surface
The challenge of pattern recognition in story-based evaluation is real, but it is not new. Qualitative researchers have been developing methods for analyzing narrative data for decades. The problem is that most of those methods were designed for academic research contexts -- dissertations, journal articles, multi-year studies with dedicated analysis teams. They were not designed for a PI who needs to analyze 120 stories alongside teaching three courses and managing a grant budget.
What you need is a streamlined version of these methods -- rigorous enough to produce defensible findings, practical enough to complete in the time you actually have. The four-step process described in this issue draws from established qualitative research methodology but adapts it for the realities of grant-funded partnership programs.
Here is what makes this important beyond the practical: pattern recognition is where story-based evaluation becomes genuinely powerful. Individual stories show that impact happened. Patterns show how it happened -- the mechanisms, the conditions, the dynamics that consistently produce outcomes. And patterns are what allow you to improve your program, not just document it. When you see that the same dynamic appears across fifteen stories from different institutions and different participant types, you have found something you can design for.
Patterns also address the credibility concern head-on. When you report that a theme appeared in 78 percent of stories collected from participants across five institutions, you are making a quantitative claim backed by qualitative depth. That combination is more compelling than either method alone.
