Participatory Evaluation for Researchers -- Super Team Guide | Issue 2 of 12
Who's This For
You work across institutional boundaries. Your partnership involves a research university, community colleges, community organizations, maybe a tribal education office or an industry partner. You write the evaluation plan, and you know -- even as you write it -- that you are defining success through your own lens. Publications. Student placements. Conference presentations. These are real outcomes, and they matter. But somewhere in the back of your mind, you suspect your partners would tell a different story about what success looks like. You are right. And that gap between your success definition and theirs is not a flaw in your partnership. It is the most important design feature you have not yet leveraged.
The Partnership Moment
You are working with a community organization on a STEM outreach program targeting middle school students in an underserved neighborhood. Your university team defined success the way most researchers would: increased science content knowledge, improved attitudes toward STEM careers, higher rates of students taking advanced science courses in high school. Standard metrics, all defensible in a proposal.
Then you sit down with the director of the neighborhood youth center and ask what success would look like from her perspective.
She says: "I need these kids to come back on Thursdays."
That is it. Retention.
At first, it feels too simple. Shouldn't you be measuring learning outcomes, attitude shifts, career aspirations? But as you listen more carefully, you begin to understand what she is really telling you.
Her organization struggles with consistent youth attendance. Programs that cannot retain participants are programs that cannot produce any impact at all. She has years of experience watching well-intentioned university programs arrive with elaborate curricula, lose half the students by midpoint, and then report "results" based on the self-selected few who stayed.
"I need these kids to come back on Thursdays" is not a rejection of your learning objectives. It is a precondition for any learning objectives to matter at all. She is telling you that sustainable engagement is the foundation everything else has to build on.
Under the Surface
This gap between university-defined and partner-defined success is not unique to your partnership. It shows up everywhere cross-institutional STEM collaborations exist. And it matters far more than most evaluation plans acknowledge.
When you define success unilaterally, several things happen. Partners comply with your metrics because you hold the grant funding, but they do not own the evaluation. They fill out your surveys, attend your reporting meetings, and provide the data you ask for. But they are performing accountability, not practicing evaluation. The data you collect reflects your questions, not theirs.
More importantly, you miss the indicators that would tell you whether the partnership is actually working for everyone involved. A community partner who sees retention as the foundational metric is telling you something critical about program sustainability. If students do not come back, your carefully designed learning assessments are measuring a biased sample. If the community organization cannot demonstrate value to their board based on your metrics, the partnership has an expiration date.
Partner-defined success is not a concession or a compromise. It is intelligence. When you understand what each partner actually needs from the collaboration, you can design a program that delivers for everyone -- and an evaluation that proves it.
The challenge is that different partners will define success differently. And sometimes those definitions will seem to pull in opposite directions.
