Participatory Evaluation for Workforce Development Team Guide | Issue 8 of 8
Who's This For
It is April. You have three reports due. The state workforce office wants quarterly enrollment and placement numbers by the fifteenth. Your institutional research office needs program outcomes for the annual accreditation report by the thirtieth. And the foundation that funded your industry partnership pilot wants a narrative progress report that was technically due last month. You are pulling data from three different spreadsheets, two different offices, and your own memory of conversations that happened in November but were never documented. You know the data exists -- somewhere. You also know that by the time you compile it into reports, the information is too old to inform any decisions. This issue is for program directors managing multi-year grants, evaluators designing reporting cycles, and anyone who has ever felt that reporting consumes the time they should be spending on the program itself.
The Partnership Moment
Dr. M, the program coordinator, sits down to write the mid-year report for her workforce consortium.
Names and scenarios in this guide are composites, drawn from presentations, case studies, professional experience, and evaluation research across multiple programs and regions. Any identified programs or individuals that are featured have opted in to share their experience and program information in support of the community.
She opens three browser tabs: the registrar's enrollment dashboard, the career services placement tracker, and a shared folder where advisory board meeting notes are supposed to live.
The enrollment numbers are current. Career services has placement data through October but not November or December because their reporting cycle runs on a different calendar. The advisory board folder has notes from September's meeting but nothing since -- the November meeting happened, but nobody typed up the notes.
She starts a fourth browser tab: email. She searches for "employer feedback" and finds scattered messages from three different industry partners, each responding to different questions at different times. One says a recent hire "is doing great." Another mentions that "communication during shift changes has been an issue." A third asks when the next cohort will be ready.
These are three genuine data points about program quality. They live in three separate inboxes. They have never been compared, aggregated, or analyzed. And they are about to become a single sentence in a report: "Employer feedback has been positive."
Meanwhile, down the hall, the biotech program coordinator has been tracking the same employer feedback independently -- for a different report to a different funder. She has placement data that overlaps with career services data but uses slightly different definitions of "placement." Her enrollment numbers match the registrar's because they come from the same source, but she tracks retention differently because her funder cares about 180-day retention while the institutional report tracks semester-to-semester persistence.
Same program. Same data. Three different collection systems, three different timelines, three different definitions. The data exists three times but answers the question zero times.
Under the Surface
The reporting problem in workforce development is not a data problem. It is a rhythm problem.
Most programs collect evaluation data on a reporting schedule -- the information gets gathered when the report is due. Quarterly report due? Pull the numbers. Annual report due? Compile the narrative. The data collection serves the deadline, not the program.
This creates three cascading problems.
First, the data is always retrospective. By the time you compile placement numbers for a quarterly report, the placements happened weeks or months ago. The information cannot inform current decisions because it describes a reality that has already passed.
Second, reporting activities are disconnected from program activities. The person writing the report is often not the person who observed the growth, heard the employer feedback, or saw the journeyer's development. Information passes through institutional filters -- from faculty to coordinator to data analyst to report writer -- and each transfer strips away context.
Third, multiple reporting requirements create parallel data streams that never converge. The state workforce office, the institutional research department, the grant program officer, and the advisory board all want information about the same program, but they ask for different metrics on different timelines using different definitions. So the program team collects the same types of data multiple times, stores it in different systems, and never has a unified picture of what is actually happening.
The deeper issue is that most workforce programs treat reporting and learning as separate activities. Reporting is what you do for funders. Learning is what you do for the program. But if your evaluation data only becomes visible when it is time to write a report, it cannot inform the decisions that would make the program better.
