Mary Styers didn't set out to become a program evaluator. In graduate school, she was studying something deeply human: how students cope when bad things happen to them. She wanted to take that research and build something real with it, a support program, something that would actually reach students. Her advisor had other ideas. Let someone else handle the applied piece, they told her. Stay in the research lane.
This stung, but it pushed her toward a group of program evaluators she'd been working alongside, people who knew how to move between the world of evidence and the world of practice. "I felt like they did a great job of doing the research, but also meeting people where they are and using that research to turn things into action," she says. That's been her north star for the 20 years since.
Today, as Director of Evidence and Learning Strategy at Instructure, Styers focuses on whether the evidence institutions collect actually changes anything once it lands. She joined hosts Melissa Loble and Ryan Lufkin on Educast 3000 recently to talk about why that question is harder to answer than it should be. The answer, in part, is the edtech landscape itself: K-12 districts accessed an average of 2,982 distinct edtech tools during the 2024-25 school year, according to the 2025 LearnPlatform's EdTech Top 40 report. Nearly 3,000 tools per district; education has never had more data and still more and more educators feel stuck.
Teacher buy-in vs. rigorous evidence
Just a few weeks before joining the podcast, Styers was talking with Erin Mote, CEO of InnovateEDU, when Mote asked her something she couldn't shake. Which matters more: rigorous evidence for a tool teachers don't believe in, or a lower bar of evidence for a tool teachers are genuinely passionate about?
Styers had spent her career being trained to answer that question one way. Randomized control trials sit at the top of the hierarchy. Causal studies carry the most weight. That's the received wisdom, and she'd internalized it. But the question made her pause. She took it to other researchers and their answers surprised her.
"If you have strong evidence but no teacher buy-in, that tool is ultimately going to sit on a shelf," she says. Styers wasn't arguing against rigor. Rigorous evidence divorced from context and trust often goes nowhere, and that's the part the field hasn't fully reckoned with. Local context matters, and teacher belief matters. A study conducted in a different district, with different students, under different conditions, tells you less than it looks like it does.
When research stops at "it worked" and calls it a day
Styers says there’s a pattern she’s watched repeat itself across districts and institutions for two decades.
- A study gets conducted
- Results come in
- A report gets handed to educators
- Nothing happens
A research report describes what occurred; it doesn't say what to do next. No one asks who it worked for, or under what conditions, or what might change if the implementation looked different. "We kind of stop at it worked or it didn't work," Styers says. That's a summative mindset, and it's the default in education. Summative findings without formative follow-through just add to the pile.
Styers was at a panel session on screen time not long ago when someone in the room said, with confidence, that every tool has evidence. She found herself thinking: does it? Instructure's 2025 EdTech Evidence Report looked at the 100 most-accessed solutions in K-12 classrooms during the first half of the 2024-25 school year and found that only 5% had met the bar for strong evidence under ESSA guidelines.
"When they can't connect that data to a real decision sitting right in front of them, it just becomes noise," Styers says. "And people stop listening." When educators are bombarded with research results, data dashboards, and assessment reports, over time they do what anyone does when the volume gets too high. They tune it out.
Evidence literacy: The skill researchers have and educators were never taught
Fifteen years ago, the conversation in education was about not having enough research, information, or insights. Time brought edtech vendors and dashboards that could slice everything every which way. Institutions invested heavily in access, but less in the capacity to use what they had.
Styers sees four places where it breaks down.
- Teachers don't have enough time to dig in.
- Nobody owns the question, so the data points in every direction at once.
- The culture treats negative results as failures rather than findings.
- Anear-total gap in what she calls evidence literacy: the ability to read a study critically, frame a research question, and decide what action to take next.
Researchers carry an evidence literary skill as a matter of professional training. Most educators were never taught it, and most institutions have done little to change that.
That gap shows up in ways that go beyond methodology. When people don't have a working relationship with research, they don't just struggle to use it. They distrust the people who bring it to them. Styers watched this play out firsthand when she changed her own job title. At a previous company, her boss suggested she drop "evaluator" from her role and go with "researcher" instead. The word evaluator made teachers nervous. It implied judgment, scrutiny, someone there to assess them rather than help them. "Essentially I'm a program evaluator at heart," she says. But the label created a wall before she'd said a word. Build evidence literacy, and that wall starts to come down. Leave the gap open, and even the most well-designed study walks into a room that isn't ready for it.
If a research study has only positive results, start asking questions
Ask Styers how to spot a compromised study and she doesn't hesitate. When she looks at a research study and sees only positive results, she gets suspicious. "There's something going on there," she says.
Real research produces mixed findings. A study with no negative or neutral results usually means the evaluation stopped at the summative question and never asked the formative ones. It also raises questions about who conducted the research, who paid for it, and whether the district saw the results before they were published. An equal playing field matters. Styers is direct about this: the incentive structure in education research is often misaligned. Vendors want positive results. Districts don't want to be seen as having made bad investments. Researchers can end up serving one party or the other rather than the truth.
Changing that requires something harder than a better methodology: a different relationship between researchers and the people they study.
How to actually build a culture of evidence, starting Monday
When Melissa asked what she'd actually do as a new leader trying to build a culture of evidence from scratch, Styers didn't offer a framework or a five-step plan. She offered a starting point. Pick one question your institution genuinely cares about. Not the data you have, not the dashboards already built, but just one question. Then work backward to figure out what evidence could actually answer it.
From there, the work is cultural as much as methodological. Negative results need to stop feeling like failures. "No one's going to get in trouble for that," she said. Reframing that instinct is how institutions start to learn, rather than just report. For districts without a dedicated research team, rapid cycle evaluation offers a practical on-ramp: short iterations, focused questions, quick feedback.
Her final ask, if she could change one thing, was simple. Stop treating evidence as binary. Framing everything as “you have it or you don't” or “it worked or it didn't” turns every research effort into a verdict and strips out everything useful. The better question is always: how is it working now, for whom, and what happens when we change something? Those questions are how rigor earns its keep. Evidence that can't answer who it worked for, under what conditions, and what to try next has stopped too soon.
Check out the full conversation tore more insight on on rapid cycle evaluation, what makes a study credible, and how to build psychological safety around data. Listen here.
