Assessing with Intention: A Behind-the-Scenes Look at IgniteAI Item Authoring Assistance

If you’ve ever had to write your own test questions, you know how tricky it can be. Figuring out how to hit the right standard, making sure the question is clear (but not too obvious), and writing wrong answers that are actually believable… it’s way more complicated than it looks.

Now imagine you could do it with a co-pilot.

That’s the vision behind IgniteAI Item Authoring Assistance, coming soon to Canvas and Mastery. The team is carefully designing a system that honors the craft of assessment while making the process faster, smarter, and aligned to real-world classroom needs.

To understand how it came together and continues to progress, I sat down with two people at the heart of it: Alli Sobiecki, who oversees Instructure’s assessment and standards strategy, and Marcus Vu, Ed.D., a former educator-turned-product specialist who’s spent years thinking about how teachers actually use tools like Canvas. They offered a behind-the-scenes look at the values, tradeoffs, and intentional design behind IgniteAI for assessment. 

What Role Should AI Play in Creating Better Assessments?

For Alli, good AI goes beyond automation to amplification. “I don’t want educators using any kind of AI model that my own team wouldn’t stand behind,” she said. “If it’s not something we’d publish ourselves, it shouldn’t end up in a classroom.”

That mindset shapes Item Authoring Assistance’s entire design. The goal isn’t to replicate what educators do, but rather to make the more complicated parts of their jobs easier. When teachers build stronger assessments, they get better data. And better data leads to better feedback, which helps drive student achievement.

Marcus approaches it from the angle of all stakeholders who could potentially interact with the feature. “We’ve got to think about the different people involved,” he said. “Teachers, admins, instructional designers… everyone interacts with assessment differently. AI has to work for all of them.”

Together, they’ve anchored the product around two simple ideas: save educators time, and improve the quality of what’s assessed. It’s not where it should be if it doesn't do both.

What Makes an AI-Generated Question ‘High-Quality’?

Ask ten educators what makes a good assessment item, and you’ll get ten different answers. But the assessment team focuses on the item actually telling you something useful about student understanding.

“A high-quality item should reveal student thinking,” Alli explained. “Too often, we see answer options that are either laughably wrong or so confusing they don’t measure anything. That’s a waste of time and a missed opportunity to help students grow.”

She’s especially wary of wrong answers that are so obviously incorrect they don’t challenge students or give teachers any meaningful data, or what she calls “Mickey Mouse distractors”. “You think you’re giving students a freebie,” she said, “but really, you’re losing the chance to find out what misconceptions are getting in the way.”

Marcus added that quality also means balance. “If every item is written at the highest level of complexity, you’re going to burn students out,” he said. “But if they’re all too easy, you don’t really know what they’ve mastered.” That’s why Item Authoring Assistance lets educators set parameters like DOK levels, standards alignment, and cognitive taxonomy. Instead of just generating content, the AI works within the instructional goals of a real classroom.

And while the technology is improving, Marcus and Alli agree that the end quality still depends on inputs. The more intentional a teacher is with source materials, standards, and focus areas, the stronger the AI’s output becomes. It’s collaboration (not magic!).

“If it’s inefficient and not saving you time, you won’t use it. If it’s not an effective, high-quality item, it won’t measure the thing you want to measure.”

- Marcus Vu, Principal Product Specialist

How Does Item Authoring Assistance Work and Learn?

Item Authoring Assistance is a guided workflow built around how teachers actually design assessments. 

1. First, educators provide their source material, like a page in a Canvas module, a reading passage, an uploaded document, etc. 
2. Then, they choose the focus topics, standards, or outcomes they want to measure, and set parameters like depth of knowledge or taxonomy levels. 
3. With those guardrails in place, IgniteAI generates items that are aligned, rigorous, and useful.

Marcus explained why those steps matter: “When an educator uses all of the guardrails we provide them to guide the Item Authoring Assistance, it produces a pretty high-quality question. But the fewer inputs you give, the more freedom the LLM will take when generating questions, which may not be exactly what an educator is looking to assess.”

That intentional design is part of what Alli calls keeping the “human in the loop.” The machine can generate items, but only the educator knows the subtle differences in language or rigor that will make an item effective in their classroom. “No matter how good we get at prompting,” she said, “the machine does not know you or your students. There’s not a one-size-fits-all approach to assessment, nor should there be.”

Efficiency in this case means teachers move past the mechanics of writing high-quality items to focus on examining the assessment results, giving more tailored feedback, and supporting student growth.

Anticipating the “What Abouts”

A conversation about AI naturally invites questions. 

What about open-ended items? 

What about tech-enhanced interactions? 

What about students using AI to answer AI-generated questions?

Alli sees the present work as informing the future. “Multiple choice is a natural starting place because it has such clear parameters, but it’s not the most authentic way students can show their learning,” she said. “Our work now is about laying the foundation—building models that understand what strong assessment looks like—so we can expand into other item types with the same level of quality.”

Marcus sees those “what abouts” as fuel for innovation. The team has gathered feedback from an Early Adopter Program, where educators are putting Item Authoring Assistance to the test. “The beauty of the EAP,” he explained, “is that it gives us a steady stream of real-world pushback. Teachers say, ‘This workflow is great, but what about digitizing my old PDFs? What about reviewing questions I already wrote?’ Those questions help us expand the feature set in ways that actually matter.”

In other words, the “what abouts” help inform the roadmap, ensuring that IgniteAI Item Authoring Assistance grows alongside educators' needs, and not ahead of them.

“There’s always a next thing we can do. The key is making sure every next step is grounded in strong assessment practices.”

- Alli Sobiecki, Director of Assessment Content

What’s Next for IgniteAI Item Authoring Assistance in Canvas and Mastery?

Educator feedback and a high standard for quality assessment guide IgniteAI’s Item Authoring Assistance evolution. 

Marcus summed it up simply: “We’re very thoughtful about AI’s utilization. We’re transparent about how it works, and our North Star is always increasing educator efficiency and efficacy.” That focus means teachers can trust the workflow and the values behind it: efficiency, efficacy, and transparency.

For Alli, the vision circles back to responsibility. “No matter how good the prompts get, only the educator knows what’s right for their classroom. “Human in the loop’ has to be true for assessment content.” 

The work continues as IgniteAI Item Authoring Assistance makes its way into Canvas and Mastery. 

• more item types 
• more guardrails
• more ways for teachers to spend less time writing assessments
• more time acting on what the results reveal. 

Because better items lead to better feedback, and better feedback leads to better learning.

Want all the latest on IgniteAI? Sign up for updates!