Making AI work for education

Welcome, everyone. We'll just give everyone a few moments to get settled into our webinar. Great to see a large number of people. If you've been in a session with me before, you know, I always love to ask in the chat where you're joining us from. So feel free to pop your details of where you're joining around the region. Amazing.

We've got some people from Australia, from the Philippines, New Zealand. Wonderful. Alright. Let's get into things in the interest of time. We do have a really exciting conversation to happen and some things to go through, so we do wanna make the best use of our time.

Welcome to our webinar, making AI work for education. I'm really excited that you could be here today for our second installment. We ran a series a few weeks ago, and this is our second session. Our first session focused on the broader strategic landscape, unpacking the why and the sustainability and the regulation side of things when it comes to AIUs. And today, we're gonna focus on the how.

We're gonna move beyond that big picture theory that we covered, and we're gonna dive into what this actually looks like inside of your institutions and, of course, how we can move into a more practical way of doing things. Before we begin, many of you are probably aware of a security incident that we recently experienced at Instructure. We are working with an outside forensics and cybersecurity firm to investigate this and have shared the verified information that we have at this time. You can find any new information and updates on Instructure's status page, and that's at status dot Instructure dot com. Given that we've shared everything that we know at this point, we do ask that we keep today's q and a focused on the materials that we're presenting on.

So with that, I'll quickly introduce myself for those of you that haven't met me before. My name is Farah King. I'm our global growth product marketing manager at Instructure, and I'm based in Australia. My role here is to really ensure that our regional needs are being advocated for and prioritized in the solutions that we build for you. And I'm gonna be your host for today's session as we navigate through the practical applications of AI.

We've got a lot to cover, but our goal is really to make sure that these concepts are really tangible for everyone here today or watching the recording. Whether you're an executive in the conversation or a practitioner who gets to work with learners on a daily basis. Firstly, I would also like to acknowledge country. So we respectfully acknowledge the custodians of the land we're meeting on today. I'm here in Australia on Woiwurrung land, which is part of Kulin Nation, and we pay our respects to elders past, present, and emerging.

We also we also recognize the enduring connection of Aboriginal and Torres Strait Islander peoples to country. If you are within Australia and know what country you're joining us from, we'd love to see that in the chat as well. And before I introduce our other incredible panel members, I thought I'd give a little recap of what we covered in the first session if those of you those of you that were able to dial in. If you weren't, here's what we did. We had a session with professor Martin Bean and professor James Adonopoulos where they explored the strategic why behind AI adoption.

They discussed a number of things, firstly, starting with that strategic mindset and really fostering integration over isolation when it comes to AI use. Evolving assessment and integrity were also a key topic. So moving beyond detection to employability. And since j n a GenAI is now a workplace reality, we know that assessments not just need to allow it, but really should consider mandating it to ensure that our students are job ready. Professor Martin Bean also noted that while some institutions are currently reverting to things like invigilated exams as a short term fix, the future will very likely involve AI led assessments.

So things like live Socratic debates where AI actually evaluates a student's competence in real time. There was also conversation related to institutional services and always on support where AI can be leveraged to promote that shift to twenty four seven personalized support for students across their learning journey. And finally, the session touched on lifelong learning. A very poignant and timely discussion was had on the lifetime of learning guarantee, noting that a degree is no longer a onetime event, but it also does require continuous updates as industry demands shift. There's rapid technology growth, and I'm sure that's something Sharon will touch on today in our session.

So the overarching message was really that AI is permanent. It is a disruptive force, and successful institutions are gonna be those that adapt their pedagogy and their business models to treat AI as a tool for human centric innovation rather than just a threat to academic integrity. And so that's what we're gonna do today. We're gonna go through that transition from the theoretical foundation from the previous webinar and take a look at that practical implication and how we go about doing that, looking at some concrete implementations that we already see happening. So on that, I am really honored to present my other two cohosts with me.

Today, we have Sharon Smolders who is joining us from the TACS Institute of Australia where she is the academic director. She's responsible for monitoring the quality of the institute's teaching and student progress as well as managing and reviewing academic issues and ensuring quality outcomes. We're really grateful that Sharon could be here today and to hear about her perspectives and the tax institutes moving from experimentation to embedded AI practice. We also have Greg Fowler, our director of solution engineering here at Instructure. Greg is also incredibly familiar with the growing use cases and expectations when using AI in education.

And so we'll be providing a technical perspective on the tools that we have and, of course, how to bring these AI strategies to life. So in today's conversation, we will be having just discussion with Sharon really to understand what what what we're seeing and where things are going. We'll touch on a few use cases as well. You can see on the slide here, which I'll probably come back to. But I'm actually gonna stop sharing just so we can start to look at what we're seeing today in the learning landscape and what we know our customers and what you are starting to do in your institutions.

So I'm really excited that Sharon was able to join us, as I said. And so I did wanna start off, Sharon, by introducing you now and really understanding where is the TACS Institute today in its AI journey, and what is driving your priorities. Thank you, Farrah, and thank you so much for having me here today. It's really a pleasure. I was quite skeptical accepting the invitation because we are a little institution.

We're not a very big higher education institution, but we're doing our best. And we I'd say we're on the established stage at this phase, but we're moving on to advancing, obviously. And I think it's this is gonna be a continuous journey. It's never gonna stop with with AI, the way it's it's changing so rapidly. So we've kind of and like I said, I caveat where we are based on us being a small online institution, so we don't have the resources that larger, higher education institutions have.

But we've done the basics. I think like most of the the institutions, we've done our surveys both with our students and our staff just to get the landscape, understand where are they on this journey and, you know, how far advanced are they or how much training do we still need to do. And we use those surveys to then update our academic integrity unit. So we included we added that to that specific unit. I know a lot of universities do that differently.

They either have a separate unit or we've combined it into the academic integrity unit. We've made it compulsory for all our students to actually complete that particular module. We've updated all our policies to include Gen AI in those policies, which was a huge task in itself. And then, obviously, we have strong governance structures around the whole GenAI function because it touches absolutely every single part of our educational journey. And so we have the teaching and learning committee, course advisory committee, as well as our, academic board and board overseeing the Gen AI action plan that we have developed.

So, yeah, we we now trying to integrate Gen AI in ways that align with the professional tax practice, which is absolutely critical, but obviously still in keeping and maintaining the educational quality behind that. That's amazing. It sounds like you've done a lot in this space. I guess I'm curious to know because I think, you know, every institution's had a slightly different driver. What was the driving force for you at the TACS Institute behind the AI journey? Was it the learners driving that need, or was it your educators? That's a very interesting question.

And I'd say neither in the beginning, and it might sound completely strange. But in the beginning, I think I think you need to understand we come from the accounting profession. So most of the accountant and lawyers then come into the tax field, generally the accountants predominantly. And the accounting profession is one that has been quite slow to adapt to Genii. They're catching up very rapidly.

But in the beginning, you know, the accountants were slightly behind the curve, unlike many other industries. And I think it's mainly because tax and accounting and refocusing on the tax side, it has to do with a lot of trust. And I think there's a lot of mistrust with AI and GenAI, etcetera. And, you know, you you need to build trust with your tax affairs, same with accounting. And we're more compliance orientated, and there's hefty penalties if you get something wrong.

So I think both the accountants being their nature, risk averse, cautious, that sort of thing, were slow to uptake. And most of our students are full time working individuals, working in accounting or law practices. And so they were also, you know, just dabbling with it, starting off to understand it. So not really forcing it or asking questions about it. On the teaching side, we're quite unique in that we use actual tax professionals that work in the tax industry or law industry, that actually teach to to our students.

And, there again, we've had some that are quite slow and far behind or just, again, very skeptical. I wouldn't say far behind, but just skeptical about the whole AI, rollout, whereas we've had others that were quite advanced and were experimenting and playing a lot with it and using it eventually in their practice. So in the last two years, from when we started the survey to now, I've seen a clear uptake in the use of Genea amongst our students. So in the beginning, it was more we're using at home for personal kind of things, a little bit at work, to now we're using it kind of every day. So there's been a clear uptick with the use.

But, again, we're not at the forefront yet. I think we'll take a while to get there. But so it was not really pushed upon us. It's just something that we realized we have to do. But, again, we took the same approach.

Let's be cautious. Let's see what practice is doing, and we've aligned with our our practice, and and we're working our way forward from there. That's amazing. And I know you said you're not at the forefront, but I feel like you you know better than anybody the disruptions that AI has brought both to teaching and learning and to the industry, which is why, again, you're you're so amazing to be speaking about this. I'm interested.

You mentioned that some of your educators were a little more comfortable. Some were a little more apprehensive. Has there been anything in particular that you noticed that was maybe harder than expected, you know, moving everyone along in their AI journey? Yeah. So I think the curiosity was there. So that's the easy part.

The practical part is the actual application. So it's I think on the teacher side, it was more of us saying, you know, it's okay to use. Don't be fearful of it because I think there was a bit of fear and apprehension. And a lot in the I don't wanna say the older cohort, but the the older generation the the the tax practitioners that are really technically focused, they they're very skeptical and worried. And to be honest, rightly so, because I think the learning and I'll get to that, I think, as we talk more about, you know, the whole impact of AI, is it's it's taking a lot of the learning away from our students.

So they're not really picking up on they're they're not getting the foundation that they used to get to build on from there. So it's it's taking our educators and guiding them. So we gave a special training module to them, showing them all the different types of AI. The legal aspects is another huge thing around AI. So at the moment, institutionally, we're not using this as as much for our processes.

We're using it more for the education side. But it's it's getting them to realize that it's okay to use it, and it it can work. Whereas some of them are saying, but if you don't know the technical elements, you're not gonna get AI is not gonna help you in any case. We'll spit out an answer. We you we always say to them, you've gotta, you know, read it, validate it, and then amend it if necessary.

And he's saying, you can't do that if you don't have the foundational knowledge, which is the part that we worried about that the learners are currently missing with AI because they go straight to the solution without having, you know, worked through the question, got it wrong, see where they went wrong, bring it to memory again, and then use their professional judgment. So I think the hard part on the teaching side is getting them to realize that there is you can't get away for it. And I think that's the big thing that I was trying to tell, you know, the ones that are more advanced in the technical front is that, yes, I agree. Technical is really important, same with doctors, whatever it, engineers, whatever it may be. But you still have to we're not gonna get away with it.

It is going to be used in practice. It's just how we use it and how we use it to teach the knowledge. And specifically in our case, the professional judgment because professional judgment and the critical thinking are the key components that I think we're missing with bringing AI in straight away. So so for those that have been through the old programs and that, you know, you had to learn the legislation, all that kind of stuff. Now it's just so easy to don't even have to learn it.

Just type it in, pops up, comes up with examples, etcetera. So you're missing a whole component of the learning part that I think is the issue. So that's the one part, the teaching. And it's been quite hard to get them over the hurdle to say, it's okay to use. Don't be fearful of it.

And I think the fear's sort of gone. It's now, okay. How do I use it? Where do I use it? And what do I stop doing and bring in the use of AI? So that's the part that we're kind of grappling with now is to get them to take the next step and and use it more actively in what we're doing, which is it's not it's easier said than done. For for others, you know, that are are embracing this change, they're just flying, and you can leave them. They're experimenting.

They're using it. They they're using it to, you know, fix the material. Not fix, but make the materials condensed, more condensed, more readable. Because, you know, the younger generation wants everything short and quick, and that's the way they want it, and I wanna see it. So we try to use that to make materials more enticing with law and accounting and tax law specifically.

There's so much reading that you need to do that this just does help put it in a more understandable, digestible manner. So, yep, that's that's where we are. It's it's not easy. It's difficult. And then I I won't even talk about the assessments yet because that's another whole issue.

Well, it's funny you should mention that because that's probably where I wanted to go. But I I really appreciate the honesty in what you're saying, and I'm sure, you know, I would be surprised if there was anyone in this session who didn't resonate with what you were saying. I also teach sessionally for a university, and I'm hearing and seeing the exact same experiences with students and, of course, you know, with my AI use too. And I think I love that you touched on the idea about it's not just that we have to use it, but it's how to use it responsibly. And I think that was a really key theme in our previous webinar and and one I wanna dive into a little bit with this one.

You touched it previously on, you know, some of the policies and the governance that you put in place, and I know we started to get a little bit into academic integrity as well given that that's probably a a a more common frame everyone's bringing. So from your perspective, what are some of the main concerns or questions that you're seeing raised internally as as your staff start to engage with AI? Yeah. So as you mentioned, ethical use, I think, is key. A lot of you have seen in the media with the accounting large the larger firms misusing AI and the ethical use behind that. So that's the key important message that we're trying to get there, and I think that's been going since AI came out.

And thanks to I hear for all the webinars on AI that I've always been watching. It's been the the ethical use. So teaching them how to use it ethically and responsibly isn't is key. That's the absolute imperative. Once we've got that, it's then teaching them to well, changing our material so that, as I said, they learn to develop professional judgment and critical thinking, which is not an easy feat at all, as I said, with AI coming in.

AI assists without a doubt, but it needs to be the way in which we structure our assessments and our materials, unfortunately, need to change. And and that's where the additional work comes in for us as as educators. And that links to, you know, authentic identification of learners in assessments. So that's also becoming which I thought was quite simple in the beginning, but, you know, it's just it's frightening, what they can use AI for now, just making sure that it's the correct person doing the assessment. We're a fully online institution, so we had huge risk.

So it's something that we take really seriously and making sure that we have proper authentication, for our students when they write the exams, and that's, you know, showing their their IDs and all that sort of stuff, before they write. And then, of course, we have proctoring on the actual assessments itself. We don't have face to face. We have students across the whole country, regional, remote areas as well. So we it's very difficult for us to do any face to face.

And I've we've got a lot of academics on our academic board at larger institutions that have said the risk is just too high for them. So they're going back to face to face assessments, you know, where they do it pen and paper. It's it's amazing, but that's where a lot of them are going back to just to overcome the risks. So we've kind of said on an assessment space that we have final assessments, no AI, which was lovely until, you know, we said they are allowed to use the ATO website, certain allowed websites, you know, with risk material that's generally available for them as it would be in practice. So we want to make our assessments as realistic as possible.

However, if you look at the ATO website or if you look at certain of the CCH, Thomson Reuters, whatever it may be, their search functions now include AI. So then we got questions like, how do we do this? Because it says AI. And so it's been quite quite a a challenging time, and we've now had to put certain criteria around what AI you can use or not. So even in the, like I said, the normal search functions, there is AI now. We can't stop it.

We can't prevent that, but we can limit the use to as much as we possibly can. And that's where, again, where the proctoring becomes really, really important. It's quite time consuming, but they do flag certain things. So we we spend a lot of time reviewing the videos of what they you know, where they're clicking, what are they looking at, that sort of thing. So the proctoring on in an online institution is absolutely essential now going forward.

But I think the yeah. The main concerns, I think, come to ensure that we're enhancing the professional judgment. And that's, you know, AI can perform the reasoning before the learner even attempts the question. And and that prediction error signal that that drives the deeper learning, that's the part that we're so cognizant they're they're actually missing out on that completely. And that's what makes a whole educational design.

It it just flips it around completely and and and requires us to do a reevaluation of how we actually deliver our material and how we assess the the material. So it's the authentic assessments. We've gotta build the internal map. So even before we get to the assessments, we've gotta build the learning materials so that they they build internally the maps on how things link so that the constraints come out. They can see the different patterns that emerge and then use the AI to improve, not to get to the answer originally.

And it's hard. It's it's not easy because we need to deepen their learning so that the, you know, the expectations are violated, and then they're corrected, and that's when the deep learning takes place. So that's what we need AI to do, not just provide them with with the solutions. And that comes back on us as educators. You know, we need to now design our materials and assessments so that we build in those professional judgment, critical thinking requirements.

I love that. And I something you said that I do wanna dig in on is you mentioned something about flipping assessment around, which I think is really interesting in this in this era where we're we're all redesigning assessment at the moment. I'm I'd be hard pressed to find someone who's not. I'm doing it in my teaching as well. And we know a lot of our institutions right now are at that stage where they're looking to implement something that can leverage AI, that can assess for critical thinking, but, you know, takes takes the the nature of just having AI generated out of that.

And so one thing I thought it would be timely to mention just with what you said about flipping assessment around is using AI experiences in assessment. And so we know that there's potential ways for assessment to be interactive or, you know, to leverage that critical thinking side of things and have students really demonstrate their thought process rather than have that end product. And so on that, I might actually hand over to our other panelist, Greg, our director of solution engineer, who can actually chat us through a little bit of some of the work we're doing on AI experiences for assessment. Yeah. And, Farah, do you want me to demonstrate this at this point? I would love that.

Perfect. So Instructure is currently building a integration with OpenAI's technology, LLM. And what we're trying to do here is actually create a a mechanism for students to chat with a large learning large language model. So the example that I've got here, a little contrived but slightly relevant to the audience or the the presenter that we have to today. I've mocked up a AI experience for students to chat with Jim Chalmers, the Australian treasurer.

So based on the proposed changes that Jim may be presenting around taxation, you know, changes in in twenty twenty six budget, what we're doing is asking for learning objectives for the end of the chat that students have with this assessment. We've then put some information behind the scenes to help this closed off LLM give context as to what the what the student is going to be chatting with this persona. So when we are creating this as an educator, we have fields where we can, add the learning objectives for students. We can provide guidance that will display through to the learner when they're actually chatting with the model, and then finally, any source material that we do want to add. With all of that in place, a student can then interact with this model.

And so we can see here that I've previously had a conversation last night. First question, what do you understand about the proposed tax reforms for twenty twenty six and the key objectives? As a student, I gave a mild response back to this. And then based on my response, the model of Jim Chalmers is then asking me follow-up questions. And as I'm going through there, it's telling me which learning objectives I'm actually taking off live. And then with all of that data in place, I can continue chatting to the model.

But, also, when an educator does come in to have a look at the student's, submission, they get the option of using the AI analysis, and this will help, understand the AI's assessment of the student's interaction against the learning objectives. And so against each of these pillars, we're getting a score. We are seeing the student's strength in their chat with the persona, areas for improvement, and then an overall summary. So this is, at this point, for the educator only, but we can then see the complete conversation, and this will help us grade the student's final work. All this can just be used as more of a formative learning opportunity as well.

So this feature is currently in development, not something that's yet available for customers to, interact with. And the intent is while this is currently developed using OpenAI, we are hoping that this will be available to, institutions to bring their own model and integrate that in. So it won't be something that requires a specific model to leverage. So we're really excited about the direction that this is going. Farrah, before I stop sharing, anything else that you want me to cover on this one? No.

I was gonna ask you, though. I guess this is, you know, incredibly timely as we were just talking about this idea of updating assessment in line with AI and and what we need from an academic integrity point of view, but, of course, from an authentic assessment point of view. I'm just curious to know, like, what else you might have been seeing in some of the institutions institutions that you meet with on a regular basis in terms of what they're experimenting with, what LLMs they might be bringing, or, yeah, how they're starting to roll this into their day to day operations. Yeah. I think it's very varied.

It really comes down to, you know, the appetite to actually sort of experiment whether they want to get the AI policies and foundations put in place to begin with. We've certainly seen quite a few organizations actually go and try and build custom chatbots and things that are more institutionally specific than relying solely on vendors developing, AI specific to an application. So we've seen some universities develop, chatbots which are more, wide reaching and span more than just the LMS. We've seen AI tools which are specific to content generation or even AI tools that are looking actually designing the content rather than actually generating the content as well. And I think it's really coming down to a few different pillars around what level of transparency do the organizations need to present to their end users, be it the learners or the educators as to how that AI is being leveraged, what's under the hood, what information is being used within the model.

I think we've also then got quite a few concerns around the granularity of controls for the AI. So who is it exposed to, and is it a limited group of educators, or is it available to everyone? And then I think we've also got a lot of conversations around rationalizing the spend of AI. So if we've got institutional AI subscriptions, how can that actually formulate part of the institution's overall strategy? So quite a lot of experiments in place, and then I think there's also organizations who are much more sort of keeping a watching brief before they actually embed that in their teaching and learning. And it may be that it's more around individual educators actually embedding that into their workflow rather than having an institutional approach at this point. Very varied.

Yeah. Thanks, Greg. That's incredibly insightful. And I might even I'm just gonna reshare a slide I had quickly previewed just at the start of our session because I think this is a nice timely one to bring in in terms of what you were just talking about. What we're seeing, just some examples of what other institution or institutions are doing.

These are just some examples we're seeing here in Australia, but I know there are many. I've heard amazing things happening in the Philippines as well. If you happen to have attended a webinar that Greg and I actually ran towards the end of last year, we did actually touch on some of these great examples that we're seeing of AI use within institutions where they're building their own things and driving their own uses. And so one exciting thing we've seen here in Australia is an example from RMIT University where they have Vowel, a GenAI chatbot. And it's not just a generic chat interface.

It actually uses specific personas to meet the different needs of students when they need it. For instance, they have Imagineo, which provides, like, a role based scenario based role play. They've got Quizzicle for generating quizzes. They've got Essay Mate to give feedbacks to students on their writing, And they even have something called Poly Chat, which brings in all of the institutional policies into one place that a student can leverage AI prompts for and get answers to. Another one that I know many in Australia will be as well is the work coming out of the University of Sydney with Cogniti, and this is a platform for educator built custom AI.

So it's really pedagogically driven. It is hosted by institutions. Institutions. So a number of institutions globally are leveraging Cogniti as well. And so it's really a standout feature that helps leverage that Socratic method I mentioned right at the start of our session, where instead of just giving answers, the AI can engage students as well in prompting and questioning to to get to that heart of critical thinking.

So we are seeing some really great things happening outside of of Canvas as well as inside Canvas. And so I do wanna bring it back to just some of the conversation, Sharon, that we were having just before and really thinking about, I guess, what this means, you know, as we start to think ahead with the friction, I guess, in this in this adoption of AI, and you did touch on that just before as well. Is is there really a challenge more with the capability of of the teachers, of the students? I know you touched on some being very confident and some not confident. What are the competing priorities that they're dealing with? Well, as you said, Farrah, I think it is. It's it's capability, it's confidence in use, and, obviously, other competing priorities and resources.

I think a lot of it comes down to resourcing. We're a small institution, so we have limited resources. So I'm listening to all this stuff, and it sounds absolutely amazing. And so I'm wishing we could introduce a lot of it. So a lot of it, we restricted because of the resources.

But I think if if you look at competing priorities, that's always been an issue. It's not unique to to AI at all. We've got a we've always had to refresh our programs for us specifically in text. We have changes every year. We've got regulatory reporting, you know, revenue enrollment pressures.

That's always there. So so that's AI is not being deprioritized. It's just one additional thing that's now being added on to all our other priorities. And it's it's what we're trying to do is integrate it into our work stream so it doesn't become an additional competing priority. It's gotta become part of what we are doing.

Capability, as I said, it's it is a constraint to a certain extent. I think we're kind of overcoming that now both with our students and with with our under our teachers or lecturers, but it's still you know, we've we've done the the foundational training. Now we need to take the next step and and enhance because it is changing so rapidly. So it is both on a student front and because, also, there's different use. Some students, you know, again, the equity in the actual access to different types of of GenAI, some have paid for and can get amazing results.

Whereas we at the moment only can offer us, not even our students, just our staff, the institutional copilot license and hoping to offer to our students similar to RMIT, I think, does at the moment. So it's it's the the capability is there, but it needs continuous work to improve the ability of both the learners and and the teachers. And then if we look at the confidence, as I said before, that's there is no I don't think the the fear is gone. I think the fear has left. It is now just making sure that what we are doing so it's recognition that Gen AI is both an integrity risk, which it is, but also a legitimate graduate capability.

So it is absolutely critical that we have we teach them to be able to use this ethically, responsibly, etcetera. And we also have the professional use. So we've got a how is it going to be used in practice? So higher education has certain requirements, but we kind of more focused on how are they gonna be using this in practice. And we need to ensure that we are training them so that when they get into practice, they have all the necessary skills to be able to do the work that they are required to do. So it's it's a combination of all of those, but it's it's making sure, I think, more from our side that it is providing the graduate capabilities that are required when when they finish studying with us.

That's so important for both, you know, the employer, but also for the economy, especially from a a tax perspective. But there's a lot of risks involved in all of that as we've mentioned along. So these new newer capabilities that any assistance will be mostly appreciated. This looks fantastic, what you guys are bringing in. So it's, we're kind of lacking on the communication side.

So we've got the written communication better down. It's the oral communication that a lot of employers are are complaining about in practice that the younger generation just can't seem to converse properly. So it's getting the oral conversation going. The the written, we've pretty much covered to a very good level, but it's the the oral communication that this kind of thing would be great. If, you know, if it's to see the written suburb, we could do a verbal one.

And we do have I mean, we've got the the viva voce, which is the supplementary exams where we it is an oral exam, which is fantastic. But, again, it is resource limiting. We have a lot of students, so getting and most of them work. Our lecturers work. So it's trying to get a time convenient for both of those.

So if something, you know, that that you guys are introducing could help with that, it would be absolutely fantastic. That is good to hear. I know live assessment is definitely something that's becoming more common, definitely, in some of the Australian institutions that I've been speaking to. So, yeah, it is very much real. And I I wanted to dig in, I guess, just a little bit on what you were talking about about the graduate capabilities and that AI is going to be a skill that's expected in jobs.

And I know earlier you mentioned at the start of our session that you've woven in AI use into some of your academic integrity modules. Have you woven AI use and how to use it effectively within some of the more specific subjects that you're doing as well? Yes. Sorry. We have. So we've actually provided AI output.

So we the the lecturers typed in a question that they wanted to ask the students. They put it in AI, and it was so funny when the one lecturer actually said to me, I was so glad that the answer was wrong. I was so worried I was going to get it right. But it was incorrect, and we put that solution into the question and say, critique the question and say, is it correct? Is it not correct? Why? Explain, etcetera. So we have embedded that into most of our assessments now as well.

So we're using AI, to test their knowledge and for them to also realize that AI is not always correct. And I also was under the false illusion that, you know, tax technical it's tax is really complicated across the world. It doesn't matter which country you're in. So I will not get that right. And I've got a lot of friends in in large corporate international banks and large corporates as well.

And, you know, they're also, again, on the skeptical level, starting to produce AI. I can't really help them much. But moving two years along the line, and they're saying it's technically becoming so much more accurate as well. So there is gonna come a stage, hopefully, not too soon when AI is correct. You put the question in, you get the correct answer.

What's frightening for me is I'm hearing a lot of individuals are now choosing to use AI instead of a tax agent to do some of their tax and bills, which is quite frightening to a certain extent. So it is again, that's where the education understanding of what AI can do and what AI can't do, and, ethically, what should you be doing. But that all comes back to technical knowledge, and I I'll always loop back to you have to understand the foundations. And AI cannot take your learning the foundations away and should not take that away, and that's what we've now gotta try and embed into our assessments. And we we're lucky enough.

We have tax agents and, you know, lawyers, etcetera, preparing our questions, so they all work in practice. So it is practical relevant questions that are novel questions that come into our assessment. So we're quite fortunate from that perspective that we kind of and don't wanna jinx us, but we're kind of AI approved to a certain extent. But, yeah, there's a lot more that we we want to do in this space, and any tools that can assist us like you've mentioned previously are most welcome because there's only a few of us. So we anything where we can save time would be fantastic.

But, yeah, we we we are embedding it in. We are lucky enough, as I said, that our assessments are are quite novel and new. It's not something that they would easily find on GenAI or get an answer to. But, yeah, it's getting there. It's getting there.

So, again, it comes to teaching the ethical use and the under foundational knowledge and understanding so they can use their professional judgment and critical thinking. And I appreciate that you touched on, you know, tax agents and educators alike. There still needs to be a human in the loop. There's very much a place for us in this space, and it's not about automation. It's about augmentation.

And so I love that you've highlighted that because I think taking it back to the teacher side of things with the educator lens, we know that this is key for what we do. We we still need to be the ones making those decisions, ensuring that our learners are having the best experience, and so really building educator confidence and and control when we're embedding AI into those educator workflows. So we do have a couple more things that we can touch on from an instructional side of you in terms of what that could look like for educators with our other parts of our Ignite AI offering. And this is where I was gonna throw back over to Greg, if that's okay with you, Greg, just to talk about some of the other AI assisted features that we have in Canvas to help help teachers with their everyday workflows but still keep them in control. And that's really around AI grading, which we know is a little bit of a contentious one.

So we do wanna dive into that controversy and discussion insights as well, which could be really interesting. Yeah. Thanks, Farrah. So a couple of different tools that I wanted to highlight today. The first being probably our most recently, introduced tool called the Ignite AI agent.

This agent is a, agentic bot that is available to both, educators and administrators. And, really, it's difficult to give you sort of the scope of what this can do because it is it has access to hundreds of different Canvas APIs. So based on your access to your Canvas course, that might mean, you know, helping with the creation of content or, like, I've input this morning here. I asked the chatbot to find students who haven't accessed the course in the past ten days and then send them an encouraging message to log in. And so it scanned my course and found forty nine students who haven't accessed this particular course in the past ten days.

It's then drafted a message, and the action that it's asking whether I want to approve or reject is sending this message to all forty nine recipients, that it has identified. And so it might be, you know, don't like the message that it's drafted, and I'll suggest a follow-up message, and so I can reject this. Or I could click approve, and that will then, take those forty nine students it's automatically identified and send them the the encouraging message. The same sort of, automation will exist in other areas of Canvas as well. It might be that you want to ask how many ungraded assessments do I have per assignment, and that can potentially identify, you know, the earliest assessment that I do need to come in and grade.

And the controversial being the thing that we get the biggest sort of varied reception on and maybe the most controversial in terms of alignment through to university policy is the newly introduced option within SpeedGrader called auto evaluate. So selecting this will then take the student submission and the content of my rubric. It will then analyze the student submission. And based on the rubric description and the criteria descriptors, it will then identify whether the student was exceptional, proficient, or whatever language we are using within our rubrics. Anytime that we see this purple highlight around this rubric criterion, that indicates it's the AI's determination, and we then provide the score rationalization here to your educators.

We can then come through and see we'll also suggest a comment. Anytime that I amend that comment, the purple highlight that AI suggests is removed, or I can revert back to the original AI suggestion. And so once we have determined all of the AI generated criterion and comments, we can come through and save those results back to the student. In terms of the transparency that we've spoken about a little bit this this afternoon, when I click submit assessment, the student will also receive an indicator that that assessment was AI assisted. So they are getting an indicator that we did, you know, select that button, and we did use AI generation in terms of the criterion and the commenting.

Doesn't mean we just click came through and click submit, but that, button was selected. The less controversial option that we've added, it's more of a a real benefit through to educators within a Canvas discussion. We can generate insights, which is now at a per student level as to whether they posted a relevant discussion post, whether it's something that potentially needs review, or whether it was irrelevant. So in this case, I've only had two, twelve students post, but we can imagine in the course of several hundred or thousand, being able to filter down to just the students that need to, we need to review would be a massive efficiency. We can then click through to see the reply for Marshall, Susan, Brad, Beth, and see why that was highlighted by the AI model as being something that we might need to review.

So the reply is brief and lacks detail about the posters background and interest as requested in the discussion topic. We can send feedback back through to the model or click see reply in context and come through and offer additional support through to Marshall or encourage a follow-up posting here as well. And, Farrah, I think that's something that you would really appreciate in your online teaching. Yes. I was just gonna say similar to Sharon, I also teach in a purely online setting and control through hundreds of discussion posts, which that would be amazing to know where I could intervene with students in a more immediate way.

Sharon, I'm not sure if you're leveraging discussions in any of your courses, but is that something your instructors spend a bit especially in the early weeks of the subject where everyone's frantically, you know, lots of activity. Is that something that you see? Yes. So we we try it's quite interesting. In the beginning, the discussion forums weren't used so much, yet we're getting the survey feedings. I think a lot of online students feel very isolated, alone, that sort of stuff.

So it is strange that it hasn't been used as much, so we try to prompt the use of it. And it has been picking up recently, so that that at least is good to get the students talking to each other and us then prompting them with certain questions. So we've started prompting the conversation sometimes to get the conversation going. So, yeah, that will be fantastic to be able to use, and we we're trying to get the engagement going within the time frames that they have available to talk to other people. Yeah.

Likewise. Look. The early weeks are always when everyone's posting. I could see that being helpful. With the grading assistance, it is a really interesting one.

As Greg said, we are seeing interesting feedback both from students and from educators in this space. You know? Educators, from my point of view, when I am marking and I spend a lot of time marking, I'm not gonna lie. It gets a bit tricky when you're a few hours in and the bias creeps in. And so being able to start from an unbiased place where, you know, hungry Farrah isn't marking. It's unbiased Farrah.

That would be lovely. But I can also appreciate why students want a human doing those things. And so seeing that the human is still in control is is absolutely key for me as an educator. Are you leveraging rubrics, Sharon, or have you had students raising these sorts of things? Yeah. So we use rubrics, but we do it all manually at the moment.

So it is the the actual assessors assessing it and writing comments in, and they get the the comments back on on Canvas. So we haven't used any AI in that regard, but it's something that, without a doubt, we would be interested in looking at. And, again, I don't think it can be used in isolation or just, you know, left and submitted. It has to be reviewed. It has to be validated same as any I use.

But if it can save time and and speed things up a bit, by all means, we'd welcome any anything like that. But, again, just double checking to ensure it is it's valid. Yeah. But, yeah, we would love to to implement that. Yeah.

I think the value add, at least the things that I see in my own teaching, is it means that I can actually give more thorough feedback. I can probably dive in more deeper than I would have when I had the volume to deal with. So, yeah, exciting to see where they can take us. I'm just checking if there's any I don't see any outstanding q and a that maybe we haven't gotten to. There's been some amazing chat happening.

I haven't been able to follow everything, but do appreciate all the q and a that's that's been going on and lots of good conversation I can see in the chat. We will be sharing a recording to this as well, and the recording for the first session is also available. I do encourage you to watch the conversation with professor Martin Bean and professor James Adnopoulos. It was a great conversation that really lent itself nicely to this one and and being able to unpack those practical implications with Sharon, which is exactly where all the work is living for us at the moment, those of us that are working with AI more closely. And, Sharon, I do wanna thank you again for giving up your time to share your insights.

I think, as I said, everyone in this in this session or watching this recording is in this space right now, and we're all grappling with how to do this thoughtfully, how to keep the learners at the core of of everything we do, and how to give them the best experience going forward. So very grateful for your time. And thank you, Greg, for giving us those insights into what's coming and how institutions can leverage those things thoughtfully while still staying in the driver's seat. I think that's incredibly important as well. If there aren't any other questions, we will wrap up for today.

We'll give a moment if there's any last questions anyone wants to raise. But, of course, those of you that that know us, please come along to our future webinars. We would love to see you at these things, and we wanna hear from you as well. So we always love hearing your stories. That's probably where the greatest value is in hearing what you do.

So we'd love to have more of these in the future. But thanks for your time. Thanks for choosing to spend an hour with us. We hope you got something helpful that you can take forward. Have a good day.