[Panel] Beyond the Buzz: Practical Pathways to Meaningful AI Integration in Canvas
I'm so excited to be running this session with two of our amazing customers and really excited for you all to hear some of the great work they've been doing, particularly in the AI space. So I think just to get us started, we'll do some introductions first. So Glenda and Sermin, we'll ask you to tell us a bit about each of your roles at Della Salipa at Ateneo. Yes. My name is Glenda and I am the managing head of the integrated digital and innovative instructions office. So I oversee the learning management system and other ed tech tools in our school.
And currently, I I spearheaded on, what's this, the AI task force to to develop things that are needed for our institution. Amazing. Thank you. So, I'm a faculty member of Ateneo Graduate School of Business. I'm with the Department of Operations and IT.
I'm also part of the Quality Assessment Group of AGSB and sitting in as representative for our educational technology group for the university. Wonderful. Thank you both. So we've got some slides that we'll be showing along the way and really just hearing some of the great work that's happening in Ateneo and at Dulles Alipa. So first, for both of you, can you share with us a bit about how you've integrated AI tools into Canvas? You know, things like streamlining your assessments or coding, any automations.
What have been your experiences so far? Do you wanna start us off? Okay. So, our first use case for Ateneo specifically is that we wanted to take a look at the pain of documenting meetings. And so, and our experience is that if you have roughly about half a day of or to one day of meetings. Even my clients would normally complain, it would take them something like three to five days to process. An mp three file.
And so for us, we actually did attempt. We In Zoom, there is a You have a host of apps. And then, for us, we use Fireflies. And then, the documentation can be fed to let's say, Notebook LM or Perplexity or Gemini. So that you can actually feed a specific template around it.
And then the the notes will naturally put in place. And we did that for Ateneo and for our for our planning sessions. And I was the one that documented it as a proof of concept. So every half day, we're able to produce the minutes of the meeting at one o'clock. So, it was thirty minutes to forty five minutes.
You could already produce the output. Instead of three to five days. Yes. And so we also when when Dean talked about documenting, We actually have a use case there where it also goes to Canvas. So the recording, the video, and then the minutes of the meetings are also there.
Amazing. Thank you. Let's have a look at another example that you can tell That's ours. So, we also have a Canvas page that's that's for the strategic planning. And so, each of the parts have their own synopsis.
The second one would be the minutes of the meetings. And then for documentation, can you please move to the next slide? Thank you. There you go. So there's the, it's viewable on YouTube but it's private. Okay.
So that's our use case. Documentation efficiency. So every meeting, the outputs are are released automatically. And your users or your staff can just jump in? Our staff have begun to use this already Amazing. As a workflow.
Thank you. So I think we've got another example now from Glenda. Yes. It's not Is there a video? So my first thinking about the use of AI and integrating it with Canvas is that I would like to think of two possible tasks, and that is regarding the process task as well as the people task. So I would like to think about how will our faculty members could easily be freed up with all of the usual tasks that they created.
So what I did is that I tried to explore things like since I am teaching, for example, computer programming, I decided to use Gemini Gem Gemini AI to develop an app or a web app that could enhance the learning and teaching at the teaching at the learning of my students, and surprisingly, they were able to, you know, grasp all the things that they would like to learn in Canvas. And I in in that particular lesson. So what I did is I utilized several tools, content that is that that is that can be used to to develop contents as well as another tools, which is a web quest creator, and then embedded it in Google site and, put it in Canvas as an external link, something like that. And I really am glad that my students are taking it are are improving, particularly in grasping the learning that they had. And how did they feel about using the AI when you said to them using Gemini? Initially, they are amazed that I am using AI because in the first the first reaction of the students, mom, you are using artificial intelligence.
Why not? So I'm being transparent with you, and I would like you to know that it's okay to use artificial intelligence as long as we're gonna use it ethically and responsibly. I love that. That's a great way to practice things and model for the students. Yeah. It's really lovely.
And so with the faculty. Yeah. I provided training for that kind of actual activity. I love that. I think we have another use case here about embedding QA into Canvas.
Before I discuss my use case number three, I'd like to just add to what Glenda cited as a best practice for her for her school. This term, I innovated my course and I actually taught the students how to write their own mini GPT. So talk to your content. Second would be, they can now they can now practice soft skills. Because using their phones, they can actually interview or get interviewed.
Next, I used to use a lot of visualization apps like Figma before. And then now, the students don't have to use that anymore because they can actually create a Gemini Canvas. To do A website. Number two, an app. The third one would be a simulator, a dashboard which they used to do in Google Data Studio.
But they don't have to do that now. And so, they get feedback not just from their peers but also from AI. Case number three is embedding QA in into Canvas. And so, based on our experiences with Pascu, there was a lot of variations in terms of how the the faculty presented their classes even in the syllabi. We used to have some degree of uniformity.
But when they implemented it in Canvas, we realized that would have been a very good way of improving look and feel. Second, we wanted to also make sure that, it's going to be consistent with PASCO and AUN standards. And so, the dean, had us come together and look for a way to be able to come up with standards. Number two, to check the syllabi and see if they're consistent with accreditation standards. And in this case, can you move to the next slide, please? We designed an internal playbook and then, we merged that with OBE, OBE principles.
And then, the AON standards. And then, assess from the playbook using those two standards, Each syllabi in the university, each syllabus. And so, we we had a Gemini gem. We created that. Next slide, please.
And so, we gave out the prompt to the faculty. So that before this one was sent to quality assurance. So the the process was QA. And then from QA, later on, it it would be vet by a person in the department or in the the quality assurance group to double check the findings, Then, it would go to the department head, then ultimately, the dean. But, the moment we realized that the prompt was stable, we released it to faculty so that even before they would submit it to us, they'd actually have an A report already from AI.
Next slide please. And so, that's what Gemini came out with and then eventually, it made its way into something that you can send out as a Google link. Alright? And so because of because this has to had worked already, we're thinking of actually converting it into an agent anymore already so that they it can process batches. So there will be no human intervention required for this one. Wow.
How much time do you think that has saved? The processing, it took me a while to get this. It took us a while to get this done correctly. And so if you try to take a look at the final output in Gemini, in in Google Sheets, That's the one. This one takes the assessment now takes about less than two minutes. What did it used to take before? Again? Before? Before.
Well, you had to go through. There are something like forty eight questions that a human evaluator has to go through because this is consistent with AUN. And then the second one is you have to double check the the content. And you have to find out and they this is not sequential though. If you try to look for it in the in the syllabus.
So it used to be about an hour, an hour and a half Wow. Per syllabus. Two minutes now. Two minutes now. And then from there I review at my level.
Next slide, please. And then I identify which part is the human portion. Because We did say that the initial AI, result is not necessarily something that you trust. So you have to double check for hallucination. Now, is what we realized with with Gemini.
The more you feed it, there's going to be a time even if I'm on pro, it hallucinates. And so you have to give it time to rest. And then, you have to clean out the process so that it dumps your prior searches or queries. And then, this is a snapshot of the email that I send, which eventually That's my review. The moment it gets to faculty, faculty works on it, puts it on Canvas, submits it to the department chair, another round for QA, and then the dean.
So similar to what I asked Glenda, what was the staff reaction when you said we're doing this with AI? This one is still on test. When you say academic staff, we did have an initial reaction that said, okay. Why is AI there? Okay. Why is AI there? And then second But it's very important also that when you're coming up with An agile and a lean way of doing it. There's not much time between Paasco and AUN accreditations.
No? So, I think we had less than about a couple of years to implement the changes from one standard to another. And so for this, you would like, we would like basically to be more efficient with the with the repetitive tasks of administration. That's a great example. Thank you. So what results what insights are you seeing at the moment in terms of student engagement with your with your use cases or workload or learning outcomes? Because I think that's a really big piece is we're using AI in a lot of ways, but how are we checking if it's actually working? Yes.
Yes. As I think we got overwhelmed with the results, the positive results that does when we start using AI, particularly, for example, in my student in in my class. And that there is an increase in student engagement. They became more collaborative. Yeah.
And they are really into it in every for example, with the webcast that I created, they were into it. They really are performing the task. And even when we go out in in the classroom, they are talking about what happened. Because another thing that I that I included is deciphering codes. Let's say with the use of binary numbers and so on, and then they decipher codes with the use of this created generated web quest, something like that.
So it's really very I I got it I'm happy with the result of of integration of AI in in all of my lessons. And then with regard to the faculty, I think Oh, It has been the result has been no. There are several faculty members that are already into AI, and we provided training regarding the way how I utilize it and that they are seeing it as very promising, and they're thinking about pre preying themselves about the usual routine regarding these processes that they usually created into a more people oriented task, is providing good judgment to students, to providing one on one interaction with students, and improving their relationships as teacher and as a student. Something like that. And have you had any student feed Back.
I know you touched on it earlier just about what the students are saying now that they're using it more. I I usually ask after every lesson. I I I do not have any written for now. But whenever we we provide sessions in the with with AI integration, They okay. Could you please provide me a thumbs up? Something like that.
A hard And then they they particularly give me a very positive feedback. That's amazing. Yeah. Thank you. What about you, sir Menon? Another thing that I would like to act is that we're we're making normal.
We're making everything normal when it comes to the use of artificial intelligence, and then I would like them to realize that it's okay to make this AI as a partner in learning, As a tool and not just for cheating. Yes. Something like that. Yeah. Focus back on the learning.
I love that. Mine would be in the area of assessments. And so, most of the classes I'm speaking now as a teacher. In my project management classes, for example, I have an eight week Class where two weeks they start to research about either pain or opportunity that they currently have in their class in their works work situations. And then, they have a six week window to be able to to implement it.
And it's we encourage the students to quantify their results. So, we're using the balanced scorecard as a framework. And so, we will normally start with what's the customer pain. The projects will be around process efficiency or sometimes it's an organizational issue where they cannot get their throughputs done. And so, what I normally just ask them is you They're they're going to be allowed to to use AI because they're non coders.
So, we encourage our students and these are non programmers. These are really managers, middle managers, those that are working in their promotion towards supervision. And then, executive managers. And we want them to be that the reason why we also teach them fluency in AI is so that they can come up with a proof of concept that works. Second, they can test it.
Number three, they can apply it to their specific circumstance and they can bring back the data. And this term, I asked them for lessons learned at least. There is an AI evaluation. And then, there is I I ask them to to challenge or to confirm how AI takes a look at based on the documents that they submit. And so AI is a little bit more tough in terms of the rigor.
And It makes me wonder how best to assess the student given this next Round of AI as a level of because AI is stricter. If you're looking for a rigor. And to me, the burden is really more, I have to understand what the AI platforms the students are using. I have to use all of them. Yes.
Number two, I have to think about how do we engage the discussions. Because if I do not change the way I facilitate the class, we will only have one answer. I love that. We joked at dinner last night where I said I used to have to be as smart as my smartest student. And and the problem is If you do not if I do not research what the platforms are saying, then I'll fall into the trap of believing that that student generation.
And and so I say, okay. I will always challenge. That's what Claude is saying. That is what chat GPT or what Microsoft is saying. They will say, okay.
You got me. And then they said, okay. And then then you flip the question. You ask them to challenge the the AI recommendation. Or, you have to understand which part is can be delegated to AI And which part requires teacher interventions? Critical thinking.
Yeah. Huge piece. I think I would like to add is that currently, most of the users of this AI, Gen AI, are just into this particular fluency, which is just automating everything, making it easier for them. But another thing that we should also consider since it was mentioned, the AI fluency, we're talking about it's not just automation. It's not just automation, just what you see on that slide.
And now we're talking about also augmenting, which is bringing up our teachers in the usual task that they do. And instead of augmenting, meaning that we are instead of doing those tasks, we are now into a more purposeful kind of action and which is to understand more our students, how they learn, and everything about it. And then there's also the third a, which is I think autonomy, and we're not into that particular part, which is we're going to give AI the decision on the things that they that we are to do in the future or what we are to do Next. Something like that. That's a really good point.
I love the authenticity part of it and Yeah. Yeah, the autonomy part of it is big. Very big. Very big. I think that's probably a nice segue to our next question, which is really about policy.
Yes. That fun fun part of the conversation. At each of your institutions, what policies had to be put in place? You know, what sort of guidelines and risk mitigation for students and for staff? How did you tackle that? Can I start with? That is the you're you're going to see on that slide three images. So the first slide has something to do with our thinking is that currently, we're not we go goes beyond the thinking that how are we going to no. No.
If we are going to use AI beyond it to how to use how are we going to use AI? And that particular first picture in that, which is our AI advisory in the absence first of the institutional AI policy, we provided the statement there that AI should be utilized ethically and responsibly, and we provided their guides on how teacher, student, and non teaching staff are gonna use AI focusing on transparency, and, we provided examples there. Now, currently, we are into getting feedback of our final draft in the institutional AI policy, and that's why you could see there are two images which has something to do with the feedback. And the final AI policy, something like that. And more we are focusing more on transparency, requiring them to disclose everything, mandatory disclosure of whether they use AI or not, Something like that. That's great.
Good to see that progress happening as well. Hopefully, by the end of September, we could provide it. Yeah. And what about at Ateneo? Well, the dean, our dean already said that we are the first probably in the university to be able to craft a policy statement around it. There is already a set of aspirations or statements available at the university level.
And so, the principles that we want to as to to live by generally sums up in is sum up in two things. Empowered use and then no use at all. But part of it is really around ethics and responsible use. For content that is not ours. Number two, if we ever partner with AI in school work, we have to identify what it is.
In my classes, for example, you just have to share the prompt. Number two, you do the screen record of how the behavior is and then share your attempts how you got to that portion And, normally, the iterations are about seven to eight times for the first, then they get it right, then they become more fluent and so it becomes they become more proficient. The second area is just academic in in integrity. The third one is basically transparency. Faculty also has to declare that they are using AI because the students will know that, because they can also search.
And the and if the content is generic or the prompt is generic, the results would be similar. The next one is literacy. We still have, we are okay with the mixed reactions towards AI. And this is an ongoing process where we try to understand. Number two, we equip our faculty.
Number three, we share best practices so that they can also be more comfortable with the use of AI. Because generally, this is more like change management. I'm used to approaching a course a specific way and then we're being disrupted by AI. And then it It kinda affects the way I normally handle my force of habit. And so fluency is very important.
Next, we will also allow for flexibility. So that means, it is not something that we impose, but it is something that we can all agree on. Number two, depending on the nature of the job to be done, I think we can also agree what might be the stages of application of AI. The last one would be misinformation. That's also very important.
We have to make sure that this is not just hallucination but responsible use of data and sources that you can rely on. Okay? So the students are also taught that. So so far, the dean was also saying that when he'd run the survey about two years ago, it was eighty percent awareness and usage already. We had an AI forum where there were six hundred people that were in Zoom and four hundred were online. Hundred percent awareness and hundred percent usage.
In two years. In two years. And so, We emphasize that it's our faculty that has to keep up. Yeah. So this is one I think is generations at play.
Generations are more, the earlier and the younger generations are more adept at technology than most senior faculty are. So you talked about flexibility then. What does that mean for your faculty in terms of trying to keep up? How are they keeping up? How are you helping them keep up? Where we're we're being sent. For those that are being sent to workshops and up up upscaling and upgrading There's always an opportunity for us to share the knowledge at the school level. The second one would be, at the department level.
Next, my meeting last Thursday with the EdTech, they started coming up proposing metrics for school leaders like us to to come up with programs related to AI. And so, I've not really thought about it yet because the mandate is fresh. And I had to focus more on Canvas gone. But when we go back to school, I need to discuss it with the dean because there are also there are also needs. Of course.
And then there are also things that we have to do to be able to to encourage the use of AI in the school. And Glenda, you mentioned you've run sessions for other staff and you've helped teach them. Yeah. We provided training, seminars and trainings. And for example, me, I was I I had this certification units in one of the provider abroad in the US.
So that's our first step In In include artificial intelligence in education. We educate some of the faculty members regarding utilization of AI. At first, they are really not into this. They cannot they cannot just start using AI without proper policy. They're they're banking on the available policy in the school because they would want to do it safely.
Of course. Something like that. Great. I add. Yeah.
At at faculty level, we've started scouting for faculty with competencies. Data science e one. AI. And then at the same time, we've started offering and beefing up our electives to include data science and also AI. AI for managers.
So this is not really Very technical, but these are really practical courses that they can use to become more productive in their in their areas of work. That's fabulous. Thank you. Okay. Moving along.
Challenges. What challenges did you face or have you faced in trying to scale? So I've heard from each of you. You've done this in your own classrooms as teachers, But now you've got to put it out there and to a lot of people and probably in not a lot of time. How have you gone about doing that? I think the first thing the first challenge that I had with my team is regarding this particular thing, academic integrity and trust. Okay.
Because we are shifting the focus from concerns about cheating to fostering ethical and responsible AIUs through leadership backed transparent guidelines. So we should have that one. Another thing is that continuous policy and training. So that is one of the challenge because one of the challenges because Not all are available during the time that we provide the schedule for the training, something like that. So These are the things that We need to further Do something.
We need to do something about it so that we could allow our faculty to, you know, to embrace everything about AI. Yeah. Thank you. What about it at to now? How do you scale? I I I see this we see this more as an opportunity. It's very important that our dean will call it air cover.
And it's also important that we get the right People in authority who are fluent, who understand And and who appreciate the value of of artificial intelligence and how it makes How it benefits not just the university, but also the students. Number two, very important about scaling is always to listen to our students. What do the advisory boards say? What do the evaluation say? What is it that the students thirst for? And that's something that we have to provide. Because we have to be more customer focused. This is what industry is calling for.
The other opportunity we're saying we're seeing is generations management. And, surprisingly, Our senior faculty initially will tend to show a little bit of concern because they will always say One funny incident will say, okay, we have a way of solving AI as a challenge in our organization. Let's turn off the Wi Fi. They just say During exam. Off the WiFi during exams and that would solve the problem.
But but yes and no. There's something more proactive around it. That's one. Number two, when we talk about equipping, there's a lot of curiosity around it. How can we at the very basic, how how can you use AI as a substitute for for Google? It's there already.
Second, move it up a little bit to say, okay. How else can we use mini GPTs? The third one we're looking at, that's worth piloting already is agents. But I just wish, we had something like that that can be put into Canvas so that we can actually link the agent. So for now, it's it's not It's coming. I promise.
It's coming. That that's why we like it. And and so those are levels of interactions. And then at the back end side, then you would have to take a look at big data. Then you take a look at analytics, which we got excited about when you said, is gonna be available.
But it's not just at the student level. It's also how people consume data, how much Canvas now provides actual usage, But that's not necessarily going to just stay there. The other one is reframing costs. So the the pandemic has allowed us to understand which technologies work. And which ones are going to be studied a second time.
But because all of us are moving towards the new normal now, they're saying, okay, maybe we might not need this. And so that's a very good question to consider. How do you reframe investments? And then number four is how do we execute? So that we're moving forward. That's why for Atanea Graduate School of Business, the dean has always emphasized, it's agile and lean. We do not solve the entire problems but for as long as we're moving one problem at a time.
We're doing well. That's amazing. I think we still have a bit of time and so I have an interest, a question that's crossed my mind. We've talked a lot about the student experience, your staff experience. I think what I'm curious to know is if you had one wish for where assessment could go in this era of AI and in the fact that we're always thinking about academic integrity, we're always thinking about how to assess authentically, What would be your wish for assessment moving forward with AI? Canvas related? Or Doesn't have to be.
You have an answer? Yeah. I'm thinking. Okay. It's a hard answer. Yeah.
Up we got the spot. There's this concept called Jagged Frontiers by Ethan Molick, where we have to understand what AI is very good at. And so use AI what it is for. So essentially, data Content. If you want Trends to be summarized.
Data AI can do it for you. But speaking to you as a faculty member, I will normally go for higher order Thinking. And so it will, my task is to engage them where AI is not going to give you the con the the context. So AI will give you the content, But AI does not. We will have to provide the context.
Number two, AI will give you the results based on how you ask it. And so the critical thinking for me is, how do you ask intelligent questions so that it gives AI the opportunity to answer it according to the way you want it. And so next When I design an agent for example, the student will solve a case and I know it's going to be AI because they can solve the case now in fifteen minutes. Yes. And so they say, we will report first.
And that was not a a behavior before. When we used assignments for cases, but now they're all volunteering. Number two, they they've started to reduce their use of Canva. They now use Agnes. They use Agnes for slides.
They use Google for slides. Then the third one now is, and then I design another simulator. Okay? I need you to interview this particular guy in the case and tell me if you're if your decisions change. Normally, it's AI against AI. I like that.
So make AI fight against each other. And so that's where you get higher order thinking. And then they say, okay. My answer fits and this is where it doesn't. Now when I do the interview, this is how it differs.
And so it's not AI design. It's still creativity of the faculty member. I like that. And you touched on critical thinking. Yes.
What what I think is that I would like the faculty to think about AI as their assistant, an ed tech assistant in developing whatever assessment that they could have. Now with regard to the student, I think the faculty could think about What are the things that they could Create with the assistance of AI, but will be based on how they prompt this AI. Because as far as I believe The the AI became smart, can be smart based on how you communicate with it. It got smarter with the prompt that you provide this AI. This is based on how I and I this is based on my own experience with regard to The by communication with, for example, Gemini and Chattypiti.
Per perplexity, Something like that. Amazing. So it's it's has something to do with a creation of whatever is required of them to do. And like you said, it's always learning Yeah. Which is exciting.
I wanna thank you both so much for your time. This has been an incredible conversation. I love Yes. Hearing what people are doing in this space. It is moving so quickly.
So it's always exciting and great to hear you're empowering your educators as well. So thank you for sharing with all of Thank you. Good afternoon.
And currently, I I spearheaded on, what's this, the AI task force to to develop things that are needed for our institution. Amazing. Thank you. So, I'm a faculty member of Ateneo Graduate School of Business. I'm with the Department of Operations and IT.
I'm also part of the Quality Assessment Group of AGSB and sitting in as representative for our educational technology group for the university. Wonderful. Thank you both. So we've got some slides that we'll be showing along the way and really just hearing some of the great work that's happening in Ateneo and at Dulles Alipa. So first, for both of you, can you share with us a bit about how you've integrated AI tools into Canvas? You know, things like streamlining your assessments or coding, any automations.
What have been your experiences so far? Do you wanna start us off? Okay. So, our first use case for Ateneo specifically is that we wanted to take a look at the pain of documenting meetings. And so, and our experience is that if you have roughly about half a day of or to one day of meetings. Even my clients would normally complain, it would take them something like three to five days to process. An mp three file.
And so for us, we actually did attempt. We In Zoom, there is a You have a host of apps. And then, for us, we use Fireflies. And then, the documentation can be fed to let's say, Notebook LM or Perplexity or Gemini. So that you can actually feed a specific template around it.
And then the the notes will naturally put in place. And we did that for Ateneo and for our for our planning sessions. And I was the one that documented it as a proof of concept. So every half day, we're able to produce the minutes of the meeting at one o'clock. So, it was thirty minutes to forty five minutes.
You could already produce the output. Instead of three to five days. Yes. And so we also when when Dean talked about documenting, We actually have a use case there where it also goes to Canvas. So the recording, the video, and then the minutes of the meetings are also there.
Amazing. Thank you. Let's have a look at another example that you can tell That's ours. So, we also have a Canvas page that's that's for the strategic planning. And so, each of the parts have their own synopsis.
The second one would be the minutes of the meetings. And then for documentation, can you please move to the next slide? Thank you. There you go. So there's the, it's viewable on YouTube but it's private. Okay.
So that's our use case. Documentation efficiency. So every meeting, the outputs are are released automatically. And your users or your staff can just jump in? Our staff have begun to use this already Amazing. As a workflow.
Thank you. So I think we've got another example now from Glenda. Yes. It's not Is there a video? So my first thinking about the use of AI and integrating it with Canvas is that I would like to think of two possible tasks, and that is regarding the process task as well as the people task. So I would like to think about how will our faculty members could easily be freed up with all of the usual tasks that they created.
So what I did is that I tried to explore things like since I am teaching, for example, computer programming, I decided to use Gemini Gem Gemini AI to develop an app or a web app that could enhance the learning and teaching at the teaching at the learning of my students, and surprisingly, they were able to, you know, grasp all the things that they would like to learn in Canvas. And I in in that particular lesson. So what I did is I utilized several tools, content that is that that is that can be used to to develop contents as well as another tools, which is a web quest creator, and then embedded it in Google site and, put it in Canvas as an external link, something like that. And I really am glad that my students are taking it are are improving, particularly in grasping the learning that they had. And how did they feel about using the AI when you said to them using Gemini? Initially, they are amazed that I am using AI because in the first the first reaction of the students, mom, you are using artificial intelligence.
Why not? So I'm being transparent with you, and I would like you to know that it's okay to use artificial intelligence as long as we're gonna use it ethically and responsibly. I love that. That's a great way to practice things and model for the students. Yeah. It's really lovely.
And so with the faculty. Yeah. I provided training for that kind of actual activity. I love that. I think we have another use case here about embedding QA into Canvas.
Before I discuss my use case number three, I'd like to just add to what Glenda cited as a best practice for her for her school. This term, I innovated my course and I actually taught the students how to write their own mini GPT. So talk to your content. Second would be, they can now they can now practice soft skills. Because using their phones, they can actually interview or get interviewed.
Next, I used to use a lot of visualization apps like Figma before. And then now, the students don't have to use that anymore because they can actually create a Gemini Canvas. To do A website. Number two, an app. The third one would be a simulator, a dashboard which they used to do in Google Data Studio.
But they don't have to do that now. And so, they get feedback not just from their peers but also from AI. Case number three is embedding QA in into Canvas. And so, based on our experiences with Pascu, there was a lot of variations in terms of how the the faculty presented their classes even in the syllabi. We used to have some degree of uniformity.
But when they implemented it in Canvas, we realized that would have been a very good way of improving look and feel. Second, we wanted to also make sure that, it's going to be consistent with PASCO and AUN standards. And so, the dean, had us come together and look for a way to be able to come up with standards. Number two, to check the syllabi and see if they're consistent with accreditation standards. And in this case, can you move to the next slide, please? We designed an internal playbook and then, we merged that with OBE, OBE principles.
And then, the AON standards. And then, assess from the playbook using those two standards, Each syllabi in the university, each syllabus. And so, we we had a Gemini gem. We created that. Next slide, please.
And so, we gave out the prompt to the faculty. So that before this one was sent to quality assurance. So the the process was QA. And then from QA, later on, it it would be vet by a person in the department or in the the quality assurance group to double check the findings, Then, it would go to the department head, then ultimately, the dean. But, the moment we realized that the prompt was stable, we released it to faculty so that even before they would submit it to us, they'd actually have an A report already from AI.
Next slide please. And so, that's what Gemini came out with and then eventually, it made its way into something that you can send out as a Google link. Alright? And so because of because this has to had worked already, we're thinking of actually converting it into an agent anymore already so that they it can process batches. So there will be no human intervention required for this one. Wow.
How much time do you think that has saved? The processing, it took me a while to get this. It took us a while to get this done correctly. And so if you try to take a look at the final output in Gemini, in in Google Sheets, That's the one. This one takes the assessment now takes about less than two minutes. What did it used to take before? Again? Before? Before.
Well, you had to go through. There are something like forty eight questions that a human evaluator has to go through because this is consistent with AUN. And then the second one is you have to double check the the content. And you have to find out and they this is not sequential though. If you try to look for it in the in the syllabus.
So it used to be about an hour, an hour and a half Wow. Per syllabus. Two minutes now. Two minutes now. And then from there I review at my level.
Next slide, please. And then I identify which part is the human portion. Because We did say that the initial AI, result is not necessarily something that you trust. So you have to double check for hallucination. Now, is what we realized with with Gemini.
The more you feed it, there's going to be a time even if I'm on pro, it hallucinates. And so you have to give it time to rest. And then, you have to clean out the process so that it dumps your prior searches or queries. And then, this is a snapshot of the email that I send, which eventually That's my review. The moment it gets to faculty, faculty works on it, puts it on Canvas, submits it to the department chair, another round for QA, and then the dean.
So similar to what I asked Glenda, what was the staff reaction when you said we're doing this with AI? This one is still on test. When you say academic staff, we did have an initial reaction that said, okay. Why is AI there? Okay. Why is AI there? And then second But it's very important also that when you're coming up with An agile and a lean way of doing it. There's not much time between Paasco and AUN accreditations.
No? So, I think we had less than about a couple of years to implement the changes from one standard to another. And so for this, you would like, we would like basically to be more efficient with the with the repetitive tasks of administration. That's a great example. Thank you. So what results what insights are you seeing at the moment in terms of student engagement with your with your use cases or workload or learning outcomes? Because I think that's a really big piece is we're using AI in a lot of ways, but how are we checking if it's actually working? Yes.
Yes. As I think we got overwhelmed with the results, the positive results that does when we start using AI, particularly, for example, in my student in in my class. And that there is an increase in student engagement. They became more collaborative. Yeah.
And they are really into it in every for example, with the webcast that I created, they were into it. They really are performing the task. And even when we go out in in the classroom, they are talking about what happened. Because another thing that I that I included is deciphering codes. Let's say with the use of binary numbers and so on, and then they decipher codes with the use of this created generated web quest, something like that.
So it's really very I I got it I'm happy with the result of of integration of AI in in all of my lessons. And then with regard to the faculty, I think Oh, It has been the result has been no. There are several faculty members that are already into AI, and we provided training regarding the way how I utilize it and that they are seeing it as very promising, and they're thinking about pre preying themselves about the usual routine regarding these processes that they usually created into a more people oriented task, is providing good judgment to students, to providing one on one interaction with students, and improving their relationships as teacher and as a student. Something like that. And have you had any student feed Back.
I know you touched on it earlier just about what the students are saying now that they're using it more. I I usually ask after every lesson. I I I do not have any written for now. But whenever we we provide sessions in the with with AI integration, They okay. Could you please provide me a thumbs up? Something like that.
A hard And then they they particularly give me a very positive feedback. That's amazing. Yeah. Thank you. What about you, sir Menon? Another thing that I would like to act is that we're we're making normal.
We're making everything normal when it comes to the use of artificial intelligence, and then I would like them to realize that it's okay to make this AI as a partner in learning, As a tool and not just for cheating. Yes. Something like that. Yeah. Focus back on the learning.
I love that. Mine would be in the area of assessments. And so, most of the classes I'm speaking now as a teacher. In my project management classes, for example, I have an eight week Class where two weeks they start to research about either pain or opportunity that they currently have in their class in their works work situations. And then, they have a six week window to be able to to implement it.
And it's we encourage the students to quantify their results. So, we're using the balanced scorecard as a framework. And so, we will normally start with what's the customer pain. The projects will be around process efficiency or sometimes it's an organizational issue where they cannot get their throughputs done. And so, what I normally just ask them is you They're they're going to be allowed to to use AI because they're non coders.
So, we encourage our students and these are non programmers. These are really managers, middle managers, those that are working in their promotion towards supervision. And then, executive managers. And we want them to be that the reason why we also teach them fluency in AI is so that they can come up with a proof of concept that works. Second, they can test it.
Number three, they can apply it to their specific circumstance and they can bring back the data. And this term, I asked them for lessons learned at least. There is an AI evaluation. And then, there is I I ask them to to challenge or to confirm how AI takes a look at based on the documents that they submit. And so AI is a little bit more tough in terms of the rigor.
And It makes me wonder how best to assess the student given this next Round of AI as a level of because AI is stricter. If you're looking for a rigor. And to me, the burden is really more, I have to understand what the AI platforms the students are using. I have to use all of them. Yes.
Number two, I have to think about how do we engage the discussions. Because if I do not change the way I facilitate the class, we will only have one answer. I love that. We joked at dinner last night where I said I used to have to be as smart as my smartest student. And and the problem is If you do not if I do not research what the platforms are saying, then I'll fall into the trap of believing that that student generation.
And and so I say, okay. I will always challenge. That's what Claude is saying. That is what chat GPT or what Microsoft is saying. They will say, okay.
You got me. And then they said, okay. And then then you flip the question. You ask them to challenge the the AI recommendation. Or, you have to understand which part is can be delegated to AI And which part requires teacher interventions? Critical thinking.
Yeah. Huge piece. I think I would like to add is that currently, most of the users of this AI, Gen AI, are just into this particular fluency, which is just automating everything, making it easier for them. But another thing that we should also consider since it was mentioned, the AI fluency, we're talking about it's not just automation. It's not just automation, just what you see on that slide.
And now we're talking about also augmenting, which is bringing up our teachers in the usual task that they do. And instead of augmenting, meaning that we are instead of doing those tasks, we are now into a more purposeful kind of action and which is to understand more our students, how they learn, and everything about it. And then there's also the third a, which is I think autonomy, and we're not into that particular part, which is we're going to give AI the decision on the things that they that we are to do in the future or what we are to do Next. Something like that. That's a really good point.
I love the authenticity part of it and Yeah. Yeah, the autonomy part of it is big. Very big. Very big. I think that's probably a nice segue to our next question, which is really about policy.
Yes. That fun fun part of the conversation. At each of your institutions, what policies had to be put in place? You know, what sort of guidelines and risk mitigation for students and for staff? How did you tackle that? Can I start with? That is the you're you're going to see on that slide three images. So the first slide has something to do with our thinking is that currently, we're not we go goes beyond the thinking that how are we going to no. No.
If we are going to use AI beyond it to how to use how are we going to use AI? And that particular first picture in that, which is our AI advisory in the absence first of the institutional AI policy, we provided the statement there that AI should be utilized ethically and responsibly, and we provided their guides on how teacher, student, and non teaching staff are gonna use AI focusing on transparency, and, we provided examples there. Now, currently, we are into getting feedback of our final draft in the institutional AI policy, and that's why you could see there are two images which has something to do with the feedback. And the final AI policy, something like that. And more we are focusing more on transparency, requiring them to disclose everything, mandatory disclosure of whether they use AI or not, Something like that. That's great.
Good to see that progress happening as well. Hopefully, by the end of September, we could provide it. Yeah. And what about at Ateneo? Well, the dean, our dean already said that we are the first probably in the university to be able to craft a policy statement around it. There is already a set of aspirations or statements available at the university level.
And so, the principles that we want to as to to live by generally sums up in is sum up in two things. Empowered use and then no use at all. But part of it is really around ethics and responsible use. For content that is not ours. Number two, if we ever partner with AI in school work, we have to identify what it is.
In my classes, for example, you just have to share the prompt. Number two, you do the screen record of how the behavior is and then share your attempts how you got to that portion And, normally, the iterations are about seven to eight times for the first, then they get it right, then they become more fluent and so it becomes they become more proficient. The second area is just academic in in integrity. The third one is basically transparency. Faculty also has to declare that they are using AI because the students will know that, because they can also search.
And the and if the content is generic or the prompt is generic, the results would be similar. The next one is literacy. We still have, we are okay with the mixed reactions towards AI. And this is an ongoing process where we try to understand. Number two, we equip our faculty.
Number three, we share best practices so that they can also be more comfortable with the use of AI. Because generally, this is more like change management. I'm used to approaching a course a specific way and then we're being disrupted by AI. And then it It kinda affects the way I normally handle my force of habit. And so fluency is very important.
Next, we will also allow for flexibility. So that means, it is not something that we impose, but it is something that we can all agree on. Number two, depending on the nature of the job to be done, I think we can also agree what might be the stages of application of AI. The last one would be misinformation. That's also very important.
We have to make sure that this is not just hallucination but responsible use of data and sources that you can rely on. Okay? So the students are also taught that. So so far, the dean was also saying that when he'd run the survey about two years ago, it was eighty percent awareness and usage already. We had an AI forum where there were six hundred people that were in Zoom and four hundred were online. Hundred percent awareness and hundred percent usage.
In two years. In two years. And so, We emphasize that it's our faculty that has to keep up. Yeah. So this is one I think is generations at play.
Generations are more, the earlier and the younger generations are more adept at technology than most senior faculty are. So you talked about flexibility then. What does that mean for your faculty in terms of trying to keep up? How are they keeping up? How are you helping them keep up? Where we're we're being sent. For those that are being sent to workshops and up up upscaling and upgrading There's always an opportunity for us to share the knowledge at the school level. The second one would be, at the department level.
Next, my meeting last Thursday with the EdTech, they started coming up proposing metrics for school leaders like us to to come up with programs related to AI. And so, I've not really thought about it yet because the mandate is fresh. And I had to focus more on Canvas gone. But when we go back to school, I need to discuss it with the dean because there are also there are also needs. Of course.
And then there are also things that we have to do to be able to to encourage the use of AI in the school. And Glenda, you mentioned you've run sessions for other staff and you've helped teach them. Yeah. We provided training, seminars and trainings. And for example, me, I was I I had this certification units in one of the provider abroad in the US.
So that's our first step In In include artificial intelligence in education. We educate some of the faculty members regarding utilization of AI. At first, they are really not into this. They cannot they cannot just start using AI without proper policy. They're they're banking on the available policy in the school because they would want to do it safely.
Of course. Something like that. Great. I add. Yeah.
At at faculty level, we've started scouting for faculty with competencies. Data science e one. AI. And then at the same time, we've started offering and beefing up our electives to include data science and also AI. AI for managers.
So this is not really Very technical, but these are really practical courses that they can use to become more productive in their in their areas of work. That's fabulous. Thank you. Okay. Moving along.
Challenges. What challenges did you face or have you faced in trying to scale? So I've heard from each of you. You've done this in your own classrooms as teachers, But now you've got to put it out there and to a lot of people and probably in not a lot of time. How have you gone about doing that? I think the first thing the first challenge that I had with my team is regarding this particular thing, academic integrity and trust. Okay.
Because we are shifting the focus from concerns about cheating to fostering ethical and responsible AIUs through leadership backed transparent guidelines. So we should have that one. Another thing is that continuous policy and training. So that is one of the challenge because one of the challenges because Not all are available during the time that we provide the schedule for the training, something like that. So These are the things that We need to further Do something.
We need to do something about it so that we could allow our faculty to, you know, to embrace everything about AI. Yeah. Thank you. What about it at to now? How do you scale? I I I see this we see this more as an opportunity. It's very important that our dean will call it air cover.
And it's also important that we get the right People in authority who are fluent, who understand And and who appreciate the value of of artificial intelligence and how it makes How it benefits not just the university, but also the students. Number two, very important about scaling is always to listen to our students. What do the advisory boards say? What do the evaluation say? What is it that the students thirst for? And that's something that we have to provide. Because we have to be more customer focused. This is what industry is calling for.
The other opportunity we're saying we're seeing is generations management. And, surprisingly, Our senior faculty initially will tend to show a little bit of concern because they will always say One funny incident will say, okay, we have a way of solving AI as a challenge in our organization. Let's turn off the Wi Fi. They just say During exam. Off the WiFi during exams and that would solve the problem.
But but yes and no. There's something more proactive around it. That's one. Number two, when we talk about equipping, there's a lot of curiosity around it. How can we at the very basic, how how can you use AI as a substitute for for Google? It's there already.
Second, move it up a little bit to say, okay. How else can we use mini GPTs? The third one we're looking at, that's worth piloting already is agents. But I just wish, we had something like that that can be put into Canvas so that we can actually link the agent. So for now, it's it's not It's coming. I promise.
It's coming. That that's why we like it. And and so those are levels of interactions. And then at the back end side, then you would have to take a look at big data. Then you take a look at analytics, which we got excited about when you said, is gonna be available.
But it's not just at the student level. It's also how people consume data, how much Canvas now provides actual usage, But that's not necessarily going to just stay there. The other one is reframing costs. So the the pandemic has allowed us to understand which technologies work. And which ones are going to be studied a second time.
But because all of us are moving towards the new normal now, they're saying, okay, maybe we might not need this. And so that's a very good question to consider. How do you reframe investments? And then number four is how do we execute? So that we're moving forward. That's why for Atanea Graduate School of Business, the dean has always emphasized, it's agile and lean. We do not solve the entire problems but for as long as we're moving one problem at a time.
We're doing well. That's amazing. I think we still have a bit of time and so I have an interest, a question that's crossed my mind. We've talked a lot about the student experience, your staff experience. I think what I'm curious to know is if you had one wish for where assessment could go in this era of AI and in the fact that we're always thinking about academic integrity, we're always thinking about how to assess authentically, What would be your wish for assessment moving forward with AI? Canvas related? Or Doesn't have to be.
You have an answer? Yeah. I'm thinking. Okay. It's a hard answer. Yeah.
Up we got the spot. There's this concept called Jagged Frontiers by Ethan Molick, where we have to understand what AI is very good at. And so use AI what it is for. So essentially, data Content. If you want Trends to be summarized.
Data AI can do it for you. But speaking to you as a faculty member, I will normally go for higher order Thinking. And so it will, my task is to engage them where AI is not going to give you the con the the context. So AI will give you the content, But AI does not. We will have to provide the context.
Number two, AI will give you the results based on how you ask it. And so the critical thinking for me is, how do you ask intelligent questions so that it gives AI the opportunity to answer it according to the way you want it. And so next When I design an agent for example, the student will solve a case and I know it's going to be AI because they can solve the case now in fifteen minutes. Yes. And so they say, we will report first.
And that was not a a behavior before. When we used assignments for cases, but now they're all volunteering. Number two, they they've started to reduce their use of Canva. They now use Agnes. They use Agnes for slides.
They use Google for slides. Then the third one now is, and then I design another simulator. Okay? I need you to interview this particular guy in the case and tell me if you're if your decisions change. Normally, it's AI against AI. I like that.
So make AI fight against each other. And so that's where you get higher order thinking. And then they say, okay. My answer fits and this is where it doesn't. Now when I do the interview, this is how it differs.
And so it's not AI design. It's still creativity of the faculty member. I like that. And you touched on critical thinking. Yes.
What what I think is that I would like the faculty to think about AI as their assistant, an ed tech assistant in developing whatever assessment that they could have. Now with regard to the student, I think the faculty could think about What are the things that they could Create with the assistance of AI, but will be based on how they prompt this AI. Because as far as I believe The the AI became smart, can be smart based on how you communicate with it. It got smarter with the prompt that you provide this AI. This is based on how I and I this is based on my own experience with regard to The by communication with, for example, Gemini and Chattypiti.
Per perplexity, Something like that. Amazing. So it's it's has something to do with a creation of whatever is required of them to do. And like you said, it's always learning Yeah. Which is exciting.
I wanna thank you both so much for your time. This has been an incredible conversation. I love Yes. Hearing what people are doing in this space. It is moving so quickly.
So it's always exciting and great to hear you're empowering your educators as well. So thank you for sharing with all of Thank you. Good afternoon.