Improving Surgical Residents' Skills With Video-Based Practice
UCSF School of Medicine used video-based assessments and peer feedback to increase student competence.
UCSF School of Medicine's Challenge:
The University of California, San Francisco is one of the country’s top medical schools — along with many of its cohorts, it leads the charge and is at the forefront of medical procedures, processes, and techniques — as a steward of developing future doctors and surgeons, training these medical professionals is critical. These skilled minds and hands will be tasked with treating thousands of patients who look to them for expert advice, treatment, and care. This is not something that Dr. Huey Chern, Associate Professor of Surgery and Associate Director of the Surgical Skills Center at UCSF, takes lightly. Along with her team, Dr. Chern wanted to transform the unnecessarily tedious and manual process of enabling surgical residents to interactively simulate critical surgical techniques fundamental to their skill set as future surgeons.
The UCSF School of Medicine’s surgical “bootcamp”, which typically runs six to eight weeks, is designed to provide evidence-based basic motor surgical skills required of their training. Given the limited timeframe and the magnitude of the scope, Practice exercises are introduced from the very beginning of their learning curriculum, as it helps reinforce and significantly increase those hard skills that the residents might not have engaged with in a while, but are essential as they graduate to the next phase of their training as world-class surgeons.
Although Dr. Huey Chern and her team are well-acquainted with Practice’s video-based solution today, before UCSF began using Practice, the process to evaluate residents’ basic surgical skills was far more manual and labor-intensive. “We were using different modalities to accomplish what we can easily do with just Practice today. As part of their home curriculum portion, residents would have to use a flip camera and videotape themselves completing the exercise. If they struggled with the technology or were not as tech-savvy, that was an added layer of complexity,” noted Chern of the process for residents to complete and submit videos of them performing surgical tasks. Added intricacies were introduced as recorded submissions needed to be distributed to geographically dispersed faculty. The only technically feasible solution was to transfer recorded submissions onto USB drives and arrange for delivery to faculty. Then came the additional complexity of gathering, collating, and comprehensively analyzing the data of the faculty assessments as they were submitted. This was painstakingly manual and often spanned lengthy periods of time due to faculty time and schedule commitments — all in all, the overall process was extremely time, labor, and resource-heavy.
Dr. Chern and her team needed a solution that would provide:
- A logical, easy-to-use user experience
- An accessible solution that required little to no manual maintenance
- An uncomplicated approach for faculty to deliver assessments
- An encyclopedic option for gathering assessments — across multiple stakeholders
UCSF School of Medicine's Approach
“When you are learning, practicing, or perfecting motor skills, it’s really important that you have the option to have your hands free so you can pause/stop/start as you need. It’s something we tried on our own but since our learning is evidence-based and to have a platform that supports what we need in the way we need — it’s a partnership with Practice that makes perfect sense for the supplemental portion of our surgical residency program,” said Dr. Chern. The technical skill levels are tested with pre- and post-evaluations on the Practice platform, leveraging knot tying and suturing checklists, which provide scoring criteria for basic surgical knot tying and suturing and can be used by novice to experienced evaluators. Its use supports peer assessment of performance in a surgical skills laboratory setting. The course activities in the bootcamp are interactive simulation cases to emphasize care of surgical patients, and skills sessions focus on knot tying and suturing, which are reinforced with video assignments captured through Practice exercises.
By leveraging Practice’s patented pedagogical design, based on deliberate practice and social learning theories, Chern and other faculty instructors built courses and modules that evaluate technical skills essential to surgical techniques. In the initial stage, residents watch a video of a specific surgical task completed by experts, then record and upload a video of themselves performing the same task. Next, in the peer assessment stage, feedback is crowdsourced from peers using an assessment rubric — based on the knot tying and suturing checklists — to provide candid yet constructive feedback. The final results stage facilitates the receipt of feedback, scores, and additional information for residents. This provides an opportunity for learners to internalize the feedback, self-reflect on their performance, and formulate a plan for improvement.
There is no doubt in Dr. Chern’s mind that using Practice has impacted her team’s ability to better facilitate the administration of the program, but the true highlight comes from feedback received from residents. Based on their feedback, residents are self-motivated in their practice and learning due to:
- Deliberate practice - practice skills and receive immediate, candid feedback
- Peer-to-peer learning - watch peer submissions to improve their techniques and skills
- Self-reflection - opportunity to view their own submissions allows learners to catch nuances that might not be captured through other modalities
How UCSF School of Medicine Measures Success
- Retention and reinforcement of key concepts - Learners who enrolled, completed, and graduated to the next phase of their residency program were able to successfully maintain their skills into their residency program. The Practice exercises, preparatory courses, and bootcamp program provide a substantial head start for the residents.
- Scores increased 20-30%, on average, pre- and post-boot camp evaluation
- Fluid learning - Learners were able to easily engage with the Practice solution to practice effective surgical skills, while Practice also addressed the program’s challenges surrounding ease of use, support of deliberate practice, and improved timeliness of feedback. Over 50% of learners spent at least 10-30 minutes practicing skills before submitting their videos, and over 80% re-recorded their submission, revealing that learners actively engaged with the educational Practice exercises and spent additional time practicing.
- Increased learner confidence and engagement - As residents actively engaged with Practice, their competence and confidence in knot tying and suturing continually improved.
- Motivated learning - Practice has empowered Dr. Chern to realize the value that a solution like Practice can provide to the broader learning opportunities where residents have the ability to engage in active learning by seeing, practicing, and receiving practical feedback that helps them improve their skill set. Dr. Chern also sees the possibilities in creating a library where residents can leverage content based on speciality rotation, used on a global basis. Additionally, Dr. Chern envisions using Practice for remediation use cases, empowering learners to improve their skills — without being constricted by time, physical space, or distance allows for “this type of deliberate and flexible learning to be leveraged and used for multiple use cases and many types of learning,” said Dr. Huey Chern.
To learn more about how UCSF School of Medicine is preparing tomorrow’s surgeons to respond with technology-first, impactful hard and soft skill peer-based learning with Practice, contact us at [email protected] or 877.576.5364.