With the support of his team, medical education associate professor Dr. Thomas Thesen created AI Patient Actor — an app that simulates how a human patient would react to medical treatment and provides students with individualized feedback. Typically, medical students practice diagnosing human actors, also called standardized patients, but AI Patient Actor offers an alternative option that can be utilized anytime and anywhere. The Dartmouth spoke to Thesen to gain more insight into the creation, use and future of AI Patient Actor.
What does AI Patient Actor look like in a classroom setting? What kinds of skills does it teach second-year medical students?
TT: When medical students are learning how to diagnose patients, they interact first with standardized patients, who are patient actors. These patient actors are trained to simulate certain conditions. Students learn to ask the right questions to elicit responses from the patient, and our app simulates that encounter. The standardized patient prepares students for that encounter, which is very resource intensive. However, if they use the app, it can really prepare students optimally for this interaction. It improves their patient communication skills and diagnostic skills in a safe environment. They also receive individualized feedback from the app about how they did.
What are the benefits of using AI to act as a patient instead of human actors?
TT: The app allows students to practice at their own pace. It delivers individualized feedback that allows students to go back and try again. That is not possible in the clinic. With standardized patients, time is limited. The app allows students to explore this skill more, hone their skills, receive feedback and become better communicators who can interact better with their patients.
What steps did you take to turn your idea into a prototype and that prototype into a reality? What initially inspired you to pursue creating an AI patient?
TT: I first created the prototype on my laptop with Python, and I was really amazed by how well it worked right from the get-go. I had no experience deploying it to the web, so I worked with Simon Stone, a data scientist from the Dartmouth library. He has built a web interface that allows the app to be shared with our medical students at the Geisel School of Medicine and beyond with students at other medical schools.
I understand that there is an evaluation rubric programmed into the app. What kind of feedback does the app give students?
TT: The rubric is built based upon the standardized patient rubric used at Geisel. It covers many different facets of what is important for a physician to do in a patient encounter. It measures appropriateness of communication, building rapport, empathy, asking the right questions about history, present illness, symptoms and also developing questions that allow for the development of the diagnoses.
How is the validation study that you are conducting going? Why is it so important to conduct a study like this to look for built-in, implicit biases?
TT: Generative AI and large language models are relatively new to medical education. We really need to thoroughly evaluate their performance. Large language models are trained based upon large amounts of data available publicly, including lots of medical textbooks. Historically, medical textbooks have been biased. At Geisel in the last few years, we’ve tried to eliminate this bias from our teaching, but it’s a slow process that requires constant vigilance. We can assume that the textbooks from 10 years ago contain this bias. If they have been ingested by the large language model, it is safe to assume that they would be included in the responses from the large language model. Even though there has been fine tuning done by companies like OpenAI, these biases still might seep through, so we need to check that.
Can you elaborate on your expansion to include Spanish and Swahili features?
TT: At Geisel, the medical school has a Spanish pathway, where Spanish-speaking students who want to work with Spanish-speaking populations in the future learn how to best do that. Unfortunately, recruiting Spanish-speaking patient actors in the Upper Valley has been difficult, so this is another opportunity for students to practice their medical Spanish speaking skills in a safe environment. We collaborate with a medical university in Nairobi. We want to roll the app out there in a different cultural context. We worked with the medical educators there to adapt the patient scripts and patient cases to the local environment. Expanding into other languages is a relatively easy function to add, so we hope to be able to do that.
How does the newly added speaking feature enhance medical students’ experiences using AI Patient Actor?
TT: The speaking feature makes it much easier to ask the patient questions, simulating a more realistic patient encounter. It also helps students get patient responses. We haven’t really worked out how to give the patient emotions, but we hope to be able to do that in the future.
As you’ve previously stated, there are concerns that AI “will take the human side out of learning.” How would you address these concerns?
TT: I think the evolution of AI in education promises a more collaborative learning environment, wherein AI tools work alongside educators to augment the learning experience, ultimately producing more competent, well-rounded healthcare professionals. With the advancement of AI, we are moving towards more individualized education in medicine. AI-based tools, such as our app, are set to provide learning paths that cater to the diverse educational requirements and pace of each student.
How do you see AI playing a role in the future of education, specifically in the medical field?
TT: As AI continues to evolve, I anticipate a transformative shift in medical education, where AI-driven tools, like our simulated patient app, will enable personalized learning pathways, adapting to each student’s unique history, educational needs and learning pace.
What are your goals for the AI Patient Actor? Are there any other features you’re hoping to include?
TT: We are hoping to make the app available worldwide to medical schools. We hope to focus more on the feedback feature as well.
