On March 27, Geisel School of Medicine professor Nicholas Jacobson published the results of the first clinical trial evaluating the effectiveness of a generative artificial intelligence psychotherapy chatbot.
The study, which tested the effects of “Therabot” on symptoms of anxiety, depression and eating disorders, showed significant improvements among participants — comparable to outcomes seen in traditional in-person therapy. Jacobson sat down with The Dartmouth to discuss the potential of AI in mental health care, ethical considerations in digital therapy and what’s next for Therabot.
What inspired you to pursue the study of therapy chatbots, and what made this trial unique?
NJ: My lab and I focus on trying to create scalable, personalized treatments — some specifically for mental health. What motivates all of our work is that mental health needs are exceedingly common. In any given year about one in five people will suffer from anxiety or depressive disorders. Most of the folks that would actually meet criteria for anxiety or depressive disorders don’t receive treatment for it — not a single session in a given year with a mental health provider of any kind.
There are far fewer providers than could possibly ever treat the total number of people within the traditional care system. These digital treatments could potentially directly scale to this large group of people that go untreated.
This is the first trial, not only of our system — which we’ve spent over five and a half years developing — but also the first trial of any generative AI based psychotherapy. I think it is really important to study if this can be effective and safe.
Were there any surprising findings or outcomes of the trial?
NJ: I wouldn’t say surprising to us because we’ve been testing these models so iteratively, but I think what is surprising for the field is how effective they could be. We tested a randomized control trial, 210 people, and we looked at how much change the treatment group had versus the waitlist control group who were free to go out and try other forms of therapy if they wanted to. Those who had access to Therabot experienced these really large reductions in anxiety, depression and eating disorders. These effect sizes and reduction in symptoms are in the magnitude of what you would see with a gold standard dose of in-person psychotherapy delivered across a longer span of time like 16 weeks.
We also measured the connection between the client and their bot through the same measure that you would give people that talk to human providers, and folks rated it nearly identical. So that suggests to us that people are demonstrating really strong bonds with Therabot.
What might the trial’s results suggest for the future role of AI in mental health care?
NJ: This system will only get better. It’s really exciting and means we could potentially deliver this personalized dynamic care to large populations in the future.
A generalization that a lot of people might make is that because this is the first study it might generalize to other AI mental health products. That is not the case. These results are very specific to the system that we designed rigorously and specifically for this purpose.
What might you say to someone who may be skeptical about talking to an AI system for therapy?
NJ: This doesn’t need to be for everybody. I do think there’s a large number of people that probably will benefit from this.
What is next for Therabot, and what challenges might you anticipate for scaling or deploying the tool more broadly?
NJ: One of the next things that we want to do is scale it up to do a head-to-head trial against human providers, largely because we already think we’re in a spot where we’re probably not inferior to them. We want to gather that evidence to get clearance through the FDA to make Therabot more widely available and continue to study its safety and efficacy.
What was the most rewarding part of working on this project for you?
NJ: If you think about the amount of people that were in the eight-week trial, 210 people got access to treatment. This treatment actually seems like it’s pretty darn good. And so, in and of itself, thinking about the individuals in the trial is inherently really rewarding. But thinking about scaling that to more people, that’s something that I’m really grateful to have the opportunity to work on, and it’s really exciting that it seems to be working out so well.
This interview has been edited for clarity and length.
Alesandra Gonzales is a reporter, photographer, and videographer for The Dartmouth from south Texas, majoring in psychology with a minor in film. Outside of The D, she likes to workout, watch The Ranch, and do EMS work. She is a licensed EMT in three states.
