Commentary
Video
Study assesses ChatGPT’s knowledge on vasectomy
Author(s):
"What we found is that overall, it was relatively effective at providing advice to patients on these pretty commonly encountered questions," says Sharath Reddy, MD.
In this video, experts from Yale School of Medicine in New Haven, Connecticut highlight the background and notable findings from the study, “High Capability of ChatGPT to Answer Common Patient Questions on Vasectomy Reviewed by Sexual Medicine Specialists” which was presented at the 24th Annual Fall Scientific Meeting of the Sexual Medicine Society of North America in San Diego, California.
Video Transcript:
Sharath Reddy, MD: Some of the other research we did was utilizing ChatGPT, which is a new artificial intelligence tool that's been in the news a lot recently. Specifically, we wanted to see whether this type of tool could be used to educate patients and counsel them, for instance, on the procedure of vasectomy. So specifically, we came up with a list of commonly asked patient questions on the topic of vasectomy, could be related to post-operative care, could be related to the timing of vasectomy, and we were able to ask these to ChatGPT. With the results that it provided, we then had andrologists [and] sexual medicine specialists go through them and rate them and their accuracy and whether they were in line with actual clinical recommendations. What we found is that overall, it was relatively effective at providing advice to patients on these pretty commonly encountered questions. There definitely were some considerations for surgical choice and some of the other issues for post-operative complications that it could do a better job with, which goes to show that the role of a sexual medicine specialist is still there, but [also shows] the potential for utilizing these types of tools in the future to help with patient counseling and education down the road.
Stanton Honig, MD: While we didn't really compare Google searches to ChatGPT, the census that we get is that when people go to Dr. Google, they get a wide range of questions and answers that many times don't make sense. What we did find was a ChatGPT was actually really good at answering many questions relating to these topics.
This transcription has been edited for clarity.
