Dr. Werneburg discusses research evaluating machine learning algorithms’ potential

Glenn T. Werneburg, MD, PhD

In this interview, Glenn T. Werneburg, MD, PhD, discusses the abstracts "Machine learning algorithms demonstrate accurate prediction of objective and patient-reported response to botulinum toxin for overactive bladder and outperform expert humans in an external cohort” and "Machine learning algorithms predict urine culture bacterial resistance to first line antibiotic therapy at the time of sample collection,” which were presented at the Society of Urodynamics, Female Pelvic Medicine & Urogenital Reconstruction 2024 Winter Meeting in Fort Lauderdale, Florida. Werneburg is a urology resident at Glickman Urological & Kidney Institute at Cleveland Clinic, Cleveland, Ohio.

Please discuss the SUFU abstract "Machine learning algorithms demonstrate accurate prediction of objective and patient-reported response to botulinum toxin for overactive bladder and outperform expert humans in an external cohort."

We'll start with our efforts regarding overactive bladder. Overactive bladder is a common and costly condition. It includes symptoms of urgency, frequency, and urinary incontinence. Patients who don't respond to behavioral therapies or medical management generally require a third-line therapy. That third-line therapy is usually either sacral neuromodulation or onabotulinumtoxinA injection into the bladder. For these patients, many of them would do well with either of these therapies, but a subset would not respond to them. Our goal was to identify which patients would respond and which patients would not respond to these different treatments for medically refractory overactive bladder. To do so, we developed machine learning algorithms. We assembled a team, of course with urologists, but also with quantitative mathematicians, who have lots of experience making predictions regarding stock market fluctuations and things like this. And we developed these algorithms to predict the responder vs the nonresponder status of our patients. We defined how a patient responded based on whether they had a reduction in urge incontinence episodes following the treatment, and whether they perceived an improvement in their symptoms following the therapy. What we found was that we were able to identify patients who responded vs didn't respond to the treatment with a high degree of accuracy. The accuracy surpassed even that of human experts and other standard algorithms. We found this really encouraging. The algorithms even held up in an external validation set, so when we trained the algorithms on 1 data set, and then validated them on a very different group of patients, they still were able to predict who would respond vs not respond to the treatment.

Please discuss the SUFU abstract "Machine learning algorithms predict urine culture bacterial resistance to first line antibiotic therapy at the time of sample collection."

Our other study was a very different application. We were looking at what antibiotics would be optimally suited for which particular patients. One of the major threats to humanity right now is antibiotic resistance. The World Health Organization tells us it's 1 of the top 10 greatest threats. One of the main drivers for antibiotic resistance is overuse and misuse of antibiotics. One of the issues in urology is that a urine culture, which is commonly used to diagnose urinary tract infection, takes 3 days to results. In that time period, the clinician provides the best treatment based on his or her clinical judgment. But we know from the literature that about 30% of the time, this isn't the optimal therapy, and this therapy needs to be changed once the final urine culture results come back. So we set out with a goal to optimally predict which antibiotic would be most suitable for which patient at the time the urine culture is ordered, so it equates to 3 days prior to the final results. The idea was that if we target the therapy very specifically, we can improve our antibiotic usage and improve the time to the resolution of symptoms for our patients. It's beneficial at the patient level and beneficial at the population level. We developed a series of algorithms. We focused on only the most clinically relevant antibiotics. We trained these algorithms on a large data set of our patients - about 6.6 million cases were used. And then we validated them on another set of patients and we tested how they would perform in terms of predicting which cultures would be sensitive and which cultures would be resistant to these different antibiotics. They held up very well. The algorithms' accuracy was really good, and it held up even in an external data set. We find this really encouraging. We have some more validation to do. We're really looking forward to being able to implement these algorithms clinically to improve patient care and also to improve our antibiotic stewardship and reduce our selection for antibiotic resistance at the population level.

What are some next steps regarding machine learning algorithms and urologic conditions?

Regarding these studies, specifically, we have some additional efforts in terms of validation. We want to validate these prospectively and establish how generalizable they are on different patient subsets. Those are aspects of the work that are ongoing right now. But thinking more broadly, I think that these 2 cases illustrate that artificial intelligence and machine learning is really poised to help us define which treatment for which patient is the most suitable. It's a large step forward toward precision medicine. In urology alone, we have many, many cases wherein we have different treatments that might work for different patients. But we don't know which treatment is best for which patient. We often have some idea; we have pretty good clinical judgment and lots of good data with clinical trials. But still, we arrive at a situation where there are different therapies that may work similarly well, but we don't know which patient is best suited for which therapy and which therapy is best suited for which patient. AI can help us with this. Just thinking about prostate cancer, for example. We have many different treatment modalities. We have prostatectomy, different types of radiation therapy. We now have different modalities for focal therapy, where only a portion of the prostate is treated. We have active surveillance, which is monitoring the condition with the idea to intervene when and if needed, and algorithms such as this can help us identify which modality will be best for which patient. We're really enthusiastic on the overactive bladder end and on the antibiotic resistance end, but much more broadly, it's set up to help us counsel our patients better and help us to improve outcomes for our patients as well.

What is the take-home message for the practicing urologist?

We're very much looking forward to being able to clinically implement these algorithms, both on the OAB side and the antibiotic resistance side. For the OAB, if we can identify who would best respond to sacral neuromodulation, and who would best respond to onabotulinumtoxinA injection, then we're helping patients achieve an acceptable outcome faster. We're improving their incontinence or their urgency in a more efficient way. So we're enthusiastic about this. Once we can implement this clinically, we believe it's going to help us in this way. It's the same for the antibiotic resistance algorithms. When we can get these into the hands of clinicians, we'll be able to have a good suggestion in terms of which is the best antibiotic to use for this patient at this time. And in doing so, we hope to be able to improve our antibiotic stewardship. Ideally, we would use an antibiotic with the narrowest spectrum that would still cover the infecting organism, and in doing so, it reduces the risk for resistance. So if that same patient requires an antibiotic later on in his or her lifetime, chances are—and we'd have to determine this with data and experiments—if we're implementing a narrower spectrum antibiotic to treat an infection, they're going to be less likely to be resistant to other antibiotics down the line.