Radical prostatectomy outcomes measured poorly by administrative data

Key Points

Chicago-Recent, well-publicized reports have used large administrative datasets, such as the Surveillance, Epidemiology, and End Results Medicare database, to compare functional outcomes following radical prostatectomy. However, it appears that this type of data correlates very poorly with validated questionnaire data, the gold standard for outcome assessment.

Research presented at the 2010 World Congress of Endourology and SWL compared the outcomes from validated questionnaires with their corresponding administrative claims data after prostatectomy.

"The correlation between administrative data and validated outcome measures was very poor," said first author Matthew Tollefson, MD, a urologic oncology fellow at the Mayo Clinic in Rochester, MN, working with Igor Frank, MD, and colleagues. "Functional outcomes research using administrative data may be inaccurate, and its use should be discouraged."

Despite these weaknesses, several publications have used administrative data to assess outcomes after prostatectomy, most notably a study by Jim Hu, MD, MPH, and colleagues (JAMA 2009; 302:1557-64). Those investigators found that after surgery, men undergoing minimally invasive radical prostatectomy were more likely to be diagnosed with erectile dysfunction and incontinence than men undergoing open prostatectomy. Dr. Tollefson says he believes that these findings may not accurately represent the course of men after open or minimally invasive surgery in light of what he and colleagues found.

The Mayo Clinic study assessed the claims of 562 patients who underwent either open or robotic radical prostatectomy and had follow-up 1 to 3 years after surgery. Urinary function was measured using the Expanded Prostate Cancer Index Composite (EPIC), and erectile function was measured with the Sexual Health Inventory for Men (SHIM). These results were then compared to the administrative diagnoses of urinary incontinence or erectile dysfunction.

Sensitivity, specificity, and the kappa correlation coefficient were calculated comparing the administrative diagnoses submitted for reimbursement with the reference standard from the questionnaires. Various definitions of incontinence were used: the use of any pads, more than two pads, or an EPIC urinary function domain less than 80. Erectile dysfunction was defined as a SHIM score of less than 12.

Sensitivity was low for both incontinence and erectile dysfunction, ranging from 11% to 59%, depending on the definitions used. The kappa correlation coefficient, which ranges from 0 (no correlation) to 1.0 (perfect correlation), was used to measure how well the two outcomes agreed with each other. Kappa correlation coefficients of less than .2 represent poorly correlated outcomes.

For incontinence, coefficients ranged from .08 to .19, with worsening definitions of incontinence more strongly correlated. Similarly, coefficients for erectile dysfunction ranged from .07 to .16. Specificity was high for incontinence (93% to 98%) but not for erectile dysfunction (52% to 60%).

"Administrative data was associated with high rates of both false-positive and false-negative results, and it was consistently poorly correlated with multiple definitions of both incontinence and erectile dysfunction, " Dr. Tollefson added.

He stressed the importance of using validated questionnaires both before and after prostate cancer treatment, as they are the preferred mechanism for comparing outcomes between treatment modalities or surgeons. Articles utilizing administrative data for functional outcomes after radical prostatectomy should be viewed in light of other research, he added.