Podium Session

Poster, Podium & Video Sessions

PD58-11: The Fragility of Statistically Significant Results from Randomized Trials in Urology

Monday, May 15
2:40 PM - 2:50 PM
Location: BCEC: Room 161

Presentation Authors: Vikram Narayan*, Minneapolis, MN, Shreyas Gandhi, Hamilton, Canada, Kristin Chrouser, Minneapolis, MN, Nathan Evaniew, Hamilton, Canada, Philipp Dahm, Minneapolis, MN

Introduction: Randomized controlled trials (RCTs) have the potential of providing high quality evidence to inform clinical practice. This quality relies not only on safeguards against bias, but also on statistical power. In this study, we determined the Fragility Index of urological RCTs as a novel metric (Walsh M et al, JCE 2014) to assess the robustness of statistically significant results.

Methods: Statistical significance implies that an observed event is unlikely to occur by chance alone. The fragility index is defined as the minimum number of patients in an arm of a trial whose status would have to change from "non-event" to "event" in order to turn a statistically significant result into a non-significant one. All RCTs published in the 4 major urology journals between 2011-2015 were identified. We excluded studies not reporting dichotomous outcomes, as well as those with non-significant results and non-parallel designs. We applied the Fisher exact test to determine fragility index values.

Results: 332 RCTs were identified, and 42 studies met inclusion criteria. Median sample size (IQR) was 99 (65, 179), while median event rate per study outcome was 38 (24, 65). The median fragility index was 3 (1, 4.5), indicating that an addition of only three alternate events to an arm of the average trial would have eliminated its statistical significance. There was statistically significant correlation between the fragility index and events per study (ρ=0.552, p=0.01) as well as sample size (ρ=0.493, p=0.01).

Conclusions: Statistically significant results in urology RCTs are often fragile, with significance hinging on few events. This is of particular concern in studies that may have large loss to follow-up numbers. Urologists should therefore interpret RCTs cautiously. There may be a role for reporting fragility index values routinely alongside the p-value to provide additional guidance as to the statistical robustness of findings.

Source Of Funding: None

Vikram Narayan

Presentation(s):

Send Email for Vikram Narayan


Assets

PD58-11: The Fragility of Statistically Significant Results from Randomized Trials in Urology



Attendees who have favorited this

Send Email for The Fragility of Statistically Significant Results from Randomized Trials in Urology