Vesicle Imaging and Data Reporting System (VI-RADS): Multi-institutional multi-reader diagnostic accuracy and inter-observer agreement study

Abstract

BACKGROUND. The Vesical Imaging Reporting and Data System (VI-RADS), based on multiparametric MRI (mpMRI), was developed to provide accurate information for the diagnosis of muscle-invasive bladder cancer (MIBC). OBJECTIVE. The purpose of our study was to evaluate the interobserver agreement and diagnostic performance of VI-RADS among readers with different levels of experience. METHODS. This retrospective study included 91 consecutive patients who underwent mpMRI before transurethral resection of bladder tumor (TURBT) from July 2010 through August 2018. After attending a training session, seven radiologists (five radiologists experienced in bladder MRI and two inexperienced radiologists) reviewed and scored all MRI examinations according to VI-RADS. The interobserver agreement was assessed by kappa statistics. ROC analysis was used to evaluate the diagnostic performance for MIBC. AUCs were estimated. RESULTS. Among 91 patients (72 men and 19 women; mean age ± SD, 73.2 ± 10.2 years), 48 (52.7%) had MIBC and 43 (47.3%) had non-muscle-invasive bladder cancer. Sixty-eight patients were treated with TURBT, and 23 were treated with radical cystectomy. Interobserver agreement was moderate to substantial (κ = 0.60-0.80) among the experienced readers, substantial (κ = 0.67) between the two inexperienced readers, and moderate to substantial (κ = 0.55-0.75) between the experienced and inexperienced readers. The pooled AUC was 0.88 (range, 0.82-0.91) for experienced readers and 0.84 (range, 0.83-0.85) for inexperienced readers, and 0.87 for all readers. Using a VI-RADS score of 4 or greater as the cutoff value for MIBC, the pooled sensitivity and specificity were 74.1% (range, 66.0-80.9%) and 94.1% (range, 88.6-97.7%) for experienced readers and 63.9% (range, 59.6-68.1%) and 86.4% (range, 84.1-88.6%) for inexperienced readers. Using a VI-RADS score of 3 or greater as the cutoff value, the pooled sensitivity and specificity were 83.4% (range, 80.9-85.1%) and 77.3% (range, 61.4-88.6%) for experienced readers and 82.0% (range, 80.9-83.0%) and 73.9% (range, 72.7-75.0%) for inexperienced readers. CONCLUSION. We observed moderate to substantial interobserver agreement and a pooled AUC of 0.87 among radiologists of different levels of expertise using VI-RADS. CLINICAL IMPACT. VI-RADS could help determine the depth and range of excision in TURBT, decreasing the risk of complications and enhancing the accuracy of pathologic diagnosis.