The study was aimed at assessing the diagnostic performance of 68Ga-PSMA-617 PET/CT in the detection of prostate cancer (PCa) in patients with a prostate-specific antigen (PSA) level of 4–20 ng/ml and to compare its efficacy with that of multiparametric MRI (mpMRI). We analyzed the data of 67 consecutive patients with PSA levels of 4–20 ng/ml who almost simultaneously underwent 68Ga-PSMA-617 PET/CT and mpMRI. 68Ga-PSMA-617 PET/CT and mpMRI diagnostic performances were compared via receiver operating characteristic (ROC) curve analysis. Of the 67 suspected PCa cases, 33 had pathologically confirmed PCa. 68Ga-PSMA-617 PET/CT showed a patient-based sensitivity, specificity, and positive and negative predictive values (PPVs and NPVs) of 87.88%, 88.24%, 87.88%, and 88.24%, respectively. The corresponding values for mpMRI were 84.85%, 52.94%, 63.64%, and 78.26%. The area under the curve values for 68Ga-PSMA-617 PET/CT and mpMRI were 0.881 and 0.689, respectively. 68Ga-PSMA-617 PET/CT showed a better diagnostic performance than mpMRI in the detection of PCa in patients with PSA levels of 4–20 ng/ml.