Reliable biomarkers are needed to recognize urologic cancer patients at high risk for recurrence. In this study, we built a novel immune-related gene pairs signature to simultaneously predict recurrence for three urologic cancers. We gathered 14 publicly available gene expression profiles including bladder, prostate and kidney cancer. A total of 2,700 samples were classified into the training set (n = 1,622) and validation set (n = 1,078). The 25 immune-related gene pairs signature consisting of 41 unique genes was developed by the least absolute shrinkage and selection operator regression analysis and Cox regression model. The signature stratified patients into high- and low-risk groups with significantly different relapse-free survival in the meta-training set and its subpopulations, and was an independent prognostic factor of urologic cancers. This signature showed a robust ability in the meta-validation and multiple independent validation cohorts. Immune and inflammatory response, chemotaxis and cytokine activity were enriched with genes relevant to the signature. A significantly higher infiltration level of M1 macrophages was found in the high-risk group versus the low-risk group. In conclusion, our signature is a promising prognostic biomarker for predicting relapse-free survival in patients with urologic cancer.