We aimed to investigate the association of pineal gland volume with the risk of isolated rapid eye movement (REM) sleep behavior disorder (RBD). We enrolled 245 community-dwelling cognitively normal elderly individuals without major psychiatric or neurological disorders at the baseline evaluation, of whom 146 completed the 2-year follow-up evaluation. We assessed RBD symptoms using the REM Sleep Behavior Disorder Screening Questionnaire (RBDSQ) and defined probable RBD (pRBD) as an RBDSQ score of ≥ 5. We manually segmented the pineal gland on 3T T1-weighted brain magnetic resonance imaging and estimated its volume. The smaller the baseline pineal gland volume, the more severe the RBD symptoms at baseline. The individuals with isolated pRBD showed smaller pineal gland volumes than those without isolated pRBD. The larger the baseline pineal gland volume, the lower the risks of prevalent isolated pRBD at the baseline evaluation and incident isolated pRBD at the 2-year follow-up evaluation. Pineal gland volume showed good diagnostic accuracy for prevalent isolated pRBD and predictive accuracy for incident isolated pRBD in the receiver operator characteristic analysis. Our findings suggest that pineal gland volume may be associated with the severity of RBD symptoms and the risk of isolated RBD in cognitively normal elderly individuals.