The dopamine receptor-related loci have been suggested to be associated with cognitive functions and neurodegenerative diseases. It is unknown whether genetic variants such as single nucleotide polymorphisms (SNPs) in the dopamine receptor-related loci could contribute to cognitive aging independently as well as by virtue of complicated interplays in the elder population. To assess whether SNPs in the dopamine receptor-related loci are associated with cognitive aging in the elder population, we evaluated SNPs in the DRD1, NCAM1-TTC12-ANKK1-DRD2, DRD3-LOC107986115-ZNF80-TIGIT-MIR568-ZBTB20, DRD4, and DRD5-SLC2A9 loci from 25,195 older Taiwanese individuals from the Taiwan Biobank. Mini-Mental State Examination (MMSE) was scrutinized for all participants, where MMSE scores were employed to evaluate cognitive functions. From our analysis, we identified three novel genes for cognitive aging that have not previously been reported: ZBTB20 on chromosome 3 and NCAM1 and TTC12 on chromosome 11. NCAM1 and ZBTB20 are strong candidates for having a role in cognitive aging with mutations in ZBTB20 resulting in intellectual disability, and NCAM1 previously found to be associated with associative memory in humans. Additionally, we found the effects of interplays between physical activity and these three novel genes. Our study suggests that genetic variants in the dopamine receptor-related loci may influence cognitive aging individually and by means of gene-physical activity interactions.