Capturing the genetic architecture of Alzheimer’s disease (AD) is challenging because of the complex interplay of genetic and non-genetic factors in its etiology. It has been suggested that AD biomarkers may improve the characterization of AD pathology and its genetic architecture. Most studies have focused on connections of individual genetic variants with AD biomarkers, whereas the role of combinations of genetic variants is substantially underexplored. We examined the associations of the APOE ε2 and ε4 alleles and polygenic profiles comprising the ε4-encoding rs429358, TOMM40 rs2075650, and APOC1 rs12721046 polymorphisms with cerebrospinal fluid (CSF) and plasma amyloid β (Aβ40 and Aβ42) and tau biomarkers. Our findings support associations of the ε4 alleles with both plasma and CSF Aβ42 and CSF tau, and the ε2 alleles with baseline, but not longitudinal, CSF Aβ42 measurements. We found that the ε4-bearing polygenic profiles conferring higher and lower AD risks are differentially associated with tau but not Aβ42. Modulation of the effect of the ε4 alleles by TOMM40 and APOC1 variants indicates the potential genetic mechanism of differential roles of Aβ and tau in AD pathogenesis.