We examined whether APOE status interacts with vascular risk factors (VRFs) to predict progression of white matter hyperintensities (WMHs) on brain MRI scans over a specific period of life in older age when risk of dementia increases. At age 73 years, baseline VRFs were assessed via self-reported history of diabetes, hypertension, smoking and hypercholesterolemia, and via objective measures of blood HbA1c, Body Mass Index, diastolic and systolic blood pressure, and blood HDL ratio. APOE e4 allele was coded as either present or absent. WMH progression was measured on MRI over 3 years in 434 older adults, in a same-year-of-birth cohort. APOE e4 carriers with either a self-reported diagnosis of diabetes (β = 0.160, p = 0.002) or higher glycated haemoglobin levels (β = 0.114, p = 0.014) exhibited greater WMH progression, and the former survived correction for multiple testing. All other APOE-VRF interactions were non-significant (βinteraction < 0.056, p > 0.228). The results suggest that carrying the APOE ‘risk’ e4 allele increases the risk of greater age-related WMH progression over the early part of the eighth decade of life, when combined with poorer glycaemic control. The interaction effect was robust to co-occurring VRFs, suggesting a possible target for mitigating brain and cognitive ageing at this age.