Elevated glucocorticoid (GC) levels putatively damage specific brain regions, which in turn may accelerate cognitive ageing. However, many studies are cross-sectional or have relatively short follow-up periods, making it difficult to relate GCs directly to changes in cognitive ability with increasing age. Moreover, studies combining endocrine, MRI and cognitive variables are scarce, measurement methods vary considerably, and formal tests of the underlying causal hypothesis (cortisol-brain-cognition) are absent. In this study, 90 men, aged 73 years, provided measures of fluid intelligence, processing speed and memory, diurnal and reactive salivary cortisol and two measures of white matter (WM) structure (WM hyperintensity volume from structural MRI and mean diffusivity averaged across 12 major tracts from diffusion tensor MRI), hippocampal volume, and also cognitive ability at age 11. We tested whether negative relationships between cognitive ageing differences (over more than 60 years) and salivary cortisol were significantly mediated by WM and hippocampal volume. Significant associations between reactive cortisol at 73 and cognitive ageing differences between 11 and 73 (r = -.28 to -.36, p < .05) were partially mediated by both WM structural measures, but not hippocampal volume. Cortisol-WM relationships were modest, as was the degree to which WM structure attenuated cortisol-cognition associations (<15%). These data support the hypothesis that GCs contribute to cognitive ageing differences from childhood to the early 70s, partly via brain WM structure.