The early 20th century warming (EW; 1910-1945) and the mid-20th century cooling (MC; 1950-1980) have been linked to both, internal variability of the climate system and changes in external radiative forcing. The degree to which either of the two factors contributed to EW and MC, or both, is still debated. Using a two-box impulse response model, we demonstrate that multidecadal ocean variability was unlikely to be the driver of observed changes in global mean surface temperature (GMST) after 1850 A.D. Instead, virtually all (97-98%) of the global low-frequency variability (> 30 years) can be explained by external forcing. We find similarly high percentages of explained variance for inter-hemispheric and land-ocean temperature evolution. Three key aspects are identified which underpin the conclusion of this new study: inhomogeneous anthropogenic aerosol forcing (AER), biases in the instrumental sea surface temperature (SST) datasets, and inadequate representation of the response to varying forcing factors. Once the spatially heterogeneous nature of AER is accounted for, the MC period is reconcilable with external drivers. SST biases and imprecise forcing responses explain the putative disagreement between models and observations during the EW period. As a consequence, Atlantic Multidecadal Variability (AMV) is found to be primarily controlled by external forcing too. Future attribution studies should account for these important factors when discriminating between externally-forced and internallygenerated influences on climate. We argue that AMV must not be used as a regressor and suggest a revised AMV index instead (North Atlantic Variability Index; NAVI). Our associated best estimate for the transient climate response (TCR) is 1.57 K (±0.70 at the 5-95% confidence level).