We use A-type stars selected from Sloan Digital Sky Survey data release 9 photometry to measure the outer slope of the Milky Way stellar halo density profile beyond 50 kpc. A likelihood-based analysis is employed that models the ugr photometry distribution of blue horizontal branch (BHB) and blue straggler (BS) stars. In the magnitude range, 18.5 <g <20.5, these stellar populations span a heliocentric distance range of: 10 kpc <D_BS <75 kpc, 40 kpc <D_BHB <100 kpc. Contributions from contaminants, such as QSOs, and the effect of photometric uncertainties, are also included in our modeling procedure. We find evidence for a very steep outer halo profile, with power-law index alpha ~ 6 beyond Galactocentric radii r=50 kpc, and even steeper slopes favored (alpha ~ 6-10) at larger radii. This result holds true when stars belonging to known overdensities, such as the Sagittarius stream, are included or excluded. We show that, by comparison to numerical simulations, stellar halos with shallower slopes at large distances tend to have more recent accretion activity. Thus, it is likely that the Milky Way has undergone a relatively quiet accretion history over the past several Gyr. Our measurement of the outer stellar halo profile may have important implications for dynamical mass models of the Milky Way, where the tracer density profile is strongly degenerate with total mass-estimates.