The transient response of model hard sphere glasses is examined during the application of steady rate start-up shear using Brownian dynamics simulations, experimental rheology and confocal microscopy. With increasing strain, the glass initially exhibits an almost linear elastic stress increase, a stress peak at the yield point and then reaches a constant steady state. The stress overshoot has a nonmonotonic dependence with Peclet number, Pe, and volume fraction, φ, determined by the available free volume and a competition between structural relaxation and shear advection. Examination of the structural properties under shear revealed an increasing anisotropic radial distribution function, g(r), mostly in the velocity-gradient (xy) plane, which decreases after the stress peak with considerable anisotropy remaining in the steady-state. Low rates minimally distort the structure, while high rates show distortion with signatures of transient elongation. As a mechanism of storing energy, particles are trapped within a cage distorted more than Brownian relaxation allows, while at larger strains, stresses are relaxed as particles are forced out of the cage due to advection. Even in the steady state, intermediate super diffusion is observed at high rates and is a signature of the continuous breaking and reformation of cages under shear.