In order to deal with strong inhomogeneities of the polymer stress in shear banding of wormlike micelles, rheological models usually include stress diffusion terms. These terms lift the degeneracy of steady flow solutions by selecting the steady-state positions of interfaces between bands and the values of the total mechanical stress. They are also responsible for slow transients of interface kinetics under imposed step shear flow. In our previous experiments, we estimated the typical lengthscale associated with these terms to be of the order of several nanometers, while several other experiments obtained significantly larger estimates. We report here a new set of experiments that reconcile our previous work with the rest of the literature and obtain a diffusive lengthscale of order of a few micrometers. Surprisingly, we find that this lengthscale is a strong function of the applied shear rate, decreasing rapidly across the stress plateau. We perform numerical simulations of several constitutive equations used to model shear-banded flows and conclude that they cannot predict this behavior. We argue that the strong dependence of the diffusive lengthscale on the applied shear rate should be incorporated into the classical constitutive models to obtain quantitative predictions.