The acoustic phonons in the organic-inorganic lead halide perovskites have been reported to have anomalously short lifetimes over a large part of the Brillouin zone. The resulting shortened mean free paths of the phonons have been implicated as the origin of the low thermal conductivity. We apply neutron spectroscopy to show that the same acoustic phonon energy linewidth broadening (corresponding to shortened lifetimes) occurs in the fully inorganic CsPbBr3 by comparing the results on the organic-inorganic CH3NH3PbCl3. We investigate the critical dynamics near the three zone boundaries of the cubic Pm¯3m Brillouin zone of CsPbBr3 and find energy and momentum broadened dynamics at momentum points where the Cs-site (A-site) motions contribute to the cross section. Neutron diffraction is used to confirm that both the Cs and Br sites have unusually large thermal displacements with an anisotropy that mirrors the low temperature structural distortions. The presence of an organic molecule is not necessary to disrupt the low-energy acoustic phonons at momentum transfers located away from the zone center in the lead halide perovskites and such damping may be driven by the large displacements or possibly disorder on the A site.