Active matter exhibits various forms of non-equilibrium states in the absence of external forcing, including macroscopic steady-state currents. Such states are often too complex to be modelled from first principles and our understanding of their physics relies heavily on minimal models. These have mostly been studied in the case of "dry" active matter, where particle dynamics are dominated by friction with their surroundings. Significantly less is known about systems with long-range hydrodynamic interactions that belong to "wet" active matter. Dilute suspensions of motile bacteria, modelled as self-propelled dipolar particles interacting solely through long-ranged hydrodynamic fields, are arguably the most studied example from this class of active systems. Their phenomenology is well-established: at sufficiently high density of bacteria, there appear large-scale vortices and jets comprising many individual organisms, forming a chaotic state commonly known as bacterial turbulence. As revealed by computer simulations, below the onset of collective motion, the suspension exhibits very strong correlations between individual microswimmers stemming from the long-ranged nature of dipolar fields. Here we demonstrate that this phenomenology is captured by the minimal model of microswimmers. We develop a kinetic theory that goes beyond the commonly used mean-field assumption, and explicitly takes into account such correlations. Notably, these can be computed exactly within our theory. We calculate the fluid velocity variance, spatial and temporal correlation functions, the fluid velocity spectrum, and the enhanced diffusivity of tracer particles. We find that correlations are suppressed by particle self-propulsion, although the mean-field behaviour is not restored even in the limit of very fast swimming.