The uneven geographical distribution of environmental pathogens and salutogens, as well as the political, social, and cultural antecedents leading to this sociospatial arrangement, have been posited as a partial explanation for the stark inequalities in health across many high-income nations. Whilst there is significant international evidence for the maldistribution of health-related environmental features, few studies have examined the material outcomes (including health) of this unequal environmental ‘exposure’. In this paper we utilise the theoretical stances offered by work in the fields of environmental justice and socioecological models of health to consider the pathway between physical environmental deprivation and health. We consider the influence of multiple aspects of the ‘natural’ physical environment on individual-level levels of physical activity—both utilitarian and physical activity for leisure. We found that, for physical activity conducted for recreational purposes, there is a strong relationship with the natural physical environment: those living in the least deprived physical environments are most likely to engage in physical activity. However, for utilitarian physical activity, physical activity whose primary purpose is not the activity itself, we observed increased levels in the most environmentally deprived areas. Finally, the important role that the environment may play in shaping capabilities, particularly during the current economic and political climate, is recognised. Our results show that the environment matters and that rhetoric regarding ‘lifestyle choice’ needs to be viewed in a broader environmental context.