This paper examines how race might be understood differently when social interaction is taken as the starting point of analysis. I argue that dominant modes of theorising race as a biological construct or epistemological marker remain insufficient for understanding the multiple, contingent, and devious ways in which race takes form in, and gives shape to, encounters. Drawing on ethnographic fieldwork in Keighley-a former mill town in northern England the paper assembles narrative fragments that reconstruct encounters with difference (that vary in intensity from the mundane to terror alerts). In each of these encounters I return to the question: what does race do? The paper offers a reconsideration of race and racism. I theorise race as a technology of differentiation that sorts human difference in ways that acknowledge the malleability of race and the more-than-human composition of social relations. I go on to outline an understanding of racism a racism of assemblages that recognises that the sorting of human difference is also accompanied by judgments that prefigure encounters. The racism of assemblages offers an opportunity to address the operation of race at the level of nonconscious thinking and the affective intensities through which the sorting and judging of human differences are performed. The work of gathering fragments to reconstruct encounters also generates insights into the microsociality of multicultural life in Keighley, disrupting narratives that argue that white and Asian communities lead 'parallel lives' in northern mill towns.