“I was a few years back a slave on your property of Houton Tower, and as a Brown woman was fancied by a Mr Tumming unto who Mr Thomas James sold me.” Thus begins Mary Williamson’s letter, which for decades sat unexamined in an attic in Scotland until a history student became interested in her family’s papers, and showed it to Diana Paton. In this article, Paton uses the letter to reflect on the history and historiography of ‘Brown’ women like Mary Williamson in Jamaica and other Atlantic slave societies. Mary Williamson’s letter offers a rare perspective on the sexual encounters between white men and Brown women that were pervasive in Atlantic slave societies. Yet its primary focus is on the greater importance of ties of place and family—particularly of relations between sisters—in a context in which the ‘severity’ of slavery was increasing. Mary Williamson’s letter is a single and thus-far not formally archived trace in a broader archive of Atlantic slavery dominated by material left by slaveholders and government officials. Paton asks what the possibilities and limits of such a document may be for generating knowledge about the lives and experiences of those who were born into slavery.