Inferring the probability of the derived vs. the ancestral allelic state at a polymorphic site

Peter D. Keightley*, Benjamin C. Jackson

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


It is known that the allele ancestral to the variation at a polymorphic site cannot be assigned with certainty, and that the most frequently used method to assign the ancestral state—maximum parsimony—is prone to misinference. Estimates of counts of sites that have a certain number of copies of the derived allele in a sample (the unfolded site frequency spectrum, uSFS) made by parsimony are therefore also biased. We previously developed a maximum likelihood method to estimate the uSFS for a focal species using information from two outgroups while assuming simple models of nucleotide substitution. Here, we extend this approach to allow multiple outgroups (implemented for three outgroups), potentially any phylogenetic tree topology, and more complex models of nucleotide substitution. We find, however, that two outgroups and the Kimura two-parameter model are adequate for uSFS inference in most cases. We show that using parsimony to infer the ancestral state at a specific site seriously breaks down in two situations. The first is where the outgroups provide no information about the ancestral state of variation in the focal species. In this case, nucleotide variation will be underestimated if such sites are excluded. The second is where the minor allele in the focal species agrees with the allelic state of the outgroups. In this situation, parsimony tends to overestimate the probability of the major allele being derived, because it fails to account for the fact that sites with a high frequency of the derived allele tend to be rare. We present a method that corrects this deficiency and is capable of providing nearly unbiased estimates of ancestral state probabilities on a site-by-site basis and the uSFS.

Original languageEnglish
Pages (from-to)897-906
Number of pages10
Issue number3
Early online date16 May 2018
Publication statusPublished - 1 Jul 2018


  • Ancestral allele
  • Derived allele
  • Misinference
  • Nucleotide polymorphism
  • Parsimony
  • Unfolded site frequency spectrum


Dive into the research topics of 'Inferring the probability of the derived <i>vs</i>. the ancestral allelic state at a polymorphic site'. Together they form a unique fingerprint.

Cite this