Poster Presentation Society for Molecular Biology and Evolution Conference 2016

The probability of monophyly of a sample of gene lineages given a species tree: an application to maize domestication. (#701)

Rohan Mehta 1 , David Bryant 2 , Noah Rosenberg 1
  1. Stanford University, Stanford, CA, United States
  2. Mathematics and Statistics, University of Otago, Dunedin, New Zealand
Monophyletic groups---groups that consist of all the descendants of a most recent common ancestor---arise naturally as a consequence of descent processes that result in meaningful distinctions between organisms. Domestication induces strong selection and population bottlenecks, both of which can lead to an increase in the observed frequency of monophyletic groups in gene genealogies involving loci that may have been important in the domestication process.  Here we present a theoretical formula for the probability that a particular sample of gene lineages is monophyletic given a species tree under a neutral coalescent model.  Our formula extends previous work on two-species trees to arbitrarily many species.  We study the effects of species tree topology and branch lengths on the monophyly probability, revealing new behavior, including the maintenance of nontrivial monophyly probabilities for gene lineage samples that span multiple species and even for lineages that do not derive from a monophyletic species group.  We also perform an example comparison of observed monophyly frequencies to theoretical monophyly probabilities for a dataset from maize and teosinte.  The observed frequencies indicate that domesticated maize lineages show increased levels of monophyly compared both to theoretical expectations and to nondomesticated teosinte lineages.  We present a software package, Monophyler, that facilitates computation of theoretical monophyly probabilities.   Our results suggest that theoretical monophyly probabilities can be useful in the study of domestication.