caa a22 4500 445330260 CHVBK 20180317142735.0 cr unu---uuuuu 170323e20110601xx s 000 0 eng 10.1007/s00285-010-0355-7 doi (NATIONALLICENCE)springer-10.1007/s00285-010-0355-7 Identifying the rooted species tree from the distribution of unrooted gene trees under the coalescent [Elektronische Daten] [Elizabeth Allman, James Degnan, John Rhodes] Gene trees are evolutionary trees representing the ancestry of genes sampled from multiple populations. Species trees represent populations of individuals—each with many genes—splitting into new populations or species. The coalescent process, which models ancestry of gene copies within populations, is often used to model the probability distribution of gene trees given a fixed species tree. This multispecies coalescent model provides a framework for phylogeneticists to infer species trees from gene trees using maximum likelihood or Bayesian approaches. Because the coalescent models a branching process over time, all trees are typically assumed to be rooted in this setting. Often, however, gene trees inferred by traditional phylogenetic methods are unrooted. We investigate probabilities of unrooted gene trees under the multispecies coalescent model. We show that when there are four species with one gene sampled per species, the distribution of unrooted gene tree topologies identifies the unrooted species tree topology and some, but not all, information in the species tree edges (branch lengths). The location of the root on the species tree is not identifiable in this situation. However, for 5 or more species with one gene sampled per species, we show that the distribution of unrooted gene tree topologies identifies the rooted species tree topology and all its internal branch lengths. The length of any pendant branch leading to a leaf of the species tree is also identifiable for any species from which more than one gene is sampled. Springer-Verlag, 2010 Multispecies coalescent nationallicence Phylogenetics nationallicence Invariants nationallicence Polytomy nationallicence Allman Elizabeth Department of Mathematics and Statistics, University of Alaska Fairbanks, PO Box 756660, 99775, Fairbanks, AX, USA aut Degnan James Department of Mathematics and Statistics, University of Canterbury, Private Bag 4800, Christchurch, New Zealand aut Rhodes John Department of Mathematics and Statistics, University of Alaska Fairbanks, PO Box 756660, 99775, Fairbanks, AX, USA aut Journal of Mathematical Biology Springer-Verlag 62/6(2011-06-01), 833-862 0303-6812 62:6<833 2011 62 285 https://doi.org/10.1007/s00285-010-0355-7 text/html Onlinezugriff via DOI 1 research-article jats NATIONALLICENCE 856 40 https://doi.org/10.1007/s00285-010-0355-7 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Allman Elizabeth Department of Mathematics and Statistics, University of Alaska Fairbanks, PO Box 756660, 99775, Fairbanks, AX, USA aut NATIONALLICENCE 700 1- Degnan James Department of Mathematics and Statistics, University of Canterbury, Private Bag 4800, Christchurch, New Zealand aut NATIONALLICENCE 700 1- Rhodes John Department of Mathematics and Statistics, University of Alaska Fairbanks, PO Box 756660, 99775, Fairbanks, AX, USA aut NATIONALLICENCE 773 0- Journal of Mathematical Biology Springer-Verlag 62/6(2011-06-01), 833-862 0303-6812 62:6<833 2011 62 285 Metadata rights reserved Springer special CC-BY-NC licence nationallicence BK010053 XK010053 XK010000 NATIONALLICENCE NATIONALLICENCE NL-springer