Questioning Neanderthal admixture: on models, robustness and consensus in human evolution

Abstract Genomic and ancient DNA data have revolutionized palaeoanthropology and our vision of human evolution, with indisputable landmarks like the sequencing of Neanderthal and Denisovan genomes. Yet, using genetic data to identify, date and quantify evolutionary events—like ancient bottlenecks or admixture—is not straightforward, as inferences may depend on model assumptions. In the last two decades, the idea that Neanderthals and members of the Homo sapiens lineage interbred has gained momentum. From the status of unlikely theory, it has reached consensus among human evolutionary biologists. This theory is mainly supported by statistical approaches that depend on demographic models minimizing or ignoring population structure, despite its widespread occurrence and the fact that when ignored, population structure can lead to infer spurious demographic events. We simulated genomic data under a structured and admixture-free model of human evolution, and found that all the tested admixture approaches identified long Neanderthal fragments in our simulated genomes and an admixture event that never took place. We also observed that several published admixture models failed to predict important empirical diversity or admixture statistics, and that our model was best at predicting these statistics jointly. Our results suggest that models accounting for population structure are fundamental to improve our understanding of human evolution, and that admixture between Neanderthals and Homo sapiens needs to be re-evaluated in the light of structured models. Beyond the Neanderthal case, we argue that ancient hybridization events, which are increasingly documented in many species, including with other hominins, may also benefit from such reevaluation. Significance statement The idea that Neanderthals and some ancestral Homo sapiens populations interbred has gained momentum in the last two decades. Yet, this theory is mainly supported by statistical approaches that assume highly simplified models of hominin evolution. A major issue is that these methods have been poorly tested in the context of population structure, despite its widespread occurrence in many vertebrate species. We simulated data under a structured model and found that all tested methods identified spurious admixture events, suggesting a lack of robustness to population structure. Besides, our structured model was better at predicting several key genomic statistics than the tested admixture models. This suggests that admixture should be re-evaluated in the light of population structure, in hominins and beyond.