The idea that Neanderthals and some ancestral populations of Homo sapiens interbred has gained traction over the past two decades. However, this theory is primarily supported by statistical approaches that assume unrealistic models of human and Neanderthal population evolution.
In this study published in November/December 2024 in Nature, ecology and evolution, researchers simulated scenarios taking into account the spatial structure of human and Neanderthal populations. They show that all tested methods detect false signals of hybridization in scenarios where, however, Neanderthals and sapiens never interbreed.
This suggests that hybridization with Neanderthals may not be as clear-cut as many researchers claim and that the numerous hybridization signals detected in many organisms could also be false positives. It also suggests that the way ancient human and non-human populations are modeled deserves critical re-evaluation.
Most studies conducted by geneticists and paleoanthropologists take it for granted that Homo sapiens and H. neandertalensis interbred and that some human populations carry 2 to 3% of Neanderthal genes. However, the evidence for this hybridization and its quantification depend on statistical methods that make very strong and unrealistic assumptions about human and Neanderthal populations.
These methods tend to ignore the existence of differentiated populations within continents. For example, they see Europe, or the entire African continent, as a large, completely homogeneous population for over 50,000 years.
It is normal in science to use simple models to address complex questions, but it is also important to determine whether the conclusions obtained with these simple models will be affected by deviations from the necessarily simplifying assumptions made by scientists. This raises the question of the robustness of the inference methods used in population genetics. For two decades, population geneticists have noted that when populations are structured, the use of statistical methodologies that ignore this structure can generate false positives.
In this publication, scientists proposed different models of human evolution from those classically used and tested the robustness of many methods that have been used to detect, quantify, and date hybridization events between Neanderthals and sapiens. It seemed important to explore models of human evolution that integrate the existence of a continuous spatial structure between human populations, and less discontinuous than those models separating continents and treating them as isolated populations or geographical regions.
To do this, they simulated computer scenarios in which human populations are represented as strings of islands, each connected to its neighbor, incorporating paleoanthropological data from the scientific literature. Finally, they also integrated both knowledge and uncertainties about events such as the separation between Neanderthals and sapiens, when the scientific literature proposed quite different values.
Several scenarios were identified that explain the known genomic data better than the current models tested, which assume hybridization. The researchers also showed that all tested methods detect false hybridization signals in simulated scenarios where Neanderthals and sapiens never interbreed. They also identified numerous inconsistencies between models and studies identifying hybridization events between Neanderthals and sapiens.
For example, some models predict much higher or lower levels of genetic diversity than those actually observed. Hybridization signals observed in ancient DNA could also be false positives, explainable in some cases by biases in the analysis of ancient polymorphisms.
Many evolutionary factors could have played a role in the emergence of the diversity patterns observed in human populations today. However, the results obtained in this study suggest that models taking population structure into account are essential to improve our understanding of human evolution, and that hybridization between Neanderthals and sapiens must be re-evaluated in light of structured models.
Beyond the case of Neanderthals, ancient hybridization events, which are increasingly documented in many species, including with other hominins, could also benefit from such a re-evaluation.
References:
Tournebize, R and Chikhi, L. (2024).
Ignoring population structure in hominin evolutionary models can lead to the inference of spurious admixture events.
Nature, ecology and evolution, published on December 13, 2024.