The evolution of reproductive barriers is the first step in the formation of new species and can help us understand the diversification of life on Earth. These reproductive barriers often take the form of hybrid incompatibilities, in which alleles derived from two different species no longer interact properly in hybrids. Theory predicts that hybrid incompatibilities may be more likely to arise at rapidly evolving genes and that incompatibilities involving multiple genes should be common, but there has been sparse empirical data to evaluate these predictions. Here we describe a mitonuclear incompatibility involving three genes whose protein products are in physical contact within respiratory complex I of naturally hybridizing swordtail fish species.

Individuals homozygous for mismatched protein combinations do not complete embryonic development or die as juveniles, whereas those heterozygous for the incompatibility have reduced complex I function and unbalanced representation of parental alleles in the mitochondrial proteome. We find that the effects of different genetic interactions on survival are non-additive, highlighting subtle complexity in the genetic architecture of hybrid incompatibilities. Finally, we document the evolutionary history of the genes involved, showing signals of accelerated evolution and evidence that an incompatibility has been transferred between species via hybridization.

Biologists have long been fascinated by the question of how new species are formed and the mechanisms that maintain isolation between them. The study of hybrid incompatibilities sheds light on these processes, offering insights into the genetic and evolutionary factors that drive speciation. This recent research on mitonuclear incompatibility in swordtail fish provides valuable empirical data supporting theoretical predictions and contributes to our understanding of the intricate genetic mechanisms underlying the formation of new species. The findings not only enhance our knowledge of evolutionary biology but also have implications for biodiversity conservation and the preservation of species boundaries.