Royal Society Publishing

Introduction. Speciation in plants and animals: pattern and process

Richard J Abbott, Michael G Ritchie, Peter M Hollingsworth


Although approximately 150 years have passed since the publication of On the origin of species by means of natural selection, the definition of what species are and the ways in which species originate remain contentious issues in evolutionary biology. The biological species concept, which defines species as groups of interbreeding natural populations that are reproductively isolated from other such groups, continues to draw support. However, there is a growing realization that many animal and plant species can hybridize with their close relatives and exchange genes without losing their identity. On occasion, such hybridization can lead to the origin of new species. A key to understanding what species are and the ways in which they originate rests to a large extent on a detailed knowledge of the nature and genetics of factors that limit gene flow between species and the conditions under which such isolation originates. The collection of papers in this issue addresses these topics and deals as well with some specific issues of hybrid speciation and the causes of species radiations. The papers included arise from a 1-day symposium on speciation held during the Sixth Biennial Meeting of the Systematics Association at Edinburgh in August 2007. In this introduction, we provide some background to these papers and highlight some key points made. The papers make clear that highly significant advances to our understanding of animal and plant speciation are currently being made across the range of this topic.


1. Defining species

Charles Darwin's On the origin of species by means of natural selection (1859) revolutionized our understanding of how changes within species brought about by natural selection may lead to the formation of new species. Despite the fact that approximately 150 years have passed since the book was published, both the definition of what species are and the ways in which species originate remain highly contentious issues in evolutionary biology. Coyne & Orr (2004) have recently documented and discussed nine different species concepts, while others have extended this list even further (Claridge et al. 1997)! Each concept has its pros and cons and it is unlikely that any one concept will ever achieve universal acceptance. The definition currently most widely accepted according to Coyne & Orr (2004) is Mayr's biological species concept, which places the evolution of complete reproductive isolation at the centre of the process and nature of speciation. In its original form, this states that ‘Species are groups of interbreeding natural populations that are reproductively isolated from other such groups’ (Mayr 1942). Mayr's strict application of the biological species concept was based on his belief in the absence of gene flow between most species, the low fitness of occasional hybrids formed and that new species rarely result from hybridization (even in plants which he recognized were prone to hybridization, Mayr 2002). Recent developments, highlighted in this issue, have challenged many of these beliefs.

Mayr recognized that disagreements in regard to the definition of species are partly because ‘…the term species is applied to two very different things, to the species as concept and to the species as taxon’ (Mayr 2002). This division is as much apparent today as it has ever been. Inevitably, when the term species is viewed as a concept rather than simply as a taxon, both pattern and process enter the definition with emphasis shifting to how discrete entities (species) form and are maintained in the wild. This is particularly true for species that co-occur in the same area where there is the potential for high levels of interspecific gene flow unless some form of reproductive barrier is in place. This has led to the argument that a suitable test for whether or not different organisms are indeed different species should rest on their ability to be maintained in sympatry. However, most organisms are geographically isolated from each other and consequently such tests cannot normally be conducted in practice, although transplant tests, where possible, could overcome this problem to a degree.

In the present issue, Mallet (2008) examines the ‘species problem’ again, pointing out some of the weaknesses (and strengths) of the biological species concept. Mallet advocates acceptance of a more Darwinian view of species based on the presence of consistent gaps between them (i.e. of the heritable morphological/physiological/behavioural kind), with the proviso that such gaps should be maintained in sympatry. Furthermore, he argues that in contrast to the views of many (e.g. Mayr 1942; Coyne & Orr 2004), Darwin provided a relevant description in The origin of species of how species originate in terms of a natural and continuous extension to the gradual changes within species brought about by divergent natural selection. In support of this, Mallet makes the case that ecotypic variation, the origin of which may frequently be an important step in speciation, is common in both animal and plant species, and that reproductive compatibility appears to decline in inverse proportion to both degree of genetic divergence and time since divergence across the species boundary. He concludes that ‘Some level of reproductive isolation is clearly important in the formation and maintenance of the sexual populations we would like to call species, but Darwin's idea that it is better to classify species taxa via whether they remain distinct, i.e. the results of these processes, seems at least eminently sensible.’ This raises the critically important question of just how much reproductive isolation is required between ‘good’ species?

In practice, taxonomists classify species according to morphological gaps between them (i.e. in terms of the original Linnaean meaning). This leads to the idea that species may be ‘barcoded’ by means of a DNA signature. Presently, there is great interest in developing DNA barcoding protocols, so that species might be readily identified for conservation and ecological purposes. The importance of this in drawing up biodiversity inventories and monitoring temporal and spatial changes in species diversity and abundance is clearly highly significant in today's world where major concerns exist over species loss due to human activities. The success of DNA barcoding is ultimately dependent upon the presence of suitable signal in the barcoding locus to differentially group sets of individuals into their respective species. Papadopoulou et al. (2008) used a combination of simulated and empirical datasets to explore the conditions under which discrete clusters of sequences are likely to arise, focusing on their recently developed likelihood method for identifying species clusters from barcode data. To date, there has been strong correspondence between barcode clusters and taxonomic species in many published studies (e.g. Hebert et al. 2004), but there have also been some exceptions (e.g. Elias et al. 2007). The crux issue for DNA barcoding is the extent to which these successes will outnumber the failures.

2. Geographical mode of speciation, and the nature and genetics of reproductive isolation

In regard to the mode of speciation, spatial context has traditionally played a dominant role with three main categories proposed—sympatric, parapatric and allopatric (Mayr 1942). Sympatric speciation proceeds without spatial isolation and with divergence occurring under conditions of random mating, while allopatric speciation occurs when diverging populations are spatially isolated and do not exchange genes with each other. Parapatric speciation describes the situation where intermediate levels of gene flow persist during divergence. Since pioneering work of the modern synthesis, it has been argued that speciation is almost always a consequence of allopatric divergence, because gene substitutions conferring reproductive incompatibilities were thought very unlikely to appear without geographical isolation. Coyne & Orr (2004) have argued that allopatric speciation should be viewed as the ‘null hypothesis’ in studies aimed at determining the geographical context of speciation. However, the criteria for demonstrating non-allopatric speciation in the wild are such that without a complete record of the history of the process, it is virtually impossible to reject the null hypothesis of an allopatric origin. This is made clear by Butlin et al. (2008) in regard to their work and others on shell morph divergence in the intertidal snail, Littorina saxatilis. Thus, they advise that rather than being overly concerned about whether or not speciation occurs in sympatry or allopatry, in situations where divergent populations currently occur in sympatry (or parapatry) it is more ‘…productive to focus on the current balance between selection and gene flow rather than trying to establish how divergence began at an unknown time in the past or how it will progress in the future.’ If allopatry and sympatry are opposite ends of a continuum, polarizing arguments over these extremes may be missing the more biologically common scenario of divergence despite limited gene flow (Nosil 2008).

Some degree of reproductive isolation is an essential step in the transition of ecotypes to species that may be maintained in sympatry or parapatry (see e.g. Abbott & Comes 2007). However, reproductive isolation can result from many different causes, and in recent years there has been widespread recognition that neither hybrid sterility nor hybrid inviability need necessarily evolve in the origin of species or in securing the maintenance of species in sympatry. For example, animal species may diverge rapidly due to sexual selection while plants need only to change pollination vectors or flowering time. Traditionally, reproductive isolating barriers have been classified into two types, prezygotic and postzygotic. Early studies of the nature and genetics of reproductive isolation in animals focused primarily on postzygotic barriers, especially hybrid sterility and inviability. An important theory to emerge was that genes that have diverged in allopatric populations due to the effects of drift or selection could be incompatible when combined in hybrids causing hybrid sterility and inviability. This model for the origin of postzygotic isolation is known as the Bateson–Dobzhansky–Muller (BDH) model of genic incompatibility after those who contributed to its development (Coyne & Orr 2004). Empirical support for the BDH model initially emerged from detailed studies conducted mainly on animals (particularly Drosophila) and in recent years a number of ‘speciation’ genes that fit the model have been isolated and characterized (Coyne & Orr 2004). Although the first example of a BDH genic incompatibility causing hybrid inviability was discovered in a plant (Hollingshead 1930), studies of the genetics of postzygotic isolation in plants have until recently lagged well behind those conducted in animals. This led Schemske (2000) to note rather gloomily in a review of a collection of papers on speciation published 10 years ago (Howard & Berlocher 1998) ‘There is a dearth of papers on plants, but the new generation of botanists is to blame…. Molecular phylogenies have replaced the extensive biosystematic studies that were once common in botany, and plant speciation studies have suffered as a result. Drosophila retain their status as the system of choice for studying the genetics of speciation.’ Gladly, this situation for plants is no longer the case.

Lowry et al. (2008) present a detailed review of the strength and genetic basis of reproductive isolation in flowering plants. They show that the strength of prezygotic reproductive isolation is approximately double that of postzygotic isolation based on the results of 19 studies of closely related pairs of taxa, and that postmating barriers are approximately three times more asymmetrical in their action than premating barriers. They make clear that the number of genes controlling reproductive isolation and the magnitude of their individual effects vary greatly between species. In some cases a few genes of major effect suffice, whereas in others a moderate number of genes are involved. Perhaps, somewhat surprisingly, current evidence suggests that chromosomal rearrangements are of limited importance in the origin of reproductive isolation in plants, reflecting what is also thought to be the case for animals (Coyne & Orr 2004; but see Noor et al. 2001; Rieseberg 2001). Similar conclusions on the genetics of reproductive isolation in plants are reached by Lexer & Widmer (2008) from a review of studies conducted on monkey flowers, sunflowers, irises, Populus and campions. Thus, the genetics of reproductive isolation in many plants is likely to fit the genic view of speciation (Wu 2001), which proposes that reproductive isolation is a consequence of differences in adaptation controlled by a moderate number of individual genes rather than the whole genome. Although alleles at these gene loci will not move between species, it is feasible that there will be interspecific gene flow at other loci. In these cases, Mayr's biological species concept requires significant modification to the extent that reproductive isolation need only apply to genes that characterize species differences; other genes that are neutral to the effects of selection may move between species without disrupting the differences that define these species. Such a situation might be common where species that can hybridize are adapted to different adjacent habitats (Minder & Widmer 2008).

In a third paper on reproductive isolation in plants, the focus shifts to orchids. Orchids are members of the most species-rich family of flowering plants and are notorious for their ability to hybridize. Understanding how orchids originate, and are maintained in sympatry, presents notable challenges to students of speciation. Cozzolino & Scopece (2008) focus on two groups of orchids—food-deceptive and sexually deceptive species—and show that members of the former group exhibit weak premating isolation due to low pollinator specificity, but strong postmating reproductive barriers, particularly for late-acting postzygotic barriers such as hybrid sterility. By contrast, sexually deceptive species rely on strong premating isolation due to high pollinator specificity and show very little postmating isolation, although for good reasons there are exceptions to these rules. These findings illustrate the general point made by Mallet (2008) in regard to types and strength of reproductive isolation, ‘In one species pair, there may be a lot of hybridization, but very strong selection against hybrids; in a second pair, low levels of hybridization may be counteracted by weak selection.’ In the case of food- and sexually deceptive orchids it is evident that degree of pollinator specificity is the key to this difference.

3. Hybrid speciation

The significance of interspecific hybridization between animal and plant species is a recurrent theme of papers in this issue. This therefore leads to a consideration of whether interspecific hybridization can be a creative force in speciation, an idea largely rejected by Mayr (2002). Botanists have long recognized the importance of hybridization as a trigger for speciation in plants (Grant 1981; Abbott 1992; Rieseberg 1997; Arnold 2006), and more recently this possibility has been extended to animals (Bullini 1994; Mallet 2007, 2008). Two forms of hybrid speciation are recognized. One involves no change in chromosome number (homoploid hybrid speciation), while the other involves a doubling of the chromosome number of the hybrid (allopolyploidy). Chapman & Burke (2007) recently have shown that in plants genetic divergence is significantly greater between parents of allopolyploid than homoploid hybrid species.

Several examples of animal homoploid hybrid speciation have recently been described (see Mallet 2007) and one of these, in Heliconius butterflies, is the subject of attention by Jiggins et al. (2008). In this particular case, the putative hybrid species has a novel colour pattern that is a combination of those found in populations of its putative parent species. In other respects, the putative hybrid is genetically more similar to one of its parents. The reason why the hybrid is viewed as a distinct species rather than an introgressant of one of the parent species is because the hybrid colour pattern itself has a direct influence on the reproductive isolation between the hybrid form and both parents. Jiggins et al. (2008) coin the term ‘hybrid trait speciation’ to describe the process they believe to have occurred in Heliconius; that is, where a particular trait or group of traits introgressed into a species causes hybrid individuals to be reproductively isolated from both parent species. They argue that genetic evidence of a hybrid species' origin via hybrid trait speciation may easily become blurred over time and consequently many examples involving such an origin will go unrecognized. Unfortunately, distinguishing this form of hybrid speciation from partial introgression or recent shared polymorphism may make empirical assessment of the importance of hybrid speciation very difficult.

The number of homoploid hybrid species in plants (Gross & Rieseberg 2005) is far greater than that recorded in animals, but is dwarfed by the number of plant species which have originated through allopolyploidy. Whereas allopolyploid speciation is known to have been of major significance in plant evolution (Tate et al. 2005; Arnold 2006), it is a very rare phenomenon in animals. Hegarty et al. (2008) focused on genomic and gene expression changes that can occur during homoploid and allopolyploid hybrid speciation in plants as a result of the union of divergent parental genomes. They show that such changes can produce novel genetic variation that might be important in adapting hybrid species to new habitats that are different from those of their parents, thus aiding the reproductive isolation of the hybrid from its parents (Rieseberg et al. 2003). Alterations to gene expression during the origin of one particular neoallopolyploid species (Senecio cambrensis) are described and shown to be due to both the initial hybridization event and the subsequent genome duplication. The observed changes in gene expression in this species are frequently ‘transgressive’, i.e. expression is either significantly greater or lower than that of both parent species. Interestingly, the classes of genes showing transgressive expression in S. cambrensis are equivalent to those influenced in a similar way by hybridization in Arabidopsis and maize, indicating that particular gene networks in plants may be susceptible to perturbation by hybridization.

4. Species radiations

One of the most dramatic outcomes of speciation is a species radiation where a group of closely related species, often highly divergent in form and/or function, originates from a common ancestor. Such radiations clearly provide excellent material for a detailed analysis of the speciation process. Although selection is normally considered as the driving force of species radiations (Schluter 2000), distinguishing the role of isolation from selection takes ingenuity as is made clear in two of the final three papers in this issue, which focus on different types of species radiations and their causes. Thorpe et al. (2008) provide an engaging attempt to test the importance of geographical isolation and ecological selection in driving divergence in the genus Anolis (iguanine lizards) on the island of Martinique in the Lesser Antilles. This island is a coalescent of smaller older islands, and patterns of divergence among mitochondrial lineages reveal the importance of allopatric divergence prior to island fusion. However, nuclear microsatellite divergence and patterns of morphological variation in important traits are instead shaped by contemporary ecological clines. This study therefore reveals a complex interaction between isolation and selection.

Although most species radiations are thought to be driven largely by selection, there are a few for which genetic drift has been considered to be the dominant force in bringing about ‘non-adaptive’ divergence. One of the best-known examples of this in plants concerns the so-called Nigella arvensis complex (6 species and 12 taxa) that is distributed across the Greek mainland and islands of the Aegean archipelago, in the Mediterranean. Comes et al. (2008) used a mix of phylogenetic, phylogeographic and population-level approaches to re-examine the proposal by Strid (1970) that this radiation is the product of drift acting in allopatric populations that occupy ecologically very similar habitats. Their results show that the radiation was brought about by rapid diversification during the last million years and suggest ‘…that allopatry (often but not exclusively vicariant) and genetic drift (coupled with restricted gene exchange) are the dominant evolutionary processes driving population differentiation and speciation in Aegean Nigella.’ However, further analysis is required to prove that phenotypic divergence within this complex is largely non-adaptive and the result of genetic drift.

In the final paper of this issue, Linder (2008) undertakes a comprehensive examination of the temporal and spatial patterns of plant species radiations. Whereas species radiations are normally thought to occur over relatively short periods of recent time, Linder identifies both ancient (mature) and recent radiations in his analysis. Using all relevant dated plant radiations (70 in total), it is shown that two phases may be recognized in the radiation of a clade. In phase 1, which is estimated to last between 2 and 8 million years on average, the rate of increase in diversity is very high. In phase 2, which is only recognized in ‘mature’ radiations, diversity in clades increases only slowly following the rapid phase 1 diversification, and consequently comparatively high diversity in such cases ‘…is more a result of the absence of episodic mass extinctions, rather than a particularly high speciation rate.’ There appears to be a distinct regional effect in the occurrence of mature and recent species radiations in plants, an effect also reported by Ricklefs (2006) in passerine birds. For example, plant species radiations in Australia, which is notable for having been climatically and geologically stable over the past 23 million years (the Neogene), are largely of the mature type. By contrast, species richness in New Zealand and in the Andes, two regions that have experienced recent periods of geological uplift and the creation of a wide range of diverse habitats, seems to stem from a number of recent radiations. Interestingly, both mature and recent radiations characterize the Cape flora, the former type associated with the stable geomorphology and climate of the Cape Fold mountains and the latter with the west coast area where a semi-arid habitat has originated in relatively recent times.

5. Concluding remarks

The great majority of the literature on speciation has emphasized the importance of reproductive incompatibilities that appear during geographical isolation. While some researchers have always questioned the dominance of allopatry and reproductive isolation in our thinking about the nature of species and the process of speciation (Howard & Berlocher 1998), the last few years have delivered particularly telling blows to these paradigms. Divergence in the face of some gene flow is almost certainly possible, especially where selection is strong. Genome-wide reproductive isolation is not necessary to maintain significant differences between what, by any reasonable definition, are entities functioning as distinct species. This means that hybridization is probably not uncommon between animal species as well as between many plant species, and some hybrid traits may aid the speciation process in both plants and animals. The last two decades have probably seen more change in our understanding of speciation than the previous 80 years. Even within the last 4 years, since the publication of Coyne & Orr's (2004) book Speciation, there have been major advances to our understanding of the nature and genetics of reproductive isolation in plants as made clear in this issue. Rapid improvements now occurring in the availability of genome databases, genomic techniques, phylogeographic and population genetic modelling, and phylogeny reconstruction and dating are expected to fuel further significant advances in this fast-moving field over the next few years.


  • One contribution of 12 to a Theme Issue ‘Speciation in plants and animals: pattern and process’.


View Abstract