Royal Society Publishing

Phylogenetic diversity measures based on Hill numbers

Anne Chao, Chun-Huo Chiu, Lou Jost


We propose a parametric class of phylogenetic diversity (PD) measures that are sensitive to both species abundance and species taxonomic or phylogenetic distances. This work extends the conventional parametric species-neutral approach (based on ‘effective number of species’ or Hill numbers) to take into account species relatedness, and also generalizes the traditional phylogenetic approach (based on ‘total phylogenetic length’) to incorporate species abundances. The proposed measure quantifies ‘the mean effective number of species’ over any time interval of interest, or the ‘effective number of maximally distinct lineages’ over that time interval. The product of the measure and the interval length quantifies the ‘branch diversity’ of the phylogenetic tree during that interval. The new measures generalize and unify many existing measures and lead to a natural definition of taxonomic diversity as a special case. The replication principle (or doubling property), an important requirement for species-neutral diversity, is generalized to PD. The widely used Rao's quadratic entropy and the phylogenetic entropy do not satisfy this essential property, but a simple transformation converts each to our measures, which do satisfy the property. The proposed approach is applied to forest data for interpreting the effects of thinning.

‘We are all blind men (and women) trying to describe a monstrous elephant of ecological and evolutionary diversity… ’(Nanney 2004, p. 721)

‘Phylogenetic measures are better indicators of conservation worth than species richness, and measures using branch-lengths are better than procedures relying solely on topology… ’(Crozier 1997, p. 243)

1. Introduction

An enormous number of diversity measures have been proposed, not only in ecology but also in genetics, economics, information science, linguistics, physics and social sciences, among others (e.g. Pielou 1975; Magurran 2004). Until recently, most of these measures were species-neutral, treating all species as if they were equally distinct. However, as Pielou (1975, p. 17) was the first to notice, the concept of diversity could be broadened to consider taxonomic differences between species. Many biologists (see Vellend et al. 2010 for a review) have recognized that, all else being equal, an assemblage of phylogenetically divergent species (say, eagle, magpie and dunlin) is in an important sense more diverse than an assemblage consisting of closely related species (magpie, blue magpie and tree pie). Since there was never real agreement among biologists about the simpler base concept of species-neutral diversity (e.g. Hulbert 1971; Routledge 1979; Patil & Taillie 1982; Purvis & Hector 2000; Jost 2007; Jost et al. 2010), and since there is even less agreement about how to incorporate phylogenetic differentiation (e.g. Crozier 1997; Faith 2002; Cavender-Bares et al. 2009; Pavoine et al. 2009; Vellend et al. 2010), we now face a hyperdiverse and rapidly increasing assemblage of non-neutral diversity measures.

We show below that most of these non-neutral measures lack an essential mathematical property implicit in biological reasoning about diversity. Conclusions based on these measures will often be invalid, especially in conservation applications (Hardy & Jost 2008). We derive a general class of measures that take into account both species abundances and species phylogenetic differences, and that possess all the mathematical properties implicit in standard biological reasoning about diversity. The new measures behave more intuitively than previous measures. Many of the previous measures can be transformed into this new class.

2. Previous non-neutral measures

Most of the non-neutral measures that have been proposed are generalization of the classic species-neutral ecological diversity measures: species richness, the Shannon entropy and the Gini–Simpson index. The pioneering work of Vane-Wright et al. (1991) generalized species richness to take into account cladistic diversity (CD), based on the total nodes in a taxonomic tree. Subsequent important work was done by Faith (1992, 1994), Crozier (1992, 1997), Weitzman (1992, 1998) and Warwick & Clarke (1995). Faith (1992) defined the phylogenetic diversity (PD) as the sum of the branch lengths of the phylogeny connecting all species in the target community. This concept of PD is essentially a measure of the total amount of evolutionary history embodied in an assemblage since the time of the most recent common ancestor of the assemblage. The branch lengths may be proportional to time of divergence, or they may be proportional to the number of base changes in a given gene or may use some other measures of change. If the branch lengths are proportional to divergence time, all branch tips are the same distance from the tree base (the first node). Such trees are called ‘ultrametric trees’ and have particularly simple mathematical properties.

These generalizations of species richness do not take into account species relative abundances, because nearly all early studies were based on a coarse spatial scale, and data were mostly collected from museum specimen records; thus, relative abundances could not be reliably estimated. These measures are still useful in many conservation purposes or in cases where species abundances are difficult to count, such as micro-organisms or clumped plants. However, species abundances, if available, provide a more complete description of the ecosystem, and it seems reasonable from the perspective of community ecology to weigh a lineage by the numerical importance of its descendants. There is also a strong practical motivation for using measures that weigh species by their abundance. In many ecosystems, most species are rare, and unreasonable or impossible effort is required to detect them all. Species richness is therefore very difficult to estimate reliably. By contrast, most abundance-based diversity measures can be reliably estimated from small samples.

Diversity measures combining both phylogeny and abundances have been proposed in the literature (Rao 1982; Solow et al. 1993; Solow & Polasky 1994; Warwick & Clarke 1995; Izsák & Papp 2000; Webb 2000; Ricotta & Szeidl 2006, 2009; Weikard et al. 2006; Hardy & Senterre 2007; Hardy & Jost 2008; Allen et al. 2009; Pavoine et al. 2009; Cadotte et al. 2010). Rao's quadratic entropy (Rao 1982), a generalization of the Gini–Simpson index, is the most well-developed of these. When pairwise differences between species are specified, Rao's Q gives the mean phylogenetic distance between any two randomly chosen individuals in the community: Embedded Image 2.1 where dij denotes the phylogenetic distance between species i and j, and pi and pj denote species relative abundance of species i and j. Ricotta & Szeidl (2009) proposed a transformation of Q if distances are normalized to the range of [0, 1]: Embedded Image 2.2 The advantages of this transformation will become clear in the following sections.

Allen et al. (2009) generalized the Shannon entropy to take into account phylogenetic differences. For a rooted tree, their phylogenetic entropy Hp is Embedded Image 2.3 where the summation is over all branches, Li is the length of branch i, and ai denotes the abundance descending from branch i. This measure includes the Shannon entropy as a special case.

For ultrametric trees, Pavoine et al. (2009) integrated Faith's PD, Allen et al.'s Hp and Rao's Q into a parametric class of measure called Iq. The parameter q corresponds to the order in the Tsallis (1988) generalized entropy. The three named measures correspond to the orders q = 0, 1 and 2; I0 = Faith's PD minus the tree height; I1 = Hp and I2 = Q.

3. The replication principle

While biologists have traditionally used the Shannon entropy and the Gini–Simpson index to quantify diversity, this practice is inconsistent with their own rules of inference about diversity. For example, users of these measures often judge the compositional similarity of two or more groups by taking the ratio of mean within-group diversity to total (pooled) diversity. If the within-group diversity is close to the total diversity, biologists infer that the groups are similar in composition. Yet, when used with the Shannon entropy or the Gini–Simpson index, this ratio does not directly reflect compositional similarity. When within-group diversity is high, the ratio approaches unity, supposedly indicating that the groups are nearly identical in composition, even if the groups are in fact completely distinct (no shared species).

Conservation biologists use diversity measures to judge the impact of human activities or to design conservation strategies. Yet, the Shannon entropy and the Gini–Simpson index can be very misleading when judging human impacts, and are logically self-contradictory when used to assess conservation plans (Jost 2009), because of their nonlinearity with respect to increasing diversity. We conclude that these measures, in spite of their popularity, do not capture biologists' notions of diversity. The forms of reasoning that biologists apply to diversity lead to invalid conclusions when used with these measures. These common forms of reasoning about diversity implicitly assume that diversity obeys the ‘replication principle’. The replication principle for species-neutral diversity states that if we have N equally large, equally diverse groups with no species in common, the diversity of the pooled groups must be N times the diversity of a single group. Many authors (MacArthur 1965, 1972; Whittaker 1972; Peet 1974; Routledge 1979; Jost 2006, 2007, 2009; Jost et al. 2010) have shown that only diversity measures that satisfy this replication principle or ‘doubling property’ (Hill 1973; Jost 2007, 2008, 2009; Ricotta & Szeidl 2009) give mathematically, logically and intuitively correct results. The replication principle is best known in economics, where it has long been recognized as an important property of concentration and diversity measures (Hannah & Kay 1977).

To see the importance of this property, compare the behaviour of the Gini–Simpson index Embedded Image with that of the inverse Simpson concentration Embedded Image, which does obey the replication principle. Consider an archipelago of 20 equally large, equally diverse islands, each with completely distinctive tree floras. There are no shared species among islands. Assume the tree floras have the frequency distributions of the trees of Barro Colorado Island, Panama. To measure the compositional similarity among the islands, ecologists will take the mean diversity of the islands (0.95) and divide it by the diversity of the archipelago as a whole (0.998). For this example, the ratio is 0.95/0.998 = 0.95, near unity, supposedly indicating that the islands are nearly identical in composition, even though the islands are actually completely distinct (no shared species). This ratio does not reflect compositional similarity. Doing the same with the inverse Simpson concentration gives a ratio of 20.3/406 = 1/20, the smallest possible value for a set of 20 equally large islands, correctly showing that they are completely distinct in composition. The same problems apply to the Shannon entropy, and are resolved by using the exponential of Shannon entropy.

Since the Shannon entropy and the Gini–Simpson index do not obey the replication principle, neither do their phylogenetic generalizations—Rao's quadratic entropy, Allen et al.'s phylogenetic entropy and Pavoine's generalized Tsallis entropy family. Though each of these has a useful interpretation, they cannot be applied directly to judge efficacy of conservation plans, magnitudes of human impacts or compositional similarity among groups. These considerations motivate our search for a family of PD measures that are sensitive to species relative abundances and that obey the replication principle.

Species-neutral diversity measures that do obey the replication principle (the ‘true’ diversity defined by Jost 2007) include species richness, the exponential of Shannon entropy and the inverse Simpson concentration. These are special cases of a general class of measures, known as Hill numbers (Hill 1973): Embedded Image 3.1 where S is the number of species, pi is the relative abundance of the ith species and the parameter q, called the ‘order’ of the diversity measure, determines its sensitivity to species frequencies. The measure 0D corresponds to species richness and 2D corresponds to the inverse Simpson concentration, giving roughly the number of ‘very abundant’ species in a community (Hill 1973). The measure is undefined when q = 1, but the limit as q approaches unity exists and equals Embedded Image 3.2 which is the exponential of Shannon entropy. Roughly, 1D measures the number of ‘common’ (or ‘typical’) species in a community. Hill numbers provide a unified framework for the three most popular groups of diversity measures, q = 0, 1 and 2.

The Hill numbers are interpreted as the ‘effective number of species’ or ‘species equivalents’ (MacArthur 1965, 1972; Hill 1973; Jost 2006, 2007). For any community, if we obtain a value qD = w, then the diversity of this community is the same as that of a community with w equally abundant species. Hill numbers will be the basis for our phylogenetic generalization. We give the appropriate phylogenetic generalization of the replication principle in §6.

4. Phylogenetic diversity measures

(a) Conceptual framework

To emphasize the conceptual simplicity of our framework, we first explain it verbally, and then derive the corresponding formulae. We start by considering a phylogenetic tree that uses divergence times to place the nodes (so that the tree is ultrametric). At any given moment t, we can find the species by slicing the tree as in figure 1a. We can find their ‘abundances’ by summing the abundances of their descendants in the present-day assemblage. These abundances are not estimates of the actual abundances of these ancestral species at time t, but rather measures of their importance for the present-day assemblage. The lineage diversity at time t can be found by dividing these abundances by the total abundance at this time t, and inserting these relative abundances into the equation for Hill numbers of order q, equation (3.1). We call this qD(t).

Figure 1.

(a) A hypothetical ultrametric rooted phylogenetic tree with four species. Three different slices corresponding to three different times are shown. For a fixed T (not restricted to the age of the root), the nodes divide the phylogenetic tree into segments 1, 2 and 3 with duration (length) T1, T2 and T3, respectively. In any moment of segment 1, there are four species (i.e. four branches cut); in segment 2, there are three species; and in segment 3, there are two species. The mean species richness over the time interval [−T, 0] is (T1/T) × 4 + (T2/T) × 3 + (T3/T) × 2. In any moment of segment 1, the species relative abundances (i.e. node abundances correspond to the four branches) are Embedded Image; in segment 2, the species relative abundances are Embedded Image; in segment 3, the species relative abundances are Embedded Image. (b) A hypothetical non-ultrametric tree. Let Embedded Image be the weighted (by species abundance) mean of the distances from root node to each of the terminal branch tips. Embedded Image. Note Embedded Image is also the weighted (by branch length) total node abundance because Embedded Image. Conceptually, the ‘branch diversity’ is defined for an assemblage of four branches: each has, respectively, relative abundance Embedded Image, Embedded Image, Embedded Image and Embedded Image; and each has, respectively, weight (i.e. branch length) 4, 3.5, 1 and 2. This is equivalent to an assemblage with 10.5 equally weighted ‘branches’: there are 4 branches with relative abundance Embedded Image; 3.5 branches with relative abundance Embedded Image; 1 branch with relative abundance Embedded Image and 2 branches with relative abundance Embedded Image.

We can average the diversities qD(t) of the phylogenetic tree over any time interval of interest. We will be interested in the time interval from −T years to the present time. While previous phylogenetic studies have focused on T as the age of the first node (root), we do not make this restriction, because we may want to compare diversities of systems with different ages of the first node. Also, how diversity varies with time for any individual tree provides important information about evolution.

The average diversity of order q over the interval [−T, 0] incorporates information about the tree's branching pattern, its relative branch lengths and the relative abundances flowing through each of its branch segments. For a given present-day diversity, this average will be large when there are many deep branches, each well represented in the present-day assemblage. It will be small when all branches emerge recently and/or when older branches are poorly represented in the present-day assemblage. It is always less than or equal to the present diversity of order q.

There are many ways to take this average. If we want the replication principle to be valid in its strongest possible form, then we must average the diversities qD(t) according to Jost's (2007) derivation of the formula for the mean (α) diversity of a set of equally weighted assemblages. This mean diversity over the time interval [−T, 0] will be called Embedded Image (mean diversity of order q over T years). With this choice of mean, when N maximally distinct trees with equal mean diversities (for fixed T) are combined, the mean diversity of the combined tree is N times the mean diversity of any individual tree. The branching patterns, abundances and richnesses of the N trees can all be different, as long as each of the trees is completely distinct (all branching off from the earliest point in the tree, at or before time T). Some choices of averaging formulae obey weaker versions of the principle, and these may be useful for some purposes. We discuss an alternative choice of mean in §8.

We may want to consider not just the mean diversity but the branch or lineage diversity of the tree as a whole, over the interval from –T to present. At any point within a branch, the abundance or importance of each branch lineage is the sum of the abundances of the present-day species descending from that point, as described above. Then the total diversity of all the ‘species’ that evolved in the tree during the time interval [−T, 0] is found by taking the Hill number of this entire virtual assemblage of ancestral species. The Hill numbers depend only on the relative abundances of each species, so we need to divide the abundances by the total abundance of all the species in the tree. If each branch is weighted by its corresponding branch length, then we show below that this diversity depends only on the branching pattern and on the relative abundances of the species in the present-day assembly. We call this measure ‘phylogenetic diversity of order q through T years ago’ or ‘branch diversity’ and denote it by qPD(T). This turns out to be just the product of the interval duration T and the mean diversity over that interval, Embedded Image. For q = 0 (only species richness is considered), and T = the age of the first node, this branch diversity is just Faith's PD.

Instead of using time as the metric for a phylogenetic tree, we often want to use a more direct measure of evolutionary work, such as the number of base changes at a selected locus, or the amount of functional or morphological differentiation from a common ancestor. The branches of the resulting tree will then be uneven, so the tree will not be ultrametric. However, we can easily apply the idea of branch diversity to such non-ultrametric trees. The branch lengths are calculated in the appropriate units, such as base changes. In non-ultrametric cases, the time T is replaced by Embedded Image, the mean of the distances from root node to each of the terminal branch tips (i.e. the mean evolutionary change per species); see figure 1b for a numerical example. Thus, we can obtain the total effective number of ‘changes’ based on Hill numbers.

(b) Formulae

To make the above discussion precise and derive formulae from it, we need to introduce some notation. Assume that for any fixed time T the phylogenetic tree is divided as k segments with duration T1, T2, … , Tk and species richness S1, S2, … , Sk as in figure 1a. Note that S1 = S, the present-day species richness. Each branching point must form a segment boundary, so that the species richness in any given segment is a constant. Our derivation and formulae would be unchanged by making finer segment divisions. To obtain the formulae for Embedded Image, assume there are Si species (i.e. Si branches cut) in the ith segment. Then, Embedded Image (mean diversity of order 0 over T years) is Embedded Image 4.1 When T is the time corresponding to the root, then 0PD(T) is Faith's PD measure. Our equation (4.1) connects Faith's PD to the mean species richness over the time interval from the terminal tips to the root.

At each moment within a given segment, the set of species relative abundances is constant. In segment 1, the species relative abundances are Embedded Image. Assume that in segment 2 the relative abundances are Embedded Image, … , and in segment k the relative abundances are Embedded Image Embedded Image (figure 1a). Without loss of generality, we can assume T1, T2, … , Tk are all positive integers, because the mean diversity Embedded Image is invariant to the units of time. Weighing each moment in time equally, we can conceptually imagine that there are T1 assemblages with abundance vector Embedded Image, T2 assemblages with abundance vector Embedded Image … , and Tk assemblages with abundance vector Embedded Image. There are a total of T1 + T2 +⋯ + Tk = T assemblages, and each is given the same weight 1/T. Jost (2007) showed that, in the context of calculating alpha diversity for equally weighted assemblages, the alpha diversity should be obtained by first averaging the sums of Embedded Image, … , and Embedded Image, and then converting this average to a ‘true’ diversity by raising it to the power 1/(1 − q). We use this same kind of average to obtain the formula for Embedded Image (mean diversity of order q over T years) Embedded Image 4.2 When q = 0, equation (4.2) reduces to equation (4.1). The same formula (4.2) may be computed more easily by numbering every branch in the time interval [−T, 0]. Denote the set of all branches in this time interval by BT. Then, Embedded Image can be calculated as Embedded Image 4.3 where Li is the length (duration) of branch i in the set BT and ai is the total abundance descended from branch i. This diversity may also be interpreted as the effective number of maximally distinct lineages (or species) during the interval [−T, 0]. For maximally distinct species we have all branch lengths equal to T, and thus Embedded Image reduces to Hill numbers qD in equation (3.1). This gives a simple reference tree for a value of Embedded Image, i.e. the observed mean diversity in the time period [−T, 0] is the same as the mean diversity of a community consisting of z equally abundant and maximally distinct species with branch length T.

The effective diversity of the whole tree during the interval [−T, 0] is the product of the effective number of lineages during the interval and the duration of the interval. We denote this measure by qPD(T) (phylogenetic diversity of order q through T years ago): Embedded Image 4.4 This has dimensions of ‘effective number of lineage years’. If q = 0, this equals 0PD(T) as defined above, regardless of branching pattern or abundances. If all species are maximally distinct and equally common, and if T is the age of the highest node, this equals Faith's PD for all q.

For an ultrametric tree, we can express the time parameter T as Embedded Image. Therefore, the time length T can also be interpreted as the total abundance (weighted by branch lengths) in the time interval [−T, 0] and ai /T represents the relative abundance of the ith branch. Using this idea, equation (4.4) suggests that instead of dividing the tree into several segments and treating the mean diversity as the alpha diversity of several assemblages, we could conceptually think of all the branch segments in the interval [−T, 0] as forming a single assemblage consisting of relative abundances Embedded Image, with each branch weighted by its corresponding branch length. (Equivalently, we can also think for each i that there are Li equally weighted ‘branches’ with the relative abundance ai/T.) Then the Hill number of order q for this assemblage is exactly the branch diversity qPD(T) given in equation (4.4). Dividing this Hill number by T, we obtain Embedded Image given in equation (4.3).

For the extension to non-ultrametric trees, let Embedded Image denote the set of branches connecting all focal species with mean base change Embedded Image. The total node abundance weighted by branch lengths is Embedded Image, which also represents the weighted (by species abundance) mean evolutionary change per species (figure 1b). (In ultrametric trees, Embedded Image.) Based on the assemblage consisting of all branches with relative abundance set Embedded Image and under the assumption that each branch is weighted by its corresponding branch length (figure 1b), parallel derivation gives the following measures, which are exactly the same as those in equations (4.3) and (4.4), except that the parameter T there must be replaced by the mean quantity Embedded Image: Embedded Image 4.5 and Embedded Image 4.6 We thus can conclude that the diversity of a non-ultrametric tree with mean evolutionary change Embedded Image (however this might be measured) is exactly the same as that of an ultrametric tree with time parameter Embedded Image. Therefore, for non-ultametric trees, if Embedded Image, then the diversity is the same as the diversity of an ultrametric tree consisting of z equally abundant and maximally distinct species with branch length Embedded Image.

(c) Relationship with Rao's Q and phylogenetic entropy Hp

In the limit as q approaches unity, the formula Embedded Image in equation (4.5) equals Embedded Image 4.7 The measure Embedded Image has the following simple relationship with the phylogenetic entropy Hp: Embedded Image 4.8 When q = 2, from equation (4.5), we have Embedded Image 4.9 After some algebra, we have the relationship between Embedded Image and Rao's quadratic entropy Q: Embedded Image 4.10 Formula (4.10) represents the equivalent number of completely distinct species (of age Embedded Image) for the assemblage. Ricotta & Szeidl (2009) derived a similar formula, given in equation (2.2), for the special case in which the pairwise distance between any two species is normalized to the range of [0, 1]. While their formula is identical to our equation (4.10) for ultrametric trees when our time parameter T is scaled to 1, for non-ultrametric trees, our theory leads to the conclusion that the equivalent number of species for Q should be Embedded Image.

We give an example to illustrate this point. Consider a non-ultrametric tree in which three equally abundant species are maximally distinct with branch lengths 1, 1 and 0.2, respectively, from a divergence point. The pairwise distances between the three species are d12 = 1, d13 = 0.6 and d23 = 0.6. We have Rao's Q = 4.4/9 = 0.489 and Embedded Image. Based on our equivalent number of species formula, we have Embedded Image maximally distinct species with equal branch lengths of 0.733, and the total length = 0.733 × 3 = 2.2, which is Faith's PD. However, based on the Ricotta & Szeidl (2009) formula, we obtain Embedded Image, implying there are 1.957 maximally distinct species with branch length of 1. The total length is thus 1.957 × 1 = 1.957, which is not Faith's PD.

5. Taxonomic diversity

Rather than using time or the number of base changes at a locus as our measure of evolutionary work, we might want to use a more holistic measure of evolutionary work, such as a phylogenetic tree based on the classical Linnaean taxonomic categories. Consider the special case in which each Linnaean taxonomic category is given unit length, and assume all species are classified in all levels. Our formulae above can be easily applied to this ultrametric tree, with T replaced by an integer representing the number of taxonomic categories needed to characterize the assemblage. We thus change the continuous time parameter T to an integer parameter L (level) to distinguish taxonomic diversity from the general PD measures Embedded Image and qPD(T). If we use species and genus, then L = 2; if we use species, genus and family, then L = 3. Additional intermediate levels, such as subgenus or subfamily, may be appropriate depending on the group. Notice that in a taxonomic tree, the total length is identical to the total number of nodes. Setting all the segment lengths Li to unity in equations (4.3) and (4.4), we have the following mean diversity of order q for L taxonomic levels, Embedded Image, Embedded Image 5.1 where i is over all nodes in the L levels taxonomy tree. The measure Embedded Image quantifies ‘the mean effective number of cladistic nodes per level in a taxonomic tree of L levels’. The diversity of a taxonomy tree with Embedded Image is the same as the diversity of a community consisting of z equally abundant species, with each species classified in its own genus and family, so that there are z species, z genera and z families.

The taxonomic diversity of order q for L levels, qTD (L), is the product of Embedded Image and the level L. This measure quantifies ‘the effective number of total cladistic nodes in a taxonomic tree of L levels’ and has the formula Embedded Image 5.2

In the special case L = 1, the measure Embedded Image. When q = 0, 0TD(L) = total number of nodes, which is Vane-Wright's CD. Equations (4.8) and (4.10) reduce to the following transformations: Embedded Image and Embedded Image; see table 1 for a summary of all proposed measures and their relationships with conventional measures. The decomposition of taxonomic diversity into diversity of each level is provided in the electronic supplementary material.

View this table:
Table 1.

A summary of species-neutral and phylogenetic diversity measures and their interpretations; all satisfy the replication principle. CD, cladistic diversity (total number of nodes) by Vane-Wright et al. (1991); PD, phylogenetic diversity (sum of branch lengths) by Faith (1992); Q, quadratic entropy, equation (2.1); Hp, phylogenetic entropy, equation (2.3).

6. Replication principle for phylogenetic diversity

Some basic properties of our proposed measures (table 1) are summarized in the electronic supplementary material; details of the proofs are provided in Chiu (2010) and Jost & Chao (in preparation). In this section, we only refine the concept of the replication principle for phylogenetic trees, and prove its validity for the most general case (i.e. non-ultrametric case), implying that it is valid for all measures in table 1.

Suppose we have N completely distinct assemblages (no shared lineages), all with the same mean branch length Embedded Image (hence same T in the case of ultrametric trees) and the same mean PD Embedded Image. Then we can prove the following strong replication principle: if these assemblages are pooled in equal proportions, the pooled assemblages have mean PD N × X.

Proof. Suppose in tree k, the branch set is Embedded Image (we omit Embedded Image in the subscript and just use Bk in the following proof for notational simplicity) with branch lengths Embedded Image and the corresponding nodes abundances Embedded Image, k =1, 2, … , N. The N trees have the same mean diversity X, implying Embedded Image for all k =1, 2, … , N. When the N trees are pooled with equal weight for each tree, each node abundance aik in the pooled tree becomes aik/N. Then, the Embedded Image measure for the pooled tree becomesEmbedded Image In our proof of this replication principle, the N assemblages must have the same average quantity Embedded Image, but may have different numbers of species if q > 0, and the tree structures of the N assemblages can be totally different.

7. Examples

To show the general behaviour of our proposed measures, we give two simple hypothetical examples in the electronic supplementary material. Here, we apply the proposed Embedded Image and qPD(T) measures to the real forest data discussed by Shimatani (2001), who collected data from the over-storey tree species in the Fred Russ experimental forest in Michigan. For illustrative purpose, we only consider the abundance data of block 4 in his paper for two sites: CT (thinned site) and CU (un-thinned site). Both sites were 28-year-old (in 1999) secondary forests. The two sites were dominated by oak trees. No thinning was conducted for the CU site after clear cutting in 1971, while thinning was done for non-oak species in the site CT in 1982 and 1996.

Shimatani (2001) proposed a four-level (species, genus, family, subclass) taxonomic measure based on the Simpson index, and concluded that the traditional diversity indices and the taxonomic diversity considering species relatedness give different conclusions about the effect of thinning. We constructed the phylogeny trees for species in each site by using the software Phylomatic (from; Webb & Donoghue 2004). The phylogenetic tree for the species in the two sites, and the two sets of species relative abundances, are shown in figure 2. We calculated three types of diversities: (i) the mean diversity Embedded Image and the phylogenetic diversity qPD(T) based on the phylogeny trees and the relative abundances in figure 2, (ii) the taxonomic diversity Embedded Image based on taxonomic classification in fig. 1 of Shimatani (2001), and (iii) the species-neutral diversity based on Hill numbers (qD) in equation (3.1) for q = 0, 1 and 2.

Figure 2.

The combined phylogenetic tree for the species in the site CT (grey line) and site CU (black line). The age of the root for the CT site is 116.6 units and 142.3 units for the CU site and the pooled site. The species relative abundance (%) in the two sites CT and CU are shown in the last two columns (abundance data are from Shimatani 2001).

In figure 3, the profile of Embedded Image and qPD(T) when 0 < T < 150 is shown for q = 0, 1 and 2. For ultrametric trees, the two measures give consistent comparison as clearly seen in figure 3. We focus on comparing the measure Embedded Image, which gives the mean effective number of species as a function of evolutionary time T. Based on species richness (q = 0), the diversity Embedded Image of the thinned site CT dominates that of un-thinned site CU for all values of T. But for the common species (q = 1) and very abundant species (q = 2), we have the reverse conclusion. When abundance is taken into account, the un-thinned CU site is more diverse than the thinned CT site for all values of T, except for a very small interval in the case of q = 2.

Figure 3.

The profile of Embedded Image and qPD(T) measures for 0 < T < 150 and q = 0, 1 and 2. Solid line, CT site; dashed line, CU site.

Table 2 shows the three types of diversity (Embedded Image and qD) for three orders of q (0, 1 and 2). All these three measures are in the same units of species. The Embedded Image measure is only shown for T = 142.3, which is the age of the root in the pooled phylogenetic tree. The taxonomic measure Embedded Image is computed for L = 4 level classifications. For any fixed order q, we had proved that qD is always greater than or equal to Embedded Image and Embedded Image, and this is seen numerically in table 2.

View this table:
Table 2.

Comparison of three types of diversities (Embedded Image, Embedded Image and qD) for q = 0, 1 and 2. The Embedded Image value is computed at T = 142.3, which is the age of the root of the pooled tree. See figure 3 for other values of Embedded Image.

Based on table 2, we confirm the finding of Shimatani (2001) that the traditional Simpson diversity measure 2D implies that the thinned site is less diverse. A similar implication is also valid for the 1D measure, whereas species richness 0D shows that the thinned site is more diverse. Based on Embedded Image, the taxonomic diversity of the thinned site for all three orders is greater, but the difference is not large. Shimatani thus concluded that the thinning operation contributed to an increase in taxonomic diversity.

In contrast to Shimatani's conclusion, our results based on Embedded Image for q = 1 and 2 imply the opposite conclusion, as shown in figure 3, and our results are consistent with those based on the species-neutral diversity. Our conclusion may be understood intuitively by noting that thinning concentrates the abundance into a few species of intermediate phylogenetic distinctiveness (figure 2), while in the un-thinned site, abundance is spread more equitably throughout the phylogenetic tree. The plots in figure 3 provide additional insights about the thinning effect when both evolutionary history and species abundances (q = 1 and 2) are considered.

8. Concluding remarks and discussion

(a) Advantages of the new measures

We have proposed a unified class of PD measures that are based on Hill numbers and that obey the replication principle (§§3 and 6). Most previous PD measures that take into account species abundances, such as Rao's (1982) quadratic entropy Q, Allen et al. (2009) phylogenetic entropy Hp and Pavoine et al. (2009) generalized phylogenetic entropy Iq, do not obey the replication principle.

Measures that do not obey the replication principle give self-contradictory results in conservation analyses (Jost 2009). Furthermore, for such measures, the commonly used ratio of within-group to total ‘diversity’ does not reflect the compositional similarity of the groups, since it always approaches unity when diversity is high (§3 and Hardy & Jost 2008). Finally, it is difficult to use such measures to judge the magnitude of human or natural impacts on the environment. The problem with these measures is their nonlinearity with species addition. A numerical example is provided in the electronic supplementary material. Our measures solve these problems.

If a dendrogram can be constructed from a trait-based distance matrix using a clustering scheme (Petchey & Gaston 2002), then we can apply our proposed measures to quantify functional diversity; see Chao & Jost (2011) for interpretation. Our proposed approach can also be extended to the case of multiple communities. The formulations of phylogenetic alpha, beta and gamma diversities as well as the construction of similarity (or differentiation) measures are developed in Chiu (2010). These results will be reported in forthcoming papers.

(b) Interpretation of the new measures

For ultrametric trees, the mean diversity (in unit of species) Embedded Image, defined in equation (4.3), quantifies ‘the mean effective number of species from the present to T time units ago’. Here the parameter q determines the diversity's sensitivity to node (or branch segment) abundances; high values of q emphasize those nodes with high relative abundances. The product of Embedded Image and T is the phylogenetic diversity measure qPD(T), defined in equation (4.4), and quantifies the ‘effective branch diversity’ of the phylogenetic tree. For a non-ultrametric tree, the only difference is in the replacement of T by the mean evolutionary change Embedded Image (the mean of the distances from root node to each of the terminal branch tips.). See table 1 for a summary of the proposed measures.

For ultrametric trees, the most complete picture of PD is provided by graphing it as a function of T. We recommend three profiles for q = 0, 1 and 2 (figure 3) and a range of time between 0 and a maximum value T (such as the age of the first node or the age at which the group of interest diverges from other groups or the time of origin of life). Profiles may also be taken for fixed T (using the T values just described, for example) as a function of q. Such profiles will show the effect of taking abundances into account (q = 0 gives no abundance accounting, while high q takes into account only the most abundant species). For non-ultrametric trees, similar recommendations can be made based on the mean base change.

Species-neutral diversity measures discussed in this paper are featured in the program SPADE (Chao & Shen 2010), which can be freely downloaded from the website The new PD measures will be featured in the program PhD (phylogenetic diversity) in the same website.

(c) Alternative formulation

We have developed our new measures to obey the strongest possible version of the replication principle, facilitating decomposition into independent within- and between-group components. This was accomplished by taking the average of qD(t) over the time interval T using the mean derived by Jost (2007). However, some other kinds of means may also yield useful results. If we had used the ordinary mean of qD(t) over the time interval T, we would obtain the expectation value of qD(t) over the interval T. Multiplying this by T would give a measure of the amount of evolutionary history embodied by the tree in this interval, or the amount of evolutionary work done on the assemblage during this interval. This product would be monotonically increasing in T, an advantage over the formulation we have developed above. However, this alternative mean does not obey the strong version of the replication principle, but only the following weaker one: when N maximally distinct trees with equal diversities at each time t, and equal total abundances, are combined, the mean diversity of the combined trees is N times the mean diversity of any individual tree. When this weaker version of the replication principle is deemed sufficient, the alternative formulation may be useful in some applications.


This paper is dedicated to Ross Crozier, a pioneer in the phylogenetic research and in the study of genetics in social insects. Ross unfortunately passed away in November 2009. A.C. sincerely thanks Ross for his friendship of many years and for his inspiring and encouraging discussion and guidance to the field of phylogenetic diversity. We also acknowledge helpful discussions with Carlo Ricotta, Olivier Hardy and Bruno Senterre. The valuable comments and suggestions from Nick Gotelli and an anonymous reviewer helped substantially improve the paper. A.C. and C.C. were supported by Taiwan National Science Council. L.J. is grateful for support by a grant from John Moore to the Population Biology Foundation.



View Abstract