Abstract
Despite the prevalence of protected areas, evidence of their impacts on people is weak and remains hotly contested in conservation policy. A key question in this debate is whether socioeconomic impacts vary according to social subgroup. Given that social inequity can create conflict and impede poverty reduction, understanding how protected areas differentially affect people is critical to designing them to achieve social and biological goals. Understanding heterogeneous responses to protected areas can improve targeting of management activities and help elucidate the pathways through which impacts of protected areas occur. Here, we assessed whether the socioeconomic impacts of marine protected areas (MPAs)—designed to achieve goals for both conservation and poverty alleviation—differed according to age, gender or religion in associated villages in North Sulawesi, Indonesia. Using data from pre-, mid- and post-implementation of the MPAs for control and project villages, we found little empirical evidence that impacts on five key socioeconomic indicators related to poverty differed according to social subgroup. We found suggestive empirical evidence that the effect of the MPAs on environmental knowledge differed by age and religion; over the medium and long terms, younger people and Muslims showed greater improvements compared with older people and Christians, respectively.
1. Introduction
An evidence-based approach to the design and implementation of conservation interventions is increasingly advocated as a means to improve their outcomes for both ecosystems and people (e.g. [1,2]). Evidence-based approaches review the impacts of past management interventions and apply the knowledge gained to decision-making about future interventions. Despite enthusiasm to move beyond ‘conservation practice based on anecdote and myth’ [3], evidence-based conservation is severely impaired by a lack of knowledge, or evidence, of what conservation actions work, where they work and why.
Most of the recent literature concerning evidence-based conservation and impact evaluation has focused on the biological realm [4,5], but understanding the impact of conservation interventions on people is central to designing interventions that are likely to meet biological objectives, let alone contribute to human well-being. Given the purpose of conservation interventions is generally to modify human behaviour, achieving biological gains rests largely on enlisting stakeholders' support, which in turn is heavily influenced by their perceptions of the costs and benefits of management. Thus, designing conservation interventions to benefit associated human communities is often crucial for achieving success in biological terms. More importantly, managers have an ethical responsibility, at the very least, to ‘do no harm’ to stakeholders [6]. Given that people's well-being is inextricably linked to their natural environment [7,8], conservation interventions have the potential to significantly impact people, both positively and negatively. For these reasons, the link between nature conservation and human well-being is increasingly emphasized in international policy (e.g. [6,8]) and reflected in conservation organizations' mandates and activities [9,10]. This integrated social–ecological approach is implemented in a number of forms, including community-based conservation, co-management, and integrated conservation and development.
The evidence base for socioeconomic impacts of protected areas, the cornerstones of many biodiversity interventions [8], is weak [5,11]. Reviews (e.g. [11–13]) have found that studies of socioeconomic impacts of protected areas are dominated by qualitative case studies, and the few existing empirical studies tend not to focus on causal identification of impacts. Although qualitative studies are critical to understanding the impacts of protected areas on people [14], quantitative impact evaluations can help untangle the effects of protected areas from confounding factors—factors that can affect the outcome of interest and that operate simultaneously with the intervention [2]. Assessing causal effects requires estimation of the counterfactual (what would have happened in the absence of the intervention). Ideally, such an assessment would use longitudinal data from before and after an intervention for both control and intervention sites [11]. However, such data are very rare; only two studies are known to have used this kind of data to estimate the socioeconomic impacts of protected areas [15,16]. Recently, a number of studies concerning the socioeconomic impacts of terrestrial protected areas have applied rigorous evaluation techniques to estimate the counterfactual (e.g. [17–19]). These studies have significantly advanced the evidence base for protected areas by providing relatively bias-free estimates of impacts. However, many of these studies adopted an aggregated approach to evaluation, focusing on net impacts determined via a single metric of poverty, over a single time period.
Empirical evidence of the heterogeneity of socioeconomic impacts of protected areas across space and time, and among dimensions of poverty and social subgroups is particularly weak, and has recently been highlighted as a research frontier (e.g. [4,20]). Understanding this heterogeneity is important for gaining a comprehensive picture of how protected areas affect people, including the trade-offs likely to arise (e.g. among dimensions of poverty or social subgroups). In regards to temporal heterogeneity, although the impacts of protected areas can be related to their duration of establishment [21,22], the few evaluations that use longitudinal data [11] tend not to examine impacts over multiple time periods (but see [15]). Existing evaluations have also generally overlooked potential heterogeneity of socioeconomic impacts, generally focusing on one or very few poverty metrics [13,23].
Understanding how protected areas can differentially affect social subgroups is particularly critical because social inequity can create conflict [24] and impede poverty reduction [25], thus jeopardizing social and biological goals [26]. The prevailing approach of using aggregated data to assess mean impacts [27] can mask inequalities, and is likely due to the failure of much of conservation practice to adequately recognize and account for the heterogeneous social structure of communities [28,29]. Priorities for resource use and management, and capacities and powers to defend those priorities, are likely to differ according to social subgroups, defined by factors such as gender, age, ethnicity, religion and occupation [30]. Failing to recognize the heterogeneous nature of communities may thus result in inequitable distribution of costs and benefits. This may manifest as ‘elite capture’, whereby local elites use their positions of power to protect and promote their interests at the expense of the marginalized [31].
Although the potential for inequitable impacts of protected areas is commonly discussed in the literature (e.g. [24,32]), empirical evidence is lacking. Existing literature relating to inequality and resource management often considers inequality as a driver of outcomes, rather than as an outcome of management [26]. Meanwhile the literature on programme evaluation has tended to focus on mean treatment effects [33]. To our knowledge, only three studies employing impact evaluation techniques have assessed heterogeneity of socioeconomic impacts of (terrestrial) protected areas according to social subgroups, in relation to livelihood strategies [34] and level of poverty at the scale of census tracts [33,35]. A related study by Sim [18] examines the impacts of terrestrial protected areas on inequality of consumption within localities (indexed by the Gini coefficient).
To contribute to a sounder evidence base for the socioeconomic impacts of protected areas, we addressed the critical question of whether the socioeconomic impacts of protected areas vary according to social subgroup. Specifically, we provide empirical evidence of how marine protected areas—designed to achieve the dual goals of conservation and poverty alleviation (hereafter ‘integrated MPAs’)—affected associated human communities in North Sulawesi, Indonesia. Using data from pre-, mid- and post-implementation for villages with and without MPAs, we asked ‘Do socioeconomic impacts of integrated MPAs differ according to age, gender or religion over the short, medium and long terms?’
2. Methods
(a) Integrated marine protected areas in North Sulawesi
The Coastal Resources Management Project (CRMP; locally known as Proyek Pesisir) implemented integrated MPAs (all less than 14 ha) during 1997–2002 in four villages in North Sulawesi, Indonesia (electronic supplementary material, figure S1). The project was jointly run by USAID and Indonesia's National Development Planning Agency (BAPPENAS), with a cost of over US$1.4 million [36]. Integrated-MPA plans were developed through a participatory planning process lasting 2 years, after which they were formally adopted by village ordinance. Various development activities were simultaneously carried out under the CRMP, including improving sanitation and access to drinking water, livelihood training and flood prevention. After the withdrawal of external support in 2002, the villages continued to manage their MPAs to varying extents; at the time of this study, MPA rules were not enforced in any of the villages and only MPAs in the villages of Blongko and Talise were still marked with buoys. Evaluation of the socioeconomic impacts of the integrated MPAs over 15 years (i.e. 1997–2012) found that they appeared to contribute to poverty alleviation, particularly during the 5-year implementation period [15]. We build on this previous evaluation by assessing whether these significant impacts differed according to social subgroups.
(b) Sampling
We studied eight villages in North Sulawesi (electronic supplementary material, figure S1). The four project villages were assessed four times: pre-, mid- and post-implementation of the integrated MPAs (1997, 2000 and 2002, respectively), and in 2012, 10 years after the withdrawal of external support. To estimate the counterfactual outcomes, we concurrently studied four control villages (electronic supplementary material, figure S1). These were selected to match project villages in terms of key factors thought to influence poverty in North Sulawesi, the most important being distance to markets and roads, but also population size and fisheries dependence. We used household surveys to gather quantitative data on several socioeconomic indicators, followed by semi-structured interviews with key informants, including heads of villages, members of MPA groups and traditional leaders, to obtain qualitative data. The two kinds of data were intended to triangulate results and aid our understanding of the possible causal mechanisms behind changes in socioeconomic indicators. In each sample period, households within villages were systematically sampled, whereby a sampling fraction of every ith household (e.g. 2nd, 3rd, 4th) was determined by dividing the total village population by the sample size [37]. Thus, panel data were collected at the community scale, not at the household scale. We surveyed over 2000 respondents during the entire study. At each village at each point in time, the number of surveys conducted per village ranged from 40 to 140, depending on the population of the village and available time at each site.
(c) Socioeconomic indicators
We examined five socioeconomic variables related to poverty (hereafter ‘socioeconomic indicators’) for heterogeneous impacts of the integrated MPAs (table 1). In the context of fishing villages in Indonesia, these represent important socioeconomic variables related to poverty that could be positively or negatively affected by MPA projects, and thus were a key focus of the poverty alleviation component of CRMP. Fisheries dependence and livelihood diversity are indicators of occupational flexibility [38], and wealth, perceived well-being and environmental knowledge represent material and human assets [15]. Flexibility and assets are key elements of people's vulnerability to shocks to the social–ecological system [39,40] and, more generally, their ability to escape and avoid poverty traps [41]. For example, understanding the role of humans as both a cause of marine environmental degradation and a source for positive change, which we captured in environmental knowledge indicator, is critical for people highly dependent on marine resources. We restricted the analysis to these five indicators because: (i) they were measured at each of the four sampling times and (ii) evidence of an unconditional treatment effect (i.e. overall impact of integrated MPAs that was not conditional on subgroups) was previously found for each of the indicators [15]. Using Pearson or Polyserial correlations as appropriate for the respective types of data, we found very weak correlations between each of the indicators (all were less than 0.14).
Descriptions of the socioeconomic indicators that were assessed for heterogeneous impacts of the integrated MPAs.
(d) Social subgroups
We examined whether the socioeconomic impacts of the integrated MPAs differed according to three social subgroups: age, gender and religion. The literature on how conservation and resource management affects people suggests that impacts can vary according to a range of social subgroups, including gender, religion, age, wealth, ethnicity and occupation (e.g. [20,42,43]). A key mechanism thought to generate heterogeneous socioeconomic impacts is level of participation in management activities and decision-making (e.g. [43–45]). Guided by this assumption, we selected our social subgroups based on the socioeconomic characteristics that Pollnac et al. [36] found were correlated with participation in integrated-MPA activities. For example, given that participation in integrated-MPA activities was significantly higher for men than women [39], we expected the impacts of the integrated MPAs to differ by gender. Of the variables that Pollnac et al. [36] found were correlated with participation, we chose three subgroups that were easily observable to MPA managers and thus could be used for targeting of management actions.
We treated age as a dichotomous variable, whereby respondents were classed as ‘younger’ if their age was below the median age at baseline (33 years), and ‘older’ if it was 33 years and above. Our results were robust to a ±10% change in the median value. The most common religions in North Sulawesi are Christianity and Islam; all of our respondents described themselves as following one of these faiths.
(e) Data analyses
To assess whether integrated MPAs significantly affected the socioeconomic indicators, we drew on the difference-in-difference method [46], a quasi-experimental technique from the econometrics literature on impact evaluation. The difference-in-difference method compares changes in outcomes over time between impact and control groups [46]. The design involved testing for a significant interaction between three explanatory variables—time, presence of integrated MPAs and a social subgroup variable (either age, gender or religion)—on each of our socioeconomic indicators (our response variables). A significant interaction between the three explanatory variables indicates that the effect of the integrated MPAs differed between the subgroup categories (e.g. between men and women). We tested for these three-way interaction effects over different time periods for each of the socioeconomic indicators individually. The time periods assessed were 1997–2000, 1997–2002 and 1997–2012, representing the short-, medium- and long-term impacts of the integrated MPAs, respectively. We used data only from household heads for the models of wealth, fisheries dependence, and livelihood diversity, because these socioeconomic indicators were measured at the household scale. Our sample of female household heads was too small to examine whether the impacts on these socioeconomic indicators differed between genders.
We tested for interaction effects using hierarchical regression models. To account for the clustered structure of our data (i.e. individuals nested within villages), we a priori set village as a random factor, which ensured more accurate estimation of the standard errors by imposing a correlation structure on all samples from the same village. We used a binomial distribution for models that had a dichotomous categorical response variable (i.e. perceived well-being and fisheries dependence), and a Gaussian distribution for models that had a continuous response variable (the remaining socioeconomic indicators). We used fixed factors to control for the subgroup variables for which we were not testing for heterogeneous impacts; for example, for models testing for heterogeneous impacts according to gender (i.e. the three-way interaction including gender), we controlled for age and religion by specifying them as fixed factors. We used Bayesian estimation, with non-informative uniform priors, so the posterior estimates were informed by the data alone (electronic supplementary material, text S2).
Our application of the difference-in-difference method assessed the conditional average treatment effect on the treated. Thus, in our analysis, the parallel trends assumption underpinning the difference-in-difference method—that in the absence of the project, changes in outcomes in project and control units are the same [46]—must hold within each subgroup. For example, changes in perceived well-being over time for Muslims in control villages should reflect changes in perceived well-being of Muslims in project villages in the absence of the project. This assumption depends in part on whether there are systematic differences in observable and unobservable factors that affect outcomes between project and control villages within subgroups. Following Ferraro & Miranda [47], to provide evidence as to whether there were systematic differences, we assessed differences in baseline values of socioeconomic indicators between project and control villages within each subgroup. We used hierarchical Bayesian regression models, with village set as a random factor, and a binomial or Gaussian distribution as appropriate for the respective types of data (electronic supplementary material, text S2). We found no evidence that there were differences in baseline values of socioeconomic indicators between project and control villages within subgroups except in relation to perceived well-being (electronic supplementary material, table S2). This suggests that the parallel trend assumption is likely to hold, except in relation to perceived well-being, and thus those results in particular should be interpreted with caution.
We used standardized effect sizes to illustrate estimated differences between the changes in socioeconomic indicators in the project and control villages within each social subgroup, which allowed us to compare across indicators based on different measures. We used Cohen's d effect statistic with a bias correction for all continuous socioeconomic indicators, and odds ratios for the remaining categorical indicators. All analyses were undertaken using R (3.02) and JAGS (3.4.0) statistics packages.
3. Results
We found little empirical evidence that the socioeconomic impacts of the integrated MPAs differed according to age, gender or religion, except in relation to environmental knowledge (table 2 and figure 1).
Estimated impacts over different time periods of the integrated MPAs on the five socioeconomic indicators conditional on three social subgroups, namely gender (pink lines), religion (grey lines) and age (orange lines). (a) Short-term changes 1997–2000, (b) medium-term changes 1997–2002 and (c) long-term changes 1997–2012. The five axes of the spider diagram relate to the five socioeconomic indicators as follows: LD, livelihood diversity; FD, fisheries diversity; W, wealth; WB, perceived well-being; EK, environmental knowledge. Labels on the rings of spider plot are effect sizes which represent estimated difference between the change in socioeconomic indicator in the project and control villages within each social subgroup. Effect sizes for fisheries dependence and perceived well-being were calculated using odds ratios and are represented here on a logarithmic scale. The effect statistic used for the other indicators was Cohen's d with a bias correction. Age is represented here as a dichotomous variable, whereby respondents were classed as ‘young’ if their age was below the median value for age, and ‘old’ if their age was above the median value for age.
Summary results of Bayesian hierarchical regressions that tested for an interaction between presence of integrated MPAs, time and social subgroup, for each socioeconomic indicator over multiple time periods. Mean posterior estimates are bolded for interactions that are supported by strong evidence for an effect (i.e. where the 95% highest posterior density intervals did not intersect zero). Regressions included as fixed factors the social subgroup variables for which we were not testing for heterogeneous effects; for example, for models testing for heterogeneous impacts according to gender (i.e. the three-way interaction included gender), we controlled for age and religion by specifying them as fixed factors.
Our results suggest that the impact of the integrated MPAs on perceived well-being and environmental knowledge did not differ between men and women. Although it appeared that men's environmental knowledge benefited more from the integrated MPAs than women's, and that the negative impact on women's perceived well-being was greater than that on men's (figure 1), we did not find strong evidence that impacts differed between genders (table 2).
We found no evidence that the socioeconomic impacts of the integrated MPAs differed by age, except in relation to environmental knowledge over the medium (i.e. 1997–2002) and long terms (i.e. 1997–2012; table 2). Our analysis suggests that younger people's environmental knowledge benefited from the integrated MPAs more than older people's (figure 1). The impact of the integrated MPAs on household-scale socioeconomic indicators did not vary according to age of the head of household (table 2); for both younger and older age groups, livelihood diversity and wealth were positively affected and fisheries dependence was negatively affected across the three time periods (figure 1).
We found no evidence that there was differential impact of the integrated MPAs according to religion, except in relation to environmental knowledge over the medium and long terms (table 2). The impact of the integrated MPAs on environmental knowledge was positive for both religious groups, although our analysis suggested that Muslims benefited more.
4. Discussion
Evidence-based policy decisions concerning the steadily increasing number of protected areas globally are inhibited by poor understanding of whether the socioeconomic impacts of this conservation tool differ according to social subgroup [11,20]. Our study found little empirical evidence that the socioeconomic impacts of the integrated MPAs differed according to age, gender or religion. Environmental knowledge was the only indicator for which we found suggestive evidence of differential effects of the integrated MPAs; over the medium and long terms, younger people and Muslims showed greater improvements compared with older people and Christians, respectively.
(a) Differential impacts according to gender
We found no evidence that the impact of the integrated MPAs on perceived well-being and environmental knowledge differed along gender lines. Given that the negative impact of the integrated MPAs on perceived well-being was felt equally by both genders, despite men participating more in integrated-MPA activities than women [36], it appears that these negative impacts were not related to individual participation. Gurney et al. [15] suggested that this negative impact could have arisen from conflict in relation to the project and unrealized expectations of project benefits, which are likely to act at the community scale. Although men made up the majority of participants in integrated-MPA activities overall, the proportion of men and women attending environmental education sessions was roughly equal (e.g. 51% of participants in environmental education during 2000–2002 were men; [36]). This equality is likely to explain the modest positive impact of the integrated MPAs on environmental knowledge for both genders.
The integrated MPAs could have differentially impacted men and women in respect to other socioeconomic indicators for which we did not have data. This is likely because the majority of participants in integrated-MPA meetings, during which decisions about resource-use rules and management activities were made, were men (e.g. 71% of participants at meetings during 2000–2002 were men; [36]). Indeed many of our female respondents mentioned that they had not wanted to attend or speak up in integrated-MPA meetings, because it was not their role to be involved in village decision-making, and that their male family members would represent the whole family. This was particularly the case in Muslim families, with women's roles tending to be more strictly defined to the household sphere. Previous studies have highlighted how existing social norms defining women's behaviour and role have led to inequitable impacts of devolved resource management through women's interests not being adequately considered in male-dominated decision-making (e.g. [43]). For example, resource restrictions can differentially affect livelihoods according to gender when women and men use different resources and these differences are not adequately considered in decision-making (e.g. [48,49]). Our qualitative data suggest that such a situation occurred in the village of Blongko; female respondents mentioned that, because they were not involved in decisions about placement of the MPA, access to the reef area used by women for gleaning was significantly restricted. To avoid potential inequitable impacts of resource management along gender lines, it is imperative that projects such as integrated MPAs initially conduct gender analyses to determine potential gendered use of resources, and identify existing norms regarding women's roles in decision-making. Even if women attend decision-making meetings, they may be unable or reluctant to make their concerns heard [44], suggesting that their inclusion should be fostered through other means, such as women's groups or informal consultation.
(b) Differential impacts by age
The socioeconomic impacts of the integrated MPAs did not differ by age, except in relation to environmental knowledge. Environmental knowledge of younger people showed large increases over the medium and long terms, compared to older people. Evaluations of environmental education associated with conservation projects in Malaysia [50] and Indonesia [51] also found that younger people were more likely to show positive changes in environmental knowledge and attitudes. Youth are often more open to new knowledge and better able to absorb information provided by education activities [52], perhaps in part because they have attended school more recently, and the education they received is more likely to align with that provided by the project. We also found differential impacts on environmental knowledge according to age over the long term (i.e. 1997–2012). Thus, it appears the positive impacts on youths' environmental knowledge during the implementation period (i.e. the medium term) prepared them to better respond to and benefit from broader scale factors that were responsible for increases in environmental knowledge in project and control villages after the project finished in 2002 [15].
We found no evidence that the impact of the integrated MPAs on household-scale socioeconomic indicators differed according to age of the head of household. Thus, it appears that the livelihoods of households headed by younger or older people were equally affected by restrictions to fishing areas and integrated-MPA programmes aimed at improving livelihoods and living conditions. However, households are not homogeneous, and intra-household variability in poverty can occur according to age and gender [14]. Resource management can contribute to intra-household inequity if, for example, age influences how and which resources people use [53]. Elderly people may be less mobile and thus less able to cope with changes in access to resources; indeed Cinner et al. [54] found that age influenced fishers' decisions to exit the fishery. Thus, variability in impacts of the integrated MPAs according to age could have been masked due to our examination of these factors at the household scale, suggesting that future socioeconomic impact evaluations use the individual as the unit of analysis.
(c) Differential impacts by religion
The socioeconomic impacts of the integrated MPAs did not differ according to religion, except in relation to environmental knowledge. While environmental knowledge of both religious groups increased over the three time periods, Muslims' environmental knowledge benefited more from the integrated MPAs in the medium and long terms. This is probably because Muslims were more likely to participate in integrated-MPA activities [36]. A number of our key informants suggested that integrated-MPA trainings related to marine-resource use and management, in particular, were more often attended by Muslims. Studies have highlighted how religious beliefs concerning nature can affect resource use and management, and thus the potential socioeconomic impacts of protected areas (e.g. [55]); for example, establishment of a strictly protected area in Mongolia adversely affected residents for whom the area was important for religious practices [56]. However, in our study, differential participation in and impacts of the integrated MPAs according to religion are likely because Muslims are more dependent on fisheries than Christians, rather than due to differences in religious outlooks. Muslims tend to live directly on the shoreline and are more reliant on fishing and other marine livelihoods (e.g. seaweed farming) than Christians, who tend to live further inland and practice farming. Thus, it is likely that Muslims' environmental knowledge benefited more from the integrated MPAs, because the trainings and education activities were more relevant to them than to Christians, leading to higher rates of participation. Further, given Muslims' lives are more related to the sea than Christians', they could have had greater existing knowledge of the marine environment, enabling them to better absorb the new information provided through the integrated-MPA activities.
The impact of the integrated MPAs on household-scale socioeconomic indicators did not differ by the religion of the head of household. Although Muslims participated more in integrated-MPA activities overall [36], we found no evidence they received more of the benefits related to wealth and livelihood diversity. This could be in part because many of the integrated-MPA activities were focused on generating village-level benefits, such as building dykes to prevent flooding. Further, many of the integrated-MPA activities that were not focused on the marine environment, which our qualitative data suggest were less often attended by Muslims, were directly focused on strengthening livelihoods (e.g. facilitating land tenure and improving farming productivity). By contrast, activities related to the marine environment, which made up the majority of the integrated-MPA activities, were more varied and included environmental education, MPA management training and mangrove protection.
(d) Critique and caveats
The robustness of our results depends in part upon whether the untestable parallel trends assumption, which underpins difference-in-difference designs, is met. Two common forms of bias that can undermine the parallel trends assumption are: (i) contemporaneous factors that are correlated with the project and that affect outcomes and (ii) systematic differences between project and control units that affect outcomes [57].
In our analysis, the parallel trends assumption is likely to hold within subgroups in regards to bias associated with contemporaneous factors that are correlated with the project and that affect outcomes. Given the proximity of the control and project villages in four districts, it is unlikely that MPA and the control villages (and thus the subgroups within them) were exposed differentially to major factors affecting poverty in North Sulawesi, such as the decimation of seaweed farming due to disease during the implementation period [36] or changes to political jurisdiction, with the division of the Minahasa regency in 2003. However, the close proximity of the MPA and control villages could have resulted in ‘spillover’ from the integrated MPAs, such that the control villages were influenced by the integrated MPAs and thus did not provide valid estimations of the counterfactual. Given the small size of the MPAs, spillover is unlikely in relation to the influence of the integrated MPAs on marine resources, but is plausible in regards to the associated development activities, such as flood prevention and livelihood training. Such spillover could move the treatment effect towards zero and help to explain why we did not find much evidence of impacts of the integrated MPAs. This highlights the trade-off associated with locating control units between having sufficient separation to minimize spillover while maximizing similarity with project units [11].
Our assessment of differences in baseline values of socioeconomic indicators provided some evidence that there were no systematic differences between project and control groups within subgroups at baseline. Perceived well-being was the only socioeconomic indicator for which we found differences between control and project groups within subgroups. The percentage of respondents describing their well-being as more positive than in the past was almost double in project villages than control villages across all subgroups. This is likely because this outcome was contaminated by the expectation of the integrated MPA's future placement. If differences in well-being were reflective of differences in observable and unobservable factors that could affect our socioeconomic indicators, the parallel trends assumption may not hold for our analysis. To improve such assessments of the parallel trends assumption in future studies, two or more pre-implementation samples could be used to compare trends in outcomes, rather than outcomes at a single point in time.
We sought to overcome bias associated with systematic differences between subgroups within project and control sites at both the village and individual scale. We selected our control villages based on key factors that influence poverty in North Sulawesi, the most important being fisheries dependence and distance to markets and roads. At the individual scale, our regression design controlled for age, gender and religion, characteristics thought to influence our socioeconomic indicators and participation in the integrated MPAs. Other individual-scale factors that could have influenced our socioeconomic indicators and participation in the integrated MPAs and could have had a time-varying impact include baseline wealth, education and fisheries dependence. In the case of fisheries dependence, given it was correlated with religion, controlling for baseline fisheries dependence would have helped isolate the impact of religion. However, we were able to explicitly control for baseline values of wealth and fisheries dependence only in the models where these factors were the response variables. The reasons were that panel data were not collected at the individual or household scale, and we could not condition on post-baseline estimates of these factors because they were influenced by the integrated MPAs.
Regarding the robustness of our results, it is important to consider the precision of our estimates. Precision, which is indicated by the width of the Bayesian highest posterior density intervals, is affected by sample size. Although our sample contained over 2000 respondents, we had a low sample size at the village scale, and our analysis involved segmenting our data according to subgroups. Further, given our post hoc analysis of randomly sampled data, our statistical design is unbalanced (i.e. different sample sizes for each combination of subgroup, time and project category). Given the variances of our outcomes (i.e. change in socioeconomic indicators over time) are large, we expect that our ability to detect an impact of the integrated MPAs was limited by our small sample size and thus the low precision of our estimates; indeed the width of the Bayesian highest posterior density intervals for many of our estimates was fairly large. To increase precision in future impact evaluations of heterogeneous impacts, subgroup categories should be selected a priori to allow stratified sampling to ensure adequate sample sizes.
Estimating heterogeneous impacts when the treatment is not randomized within subgroups, as in our case, involves a risk of designating spurious correlations as evidence of heterogeneous impacts, particularly when a large number of subgroups are considered without pre-specification [58,59]. Ferraro & Hanauer [58] provide a number of recommendations for evaluations of heterogeneous impacts to help mitigate the likelihood of mislabelling spurious correlations, including: (i) selecting a small (less than 5) number of subgroups based on theory and policy relevance; (ii) proceeding with caution to assess heterogeneous impacts if an unconditional treatment effect is not found; (iii) conducting an omnibus test to test for heterogeneity across all subgroups (e.g. [60]), and (iv) maintaining a constant family-size error rate. Note that some of these recommendations (e.g. the omnibus test and a family-wise error rate) are more relevant in a Frequentist than a Bayesian statistical framework due to fundamental differences in the way data and model parameters are considered. In our study, we assessed only socioeconomic indicators for which we had previously found evidence of an unconditional treatment effect [15], and examined just three subgroups, which we chose based on relevant theoretical literature and previous research concerning the integrated MPAs.
In sum, robust estimates for heterogeneous treatment effects are far more difficult to achieve than for unconditional effects [59]. Given our dataset of multiple socioeconomic indicators sampled several points in time before and after MPA implementation, our estimates of the socioeconomic impacts of MPAs are likely to be less susceptible to bias than most evaluations of socioeconomic impacts of protected areas that rely on comparisons of outcomes inside and outside protected areas for a single time period (e.g. [61,62]), or track outcomes in protected-area sites over time (e.g. [51,63]). Nevertheless, our analysis should be interpreted with caution given the caveats discussed above. While we provide a number of suggestions of how to strengthen future evaluations of heterogeneous socioeconomic impacts of protected areas (e.g. collection of individual-scale panel data, multiple pre-implementation sampling points), many of these suggestions would be hard to implement in practice due to ethical, logistical and financial reasons. Thus, the statistical rigor possible with large-n but low-resolution datasets, which have been employed recently to assess the socioeconomic impacts of protected areas (e.g. [17,35]), is difficult to achieve when assessing socioeconomic impacts of protected areas at the scales of individuals and small communities, especially if multiple outcomes, groups and time periods are assessed. This highlights an important challenge for programme evaluators, given the importance of information provided by high-resolution studies in informing policy and management to guide the implementation of effective protected areas.
5. Conclusion
Our study is one of the first to provide empirical evidence of whether the socioeconomic impacts of protected areas vary according to social subgroups. We found little empirical evidence that the socioeconomic impacts of the integrated MPAs differed by gender, age or religion, except in relation to environmental knowledge. Our findings help elucidate some of the pathways through which the socioeconomic impacts of the integrated MPAs occurred, and may be used to improve targeting of management activities in the region. Given that the socioeconomic impacts of protected areas are likely to vary with project and context [64], only through the accumulation of studies such as ours can we understand the heterogeneous impacts of protected areas, and learn to design projects to achieve social equity, and positive socioeconomic impacts more broadly.
Building the evidence base of potential differential impacts of protected areas according to social subgroup is critical given that social inequity can lead to conflict [24] and impede poverty reduction [25], thus jeopardizing social and biological goals [26]. The current weak evidence base for the socioeconomic impacts of protected areas, particularly in relation to heterogeneous impacts according to social subgroups, demands further research into how protected areas affect people. To strengthen future evaluations of heterogeneous socioeconomic impacts of protected areas, monitoring should be designed for evaluation of heterogeneous impacts from the outset, in contrast to post hoc analyses such as ours. Such an approach could include collection of individual-scale panel data and targeted analyses, involving identifying appropriate subgroups for analysis through assessing the access mechanisms (e.g. social and institutional) that operate in that context to determine who benefits from or is harmed by protected areas.
Ethics
The study was conducted with permission from the Indonesian government. Data from 2012 were collected by The Wildlife Conservation Society (WCS), which has a Memorandum of Understanding with the Indonesian Ministry of Forestry and Conservation. Data collected in 1997, 2000 and 2002 were collected as part of Proyek Pesisir, which was jointly run by USAID and Indonesia's National Development Planning Agency (BAPPENAS). Consent to participate in the interviews was given by all respondents.
Authors' contributions
G.G.G. conceived the study and analysed the data. S.J.C., R.P. and G.G.G. performed design and/or coordination of field surveys. All authors wrote the paper and gave final approval for publication.
Competing interests
We declare we have no competing interests.
Funding
G.G.G., R.L.P. and J.E.C. acknowledge support from the Australian Research Council Centre of Excellence for Coral Reef Studies. G.G.G. also acknowledges the support of the Commonwealth Scientific and Industrial Research Organisation (CSIRO). R.L.P. acknowledges the support of the United States Agency for International Development (USAID).
Acknowledgements
We thank all the people in North Sulawesi who supported this project, particularly those who participated in the interviews. A big thank you to the students from Sam Ratulangi University and the Wildlife Conservation Society staff (particularly S. Tasidjawa and F. Setiawan) who contributed to the design, coordination and implementation of field surveys in 2012. We also thank the reviewers for their constructive comments that greatly improved the manuscript.
Footnotes
One contribution of 16 to a theme issue ‘Measuring the difference made by protected areas: methods, applications and implications for policy and practice’.
- Accepted August 10, 2015.
- © 2015 The Author(s)
References
Keywords
