The difficulties of systematic reviews

The need for robust evidence to support conservation actions has driven the adoption of systematic approaches to research synthesis in ecology. However, applying systematic review to complex or open questions remains challenging, and this task is becoming more difficult as the quantity of scientific literature increases. We drew on the science of linguistics for guidance as to why the process of identifying and sorting information during systematic review remains so labor intensive, and to provide potential solutions. Several linguistic properties of peer‐reviewed corpora—including nonrandom selection of review topics, small‐world properties of semantic networks, and spatiotemporal variation in word meaning—greatly increase the effort needed to complete the systematic review process. Conversely, the resolution of these semantic complexities is a common motivation for narrative reviews, but this process is rarely enacted with the rigor applied during linguistic analysis. Therefore, linguistics provides a unifying framework for understanding some key challenges of systematic review and highlights 2 useful directions for future research. First, in cases where semantic complexity generates barriers to synthesis, ecologists should consider drawing on existing methods—such as natural language processing or the construction of research thesauri and ontologies—that provide tools for mapping and resolving that complexity. These tools could help individual researchers classify research material in a more robust manner and provide valuable guidance for future researchers on that topic. Second, a linguistic perspective highlights that scientific writing is a rich resource worthy of detailed study, an observation that can sometimes be lost during the search for data during systematic review or meta‐analysis. For example, mapping semantic networks can reveal redundancy and complementarity among scientific concepts, leading to new insights and research questions. Consequently, wider adoption of linguistic approaches may facilitate improved rigor and richness in research synthesis.     

Contribution of Systematic Reviews to Management Decisions

Systematic reviews comprehensively summarize evidence about the effectiveness of conservation interventions. We investigated the contribution to management decisions made by this growing body of literature. We identified 43 systematic reviews of conservation evidence, 23 of which drew some concrete conclusions relevant to management. Most reviews addressed conservation interventions relevant to policy decisions; only 35% considered practical on‐the‐ground management interventions. The majority of reviews covered only a small fraction of the geographic and taxonomic breadth they aimed to address (median = 13% of relevant countries and 16% of relevant taxa). The likelihood that reviews contained at least some implications for management tended to increase as geographic coverage increased and to decline as taxonomic breadth increased. These results suggest the breadth of a systematic review requires careful consideration. Reviews identified a mean of 312 relevant primary studies but excluded 88% of these because of deficiencies in design or a failure to meet other inclusion criteria. Reviews summarized on average 284 data sets and 112 years of research activity, yet the likelihood that their results had at least some implications for management did not increase as the amount of primary research summarized increased. In some cases, conclusions were elusive despite the inclusion of hundreds of data sets and years of cumulative research activity. Systematic reviews are an important part of the conservation decision making tool kit, although we believe the benefits of systematic reviews could be significantly enhanced by increasing the number of reviews focused on questions of direct relevance to on‐the‐ground managers; defining a more focused geographic and taxonomic breadth that better reflects available data; including a broader range of evidence types; and appraising the cost‐effectiveness of interventions. Contribuciones de las Revisiones Sistemáticas a las Decisiones de Manejo     

Predicting the time needed for environmental systematic reviews and systematic maps

Systematic reviews (SRs) and systematic mapping aim to maximize transparency and comprehensiveness while minimizing subjectivity and bias. These are time-consuming and complex tasks, so SRs are considered resource intensive, but published estimates of systematic-review resource requirements are largely anecdotal. We analyzed all Collaboration for Environmental Evidence (CEE) SRs (n = 66) and maps (n = 20) published from 2012 to 2017 to estimate the average number of articles retained at each review stage. We also surveyed 33 experienced systematic reviewers to collate information on the rate at which those stages could be completed. In combination, these data showed that the average CEE SR takes an estimated 164 d (full-time equivalent) (SD 23), and the average CEE systematic map (SM) (excluding critical appraisal) takes 211 d (SD 53). While screening titles and abstracts is widely considered time-consuming, metadata extraction and critical appraisal took as long or longer to complete, especially for SMs. Given information about the planned methods and evidence base, we created a software tool that predicts time requirements of a SR or map with evidence-based defaults as a starting point. Our results shed light on the most time-consuming stages of the SR and mapping processes, will inform review planning, and can direct innovation to streamline processes. Future predictions of effort required to complete SRs and maps could be improved if authors provide more details on methods and results.     

Using machine learning to disentangle homonyms in large text corpora

Systematic reviews are an increasingly popular decision‐making tool that provides an unbiased summary of evidence to support conservation action. These reviews bridge the gap between researchers and managers by presenting a comprehensive overview of all studies relating to a particular topic and identify specifically where and under which conditions an effect is present. However, several technical challenges can severely hinder the feasibility and applicability of systematic reviews, for example, homonyms (terms that share spelling but differ in meaning). Homonyms add noise to search results and cannot be easily identified or removed. We developed a semiautomated approach that can aid in the classification of homonyms among narratives. We used a combination of automated content analysis and artificial neural networks to quickly and accurately sift through large corpora of academic texts and classify them to distinct topics. As an example, we explored the use of the word reintroduction in academic texts. Reintroduction is used within the conservation context to indicate the release of organisms to their former native habitat; however, a Web of Science search for this word returned thousands of publications in which the term has other meanings and contexts. Using our method, we automatically classified a sample of 3000 of these publications with over 99% accuracy, relative to a manual classification. Our approach can be used easily with other homonyms and can greatly facilitate systematic reviews or similar work in which homonyms hinder the harnessing of large text corpora. Beyond homonyms we see great promise in combining automated content analysis and machine‐learning methods to handle and screen big data for relevant information in conservation science.     

Response of Neotropical Bat Assemblages to Human Land Use

Neotropical bats are sensitive to human‐induced habitat changes, and some authors believe bats can be used as bioindicators. In the literature, however, the results are disparate. Some results show bat diversity deceases as disturbance increases, whereas others indicate no effect. Determining the general response patterns of bats when they encounter different degrees of human‐induced disturbance across the Neotropics would help to determine their usefulness as bioindicators. In a series of meta‐analyses, we compared the occurrence frequency of bat species between well‐preserved forests and human‐use areas. We obtained data through an extensive review of published peer‐reviewed articles, theses, and reports. The overall effect size indicated that human‐use areas harbored more bat species than well‐preserved forests. Different response patterns emerged when meta‐analyses were conducted separately by family, feeding habit, vegetation stratum, and conservation status. Our results suggest that bat assemblages display strong responses to forest loss and land‐use change and that the direction and magnitude of these responses depends on the bat group under study and the type of disturbance. Our results are consistent with the idea that bats are useful for assessing the effects of habitat changes in the Neotropics. However, with our meta‐analyses we could not detect fine differences in bat feeding habits, especially within Phyllostomidae, or elucidate the effect of landscape configuration. Respuesta de Ensambles de Murciélagos Neotropicales al Uso de Suelo por Humanos     

Effectiveness of Predator Removal for Enhancing Bird Populations

Abstract: Predation pressure on vulnerable bird species has made predator control an important issue for international nature conservation. Predator removal by culling or translocation is controversial, expensive, and time‐consuming, and results are often temporary. Thus, it is important to assess its effectiveness from all available evidence. We used explicit systematic review methodology to determine the impact of predator removal on four measurable responses in birds: breeding performance (hatching success and fledging success) and population size (breeding and postbreeding). We used meta‐analysis to summarize results from 83 predator removal studies from six continents. We also investigated whether characteristics of the prey, predator species, location, and study methodology explained heterogeneity in effect sizes. Removing predators increased hatching success, fledging success, and breeding populations. Removing all predator species achieved a significantly larger increase in breeding population than removing only a subset. Postbreeding population size was not improved on islands, or overall, but did increase on mainlands. Heterogeneity in effect sizes for the four population parameters was not explained by whether predators were native or introduced; prey were declining, migratory, or game species; or by the study methodology. Effect sizes for fledging success were smaller for ground‐nesting birds than those that nest elsewhere, but the difference was not significant. We conclude that current evidence indicates that predator removal is an effective strategy for the conservation of vulnerable bird populations. Nevertheless, the ethical and practical problems associated with predator removal may lead managers to favor alternative, nonlethal solutions. Research is needed to provide and synthesize data to determine whether these are effective management practices for future policies on bird conservation.     

Understanding Publication Bias in Reintroduction Biology by Assessing Translocations of New Zealand's Herpetofauna

The intentional translocation of animals is an important tool for species conservation and ecosystem restoration, but reported success rates are low, particularly for threatened and endangered species. Publication bias further distorts success rates because the results of successful translocations may be more likely to be published than failed translocations. We conducted the first comprehensive review of all published and unpublished translocations of herpetofauna in New Zealand to assess publication bias. Of 74 translocations of 29 species in 25 years, 35 have been reported in the published literature, and the outcomes of 12 have been published. Using a traditional definition of success, publication bias resulted in a gross overestimate of translocation success rates (41.7% and 8.1% for published and all translocations, respectively), but bias against failed translocations was minimal (8.3% and 6.8%, respectively). Publication bias against translocations with uncertain outcomes, the vast majority of projects, was also strong (50.0% and 85.1% for published and all translocations, respectively). Recent translocations were less likely to be published than older translocations. The reasons behind translocations were related to publication. A greater percentage of translocations for conservation and research were published (63.3% and 40.0%, respectively) than translocations for mitigation during land development (10.0%). Translocations conducted in collaboration with a university were more frequently published (82.7% and 24.4%, respectively). To account for some of this publication bias, we reassessed the outcome of each translocation using a standardized definition of success, which takes into consideration the species’ life history and the time since release. Our standardized definition of translocation success provided a more accurate summary of success rates and allows for a more rigorous evaluation of the causes of translocation success and failure in large‐scale reviews. Entendiendo el Sesgo de Publicaciones en la Biología de la Reintroducción Mediante el Estudio de Traslocaciones de la Herpetofauna de Nueva Zelanda     

The importance of defining focal assemblages when evaluating amphibian and reptile responses to land use

Habitat loss and degradation are primary threats to amphibians and reptiles, but the relative effects of common land uses on assemblages and the mechanisms that underlie faunal responses are poorly studied. We reviewed the effects of four prevalent types of habitat alteration (urbanization, agriculture, livestock grazing, and silviculture) on amphibian and reptile species richness and abundance by summarizing reported responses in the literature and by estimating effect sizes across studies for species richness in each land‐use type. We then used a multinomial model to classify species as natural habitat specialists, generalists, and disturbed habitat specialists and examined variation in effect sizes for each land‐use type according to habitat specialization categories. There were mixed conclusions from individual studies, some reporting negative, neutral, or positive effects of land use on species richness and total abundance. A large proportion of studies reported species‐specific effects of individual species abundance. However, in our analysis of effect sizes, we found a general trend of negative effects of land use on species richness. We also demonstrate that habitat associations of common species and species turnover can explain variation in the effect of land use on herpetofauna. Our review highlights the pervasive negative effects of common land uses on amphibians and reptiles, the importance of identifying groups vulnerable to land‐use change (e.g., forest‐associated species) in conservation studies, and the potential influence of disturbance‐associated species on whole assemblage analyses.     

Improving credibility and transparency of conservation impact evaluations through the partial identification approach

The fundamental challenge of evaluating the impact of conservation interventions is that researchers must estimate the difference between the outcome after an intervention occurred and what the outcome would have been without it (counterfactual). Because the counterfactual is unobservable, researchers must make an untestable assumption that some units (e.g., organisms or sites) that were not exposed to the intervention can be used as a surrogate for the counterfactual (control). The conventional approach is to make a point estimate (i.e., single number along with a confidence interval) of impact, using, for example, regression. Point estimates provide powerful conclusions, but in nonexperimental contexts they depend on strong assumptions about the counterfactual that often lack transparency and credibility. An alternative approach, called partial identification (PI), is to first estimate what the counterfactual bounds would be if the weakest possible assumptions were made. Then, one narrows the bounds by using stronger but credible assumptions based on an understanding of why units were selected for the intervention and how they might respond to it. We applied this approach and compared it with conventional approaches by estimating the impact of a conservation program that removed invasive trees in part of the Cape Floristic Region. Even when we used our largest PI impact estimate, the program's control costs were 1.4 times higher than previously estimated. PI holds promise for applications in conservation science because it encourages researchers to better understand and account for treatment selection biases; can offer insights into the plausibility of conventional point‐estimate approaches; could reduce the problem of advocacy in science; might be easier for stakeholders to agree on a bounded estimate than a point estimate where impacts are contentious; and requires only basic arithmetic skills.     

Joining forces toward proactive elephant and rhinoceros conservation

Proactive approaches that anticipate the long-term effects of current and future conservation threats could increase the effectiveness and efficiency of biodiversity conservation. However, such approaches can be obstructed by a lack of knowledge of habitat requirements for wildlife. To aggregate and assess the suitability of current information available on habitat requirements needed for proactive conservation, we conducted a systematic review of the literature on elephant and rhinoceros habitat requirements and synthesized data by combining a vote counting assessment with bibliometric and term maps. We contextualized these numeric and terminological results with a narrative review. We mapped current methods, results, terminology, and collaborations of 693 studies. Quantitative evidence for factors that influence the suitability of an area for elephants and rhinoceros was biased toward African savanna elephants and ecological variables. Less than one third of holistic approaches considered equal amounts of ecological and anthropogenic variables in their assessments. There was a general lack of quantitative evidence for direct proxies of anthropogenic variables that were expected to play an important role based on qualitative evidence and policy documents. However, there was evidence for a segregation in conceptual frameworks among countries and species and between science versus policy literature. There was also evidence of unused potential for collaborations among southern hemisphere researchers. Our results indicated that the success of proactive conservation interventions can be increased if ecological and anthropogenic dimensions are integrated into holistic habitat assessments and holistic carrying capacities and quantitative evidence for anthropogenic variables is improved. To avoid wasting limited resources, it is necessary to form inclusive collaborations within and across networks of researchers studying different species across regional and continental borders and in the science–policy realm.     

Citation Rate and Perceived Subject Bias in the Amphibian‐Decline Literature

Abstract: As a result of global declines in amphibian populations, interest in the conservation of amphibians has grown. This growth has been fueled partially by the recent discovery of other potential causes of declines, including chytridiomycosis (the amphibian chytrid, an infectious disease) and climate change. It has been proposed that researchers have shifted their focus to these novel stressors and that other threats to amphibians, such as habitat loss, are not being studied in proportion to their potential effects. We tested the validity of this proposal by reviewing the literature on amphibian declines, categorizing the primary topic of articles within this literature (e.g., habitat loss or UV‐B radiation) and comparing citation rates among articles on these topics and impact factors of journals in which the articles were published. From 1990 to 2009, the proportion of papers on habitat loss remained fairly constant, and although the number of papers on chytridiomycosis increased after the disease was described in 1998, the number of published papers on amphibian declines also increased. Nevertheless, papers on chytridiomycosis were more highly cited than papers not on chytridiomycosis and were published in journals with higher impact factors on average, which may indicate this research topic is more popular in the literature. Our results were not consistent with a shift in the research agenda on amphibians. We believe the perception of such a shift has been supported by the higher citation rates of papers on chytridiomycosis.     

Disentangling the causes of protected‐species bycatch in gillnet fisheries

Gillnet fisheries are widely thought to pose a conservation threat to many populations of marine mammals, seabirds, and turtles. Gillnet fisheries also support a significant proportion of small‐scale fishing communities worldwide. Despite a large number of studies on protected‐species bycatch in recent decades, relatively few have examined the underlying causes of bycatch and fewer still have considered the issue from a multitaxon perspective. We used 3 bibliographic databases and one search engine to identify studies by year of publication and taxon. The majority of studies on the mechanisms of gillnet bycatch are not accessible through the mainstream published literature. Many are reported in technical papers, government reports, and university theses. We reviewed over 600 published and unpublished studies of bycatch in which causal or correlative factors were considered and identified therein 28 environmental, operational, technical, and behavioral factors that may be associated with high or low bycatch rates of the taxa. Of the factors considered, 11 were associated with potential bycatch reduction in 2 out of the 3 taxa, and 3 factors (water depth, mesh size, and net height) were associated with trends in bycatch rate for all 3 taxa. These findings provide a basis to guide further experimental work to test hypotheses about which factors most influence bycatch rates and to explore ways of managing fishing activities and improving gear design to minimize the incidental capture of species of conservation concern while ensuring the viability of the fisheries concerned.     

Estimating the extent and structure of trade in horticultural orchids via social media

The wildlife trade is a lucrative industry involving thousands of animal and plant species. The increasing use of the internet for both legal and illegal wildlife trade is well documented, but there is evidence that trade may be emerging on new online technologies such as social media. Using the orchid trade as a case study, we conducted the first systematic survey of wildlife trade on an international social‐media website. We focused on themed forums (groups), where people with similar interests can interact by uploading images or text (posts) that are visible to other group members. We used social‐network analysis to examine the ties between 150 of these orchid‐themed groups to determine the structure of the network. We found 4 communities of closely linked groups based around shared language. Most trade occurred in a community that consisted of English‐speaking and Southeast Asian groups. In addition to the network analysis, we randomly sampled 30 groups from the whole network to assess the prevalence of trade in cultivated and wild plants. Of 55,805 posts recorded over 12 weeks, 8.9% contained plants for sale, and 22–46% of these posts pertained to wild‐collected orchids. Although total numbers of posts about trade were relatively small, the large proportion of posts advertising wild orchids for sale supports calls for better monitoring of social media for trade in wild‐collected plants.     

Defending the scientific integrity of conservation‐policy processes

Government agencies faced with politically controversial decisions often discount or ignore scientific information, whether from agency staff or nongovernmental scientists. Recent developments in scientific integrity (the ability to perform, use, communicate, and publish science free from censorship or political interference) in Canada, Australia, and the United States demonstrate a similar trajectory. A perceived increase in scientific‐integrity abuses provokes concerted pressure by the scientific community, leading to efforts to improve scientific‐integrity protections under a new administration. However, protections are often inconsistently applied and are at risk of reversal under administrations publicly hostile to evidence‐based policy. We compared recent challenges to scientific integrity to determine what aspects of scientific input into conservation policy are most at risk of political distortion and what can be done to strengthen safeguards against such abuses. To ensure the integrity of outbound communications from government scientists to the public, we suggest governments strengthen scientific integrity policies, include scientists’ right to speak freely in collective‐bargaining agreements, guarantee public access to scientific information, and strengthen agency culture supporting scientific integrity. To ensure the transparency and integrity with which information from nongovernmental scientists (e.g., submitted comments or formal policy reviews) informs the policy process, we suggest governments broaden the scope of independent reviews, ensure greater diversity of expert input and transparency regarding conflicts of interest, require a substantive response to input from agencies, and engage proactively with scientific societies. For their part, scientists and scientific societies have a responsibility to engage with the public to affirm that science is a crucial resource for developing evidence‐based policy and regulations in the public interest.     

The effect of scientific evidence on conservation practitioners’ management decisions

A major justification of environmental management research is that it helps practitioners, yet previous studies show it is rarely used to inform their decisions. We tested whether conservation practitioners focusing on bird management were willing to use a synopsis of relevant scientific literature to inform their management decisions. This allowed us to examine whether the limited use of scientific information in management is due to a lack of access to the scientific literature or whether it is because practitioners are either not interested or unable to incorporate the research into their decisions. In on‐line surveys, we asked 92 conservation managers, predominantly from Australia, New Zealand, and the United Kingdom, to provide opinions on 28 management techniques that could be applied to reduce predation on birds. We asked their opinions before and after giving them a summary of the literature about the interventions’ effectiveness. We scored the overall effectiveness and certainty of evidence for each intervention through an expert elicitation process—the Delphi method. We used the effectiveness scores to assess the practitioners’ level of understanding and awareness of the literature. On average, each survey participant changed their likelihood of using 45.7% of the interventions after reading the synopsis of the evidence. They were more likely to implement effective interventions and avoid ineffective actions, suggesting that their intended future management strategies may be more successful than current practice. More experienced practitioners were less likely to change their management practices than those with less experience, even though they were not more aware of the existing scientific information than less experienced practitioners. The practitioners’ willingness to change their management choices when provided with summarized scientific evidence suggests that improved accessibility to scientific information would benefit conservation management outcomes. El Efecto de la Evidencia Científica sobre las Decisiones de Manejo de Quienes Practican la Conservación     

Improving effectiveness of systematic conservation planning with density data

Systematic conservation planning aims to design networks of protected areas that meet conservation goals across large landscapes. The optimal design of these conservation networks is most frequently based on the modeled habitat suitability or probability of occurrence of species, despite evidence that model predictions may not be highly correlated with species density. We hypothesized that conservation networks designed using species density distributions more efficiently conserve populations of all species considered than networks designed using probability of occurrence models. To test this hypothesis, we used the Zonation conservation prioritization algorithm to evaluate conservation network designs based on probability of occurrence versus density models for 26 land bird species in the U.S. Pacific Northwest. We assessed the efficacy of each conservation network based on predicted species densities and predicted species diversity. High‐density model Zonation rankings protected more individuals per species when networks protected the highest priority 10‐40% of the landscape. Compared with density‐based models, the occurrence‐based models protected more individuals in the lowest 50% priority areas of the landscape. The 2 approaches conserved species diversity in similar ways: predicted diversity was higher in higher priority locations in both conservation networks. We conclude that both density and probability of occurrence models can be useful for setting conservation priorities but that density‐based models are best suited for identifying the highest priority areas. Developing methods to aggregate species count data from unrelated monitoring efforts and making these data widely available through ecoinformatics portals such as the Avian Knowledge Network will enable species count data to be more widely incorporated into systematic conservation planning efforts.     

Testing Decision Rules for Categorizing Species’ Extinction Risk to Help Develop Quantitative Listing Criteria for the U.S. Endangered Species Act

Lack of guidance for interpreting the definitions of endangered and threatened in the U.S. Endangered Species Act (ESA) has resulted in case‐by‐case decision making leaving the process vulnerable to being considered arbitrary or capricious. Adopting quantitative decision rules would remedy this but requires the agency to specify the relative urgency concerning extinction events over time, cutoff risk values corresponding to different levels of protection, and the importance given to different types of listing errors. We tested the performance of 3 sets of decision rules that use alternative functions for weighting the relative urgency of future extinction events: a threshold rule set, which uses a decision rule of x% probability of extinction over y years; a concave rule set, where the relative importance of future extinction events declines exponentially over time; and a shoulder rule set that uses a sigmoid shape function, where relative importance declines slowly at first and then more rapidly. We obtained decision cutoffs by interviewing several biologists and then emulated the listing process with simulations that covered a range of extinction risks typical of ESA listing decisions. We evaluated performance of the decision rules under different data quantities and qualities on the basis of the relative importance of misclassification errors. Although there was little difference between the performance of alternative decision rules for correct listings, the distribution of misclassifications differed depending on the function used. Misclassifications for the threshold and concave listing criteria resulted in more overprotection errors, particularly as uncertainty increased, whereas errors for the shoulder listing criteria were more symmetrical. We developed and tested the framework for quantitative decision rules for listing species under the U.S. ESA. If policy values can be agreed on, use of this framework would improve the implementation of the ESA by increasing transparency and consistency. Evaluando Reglas de Decisión para Categorizar el Riesgo de Extinción de Especies con el Fin de Desarrollar de Criterios Cuantitativos de Alistamiento en el Acta de Especies en Peligro de los EE. UU.     

Bridging research and practice in conservation

Calls for biodiversity conservation practice to be more evidence based are growing, and we agree evidence use in conservation practice needs improvement. However, evidence-based conservation will not be realized without improved access to evidence. In medicine, unlike in conservation, a well-established and well-funded layer of intermediary individuals and organizations engage with medical practitioners, synthesize primary research relevant to decision making, and make evidence easily accessible. These intermediaries prepare targeted evidence summaries and distribute them to practitioners faced with time-sensitive and value-laden decisions. To be effective, these intermediaries, who we refer to as evidence bridges, should identify research topics based on the priorities of practitioners; synthesize evidence; prepare and distribute easy-to-find and easy-to-use evidence summaries; and develop and maintain networks of connections with researchers and practitioners. Based on a review of the literature regarding evidence intermediaries in conservation and environmental management, as well as an anonymous questionnaire searching for such organizations, we found few intermediaries that met all these criteria. Few evidence bridges that do exist are unable to reach most conservation practitioners, which include resource managers in government and industry, conservation organizations, and farmers and other private landowners. We argue that the lack of evidence bridges from research to practitioners contributes to evidence complacency and limits the use of evidence in conservation action. Nevertheless, several existing organizations help reduce the gap between evidence and practice and could serve as a foundation for building additional components of evidence bridges in conservation. Although evidence bridges need expertise in research and evidence synthesis, they also require expertise in identifying and communicating with the community of practitioners most in need of clear and concise syntheses of evidence. Article Impact Statement: Evidence-based conservation will not be realized without improved access to evidence. We call for intermediary evidence bridges.     

A generalized approach for producing,quantifying, and validating citizen science data from wildlife images

Citizen science has the potential to expand the scope and scale of research in ecology and conservation, but many professional researchers remain skeptical of data produced by nonexperts. We devised an approach for producing accurate, reliable data from untrained, nonexpert volunteers. On the citizen science website www.snapshotserengeti.org , more than 28,000 volunteers classified 1.51 million images taken in a large‐scale camera‐trap survey in Serengeti National Park, Tanzania. Each image was circulated to, on average, 27 volunteers, and their classifications were aggregated using a simple plurality algorithm. We validated the aggregated answers against a data set of 3829 images verified by experts and calculated 3 certainty metrics—level of agreement among classifications (evenness), fraction of classifications supporting the aggregated answer (fraction support), and fraction of classifiers who reported “nothing here” for an image that was ultimately classified as containing an animal (fraction blank)—to measure confidence that an aggregated answer was correct. Overall, aggregated volunteer answers agreed with the expert‐verified data on 98% of images, but accuracy differed by species commonness such that rare species had higher rates of false positives and false negatives. Easily calculated analysis of variance and post‐hoc Tukey tests indicated that the certainty metrics were significant indicators of whether each image was correctly classified or classifiable. Thus, the certainty metrics can be used to identify images for expert review. Bootstrapping analyses further indicated that 90% of images were correctly classified with just 5 volunteers per image. Species classifications based on the plurality vote of multiple citizen scientists can provide a reliable foundation for large‐scale monitoring of African wildlife.