The Consistency of Extinction Risk Classification Protocols

Abstract:  Systematic protocols that use decision rules or scores are seen to improve consistency and transparency in classifying the conservation status of species. When applying these protocols, assessors are typically required to decide on estimates for attributes that are inherently uncertain. Input data and resulting classifications are usually treated as though they are exact and hence without operator error. We investigated the impact of data interpretation on the consistency of protocols of extinction risk classifications and diagnosed causes of discrepancies when they occurred. We tested three widely used systematic classification protocols employed by the World Conservation Union, NatureServe, and the Florida Fish and Wildlife Conservation Commission. We provided 18 assessors with identical information for 13 different species to infer estimates for each of the required parameters for the three protocols. The threat classification of several of the species varied from low risk to high risk, depending on who did the assessment. This occurred across the three protocols investigated. Assessors tended to agree on their placement of species in the highest (50–70%) and lowest risk categories (20–40%), but there was poor agreement on which species should be placed in the intermediate categories. Furthermore, the correspondence between the three classification methods was unpredictable, with large variation among assessors. These results highlight the importance of peer review and consensus among multiple assessors in species classifications and the need to be cautious with assessments carried out by a single assessor. Greater consistency among assessors requires wide use of training manuals and formal methods for estimating parameters that allow uncertainties to be represented, carried through chains of calculations, and reported transparently.