Rethinking receiver operating characteristic analysis applications in ecological niche modeling

The area under the curve (AUC) of the receiver operating characteristic (ROC) has become a dominant tool in evaluating the accuracy of models predicting distributions of species. ROC has the advantage of being threshold-independent, and as such does not require decisions regarding thresholds of what constitutes a prediction of presence versus a prediction of absence. However, we show that, comparing two ROCs, using the AUC systematically undervalues models that do not provide predictions across the entire spectrum of proportional areas in the study area. Current ROC approaches in ecological niche modeling applications are also inappropriate because the two error components are weighted equally. We recommend a modification of ROC that remedies these problems, using partial-area ROC approaches to provide a firmer foundation for evaluation of predictions from ecological niche models. A worked example demonstrates that models that are evaluated favorably by traditional ROC AUCs are not necessarily the best when niche modeling considerations are incorporated into the design of the test.     

The crucial role of the accessible area in ecological niche modeling and species distribution modeling

Using known occurrences of species and correlational modeling approaches has become a common paradigm in broad-scale ecology and biogeography, yet important aspects of the methodology remain little-explored in terms of conceptual basis. Here, we explore the conceptual and empirical reasons behind choice of extent of study area in such analyses, and offer practical, but conceptually justified, reasoning for such decisions. We assert that the area that has been accessible to the species of interest over relevant time periods represents the ideal area for model development, testing, and comparison.     

Sensitivity of species-distribution models to error,bias, and model design: An application to resource selection functions for woodland caribou

Models that predict distribution are now widely used to understand the patterns and processes of plant and animal occurrence as well as to guide conservation and management of rare or threatened species. Application of these methods has led to corresponding studies evaluating the sensitivity of model performance to requisite data and other factors that may lead to imprecise or false inferences. We expand upon these works by providing a relative measure of the sensitivity of model parameters and prediction to common sources of error, bias, and variability. We used a one-at-a-time sample design and GPS location data for woodland caribou (Rangifer tarandus caribou) to assess one common species-distribution model: a resource selection function. Our measures of sensitivity included change in coefficient values, prediction success, and the area of mapped habitats following the systematic introduction of geographic error and bias in occurrence data, thematic misclassification of resource maps, and variation in model design. Results suggested that error, bias and model variation have a large impact on the direct interpretation of coefficients. Prediction success and definition of important habitats were less responsive to the perturbations we introduced to the baseline model. Model coefficients, prediction success, and area of ranked habitats were most sensitive to positional error in species locations followed by sampling bias, misclassification of resources, and variation in model design. We recommend that researchers report, and practitioners consider, levels of error and bias introduced to predictive species-distribution models. Formal sensitivity and uncertainty analyses are the most effective means for evaluating and focusing improvements on input data and considering the range of values possible from imperfect models.     

An improved approach for identifying suitable habitat of Sambar Deer (Cervus unicolor Kerr) using ecological niche analysis and environmental categorization: Case study at Phu-Khieo Wildlife Sanctuary,Thailand

To assess habitat suitability (HS) has become an increasingly important component of species/ecosystem management. HS assessment is usually based on presence/absence data related to environmental variables. An exception is Ecological Niche Factor Analysis (ENFA), which uses only presence data and which does not require absence data. Most HS modelling is based on input of all environmental parameters (EnvPs) without environmental categorization, and does not take into account species interaction and human intervention for an assessment of HS. In this study, the EnvPs are arranged into four features: geographical features, consumable features, human-factor features, and species–human interaction features. These features affect species with respect to movement, behavior and activity. The research presented here has used an already existing dataset of wildlife species and human activities/visitations, which was compiled during 2004–2006 in Phu-Khieo Wildlife Sanctuary (PKWS). Data from 2004 to 2005 were used to produce HS maps, while the data of 2006 were used for evaluating these maps. Sambar Deer (SD) was chosen to predict its own HS. Six HS maps of SD were analyzed using ENFA in the following manner: (1) inputting all EnvPs together, (2) inputting each feature, separately and (3) integrating the four resulting HS maps by model averaging. It was found that model averaging was capable of predicting the HS of SD more reliably than the model with all EnvPs put in together. Multiple linear regressions were computed between the HS map with all EnvPs and the HS maps with each feature. The results show that the HS map with only geographical features has the highest coefficient value (0.516) while the coefficient values of other HS maps with the above features are 0.296, 0.53 and −0.006, respectively. This indicates that the geographical features have an influence on the other features and that the predicting power is lower when all EnvPs are computed in the ENFA model. Therefore, in order to generate HS, each feature should at first be put into the model separately. Following that, the average of all features will be combined.