Testing ecological models: the meaning of validation

The ecological literature reveals considerable confusion about the meaning of validation in the context of simulation models. The confusion arises as much from semantic and philosophical considerations as from the selection of validation procedures. Validation is not a procedure for testing scientific theory or for certifying the ‘truth’ of current scientific understanding, nor is it a required activity of every modelling project. Validation means that a model is acceptable for its intended use because it meets specified performance requirements.Before validation is undertaken, (1) the purpose of the model, (2) the performance criteria, and (3) the model context must be specified. The validation process can be decomposed into several components: (1) operation, (2) theory, and (3) data. Important concepts needed to understand the model evaluation process are verification, calibration, validation, credibility, and qualification. These terms are defined in a limited technical sense applicable to the evaluation of simulation models, and not as general philosophical concepts. Different tests and standards are applied to the operational, theoretical, and data components. The operational and data components can be validated; the theoretical component cannot.The most common problem with ecological and environmental models is failure to state what the validation criteria are. Criteria must be explicitly stated because there are no universal standards for selecting what test procedures or criteria to use for validation. A test based on comparison of simulated versus observed data is generally included whenever possible. Because the objective and subjective components of validation are not mutually exclusive, disagreements over the meaning of validation can only be resolved by establishing a convention.     

Modelling forest management within a global vegetation model—Part 2: Model validation from a tree to a continental scale

The construction of a new forest management module (FMM) within the ORCHIDEE global vegetation model (GVM) allows a realistic simulation of biomass changes during the life cycle of a forest, which makes many biomass datasets suitable as validation data for the coupled ORCHIDEE-FM GVM. This study uses three datasets to validate ORCHIDEE-FM at different temporal and spatial scales: permanent monitoring plots, yield tables, and the French national inventory data. The last dataset has sufficient geospatial coverage to allow a novel type of validation: inventory plots can be used to produce continuous maps that can be compared to continuous simulations for regional trends in standing volumes and volume increments. ORCHIDEE-FM performs better than simple statistical models for stand-level variables, which include tree density, basal area, standing volume, average circumference and height, when management intensity and initial conditions are known: model efficiency is improved by an average of 0.11, and its average bias does not exceed 25%. The performance of the model is less satisfying for tree-level variables, including extreme circumferences, tree circumference distribution and competition indices, or when management and initial conditions are unknown. At the regional level, when climate forcing is accurate for precipitation, ORCHIDEE-FM is able to reproduce most productivity patterns in France, such as the local lows of needleleaves in the Parisian basin and of broadleaves in south-central France. The simulation of water stress effects on biomass in the Mediterranean region, however, remains problematic, as does the simulation of the wood increment for coniferous trees. These pitfalls pertain to the general ORCHIDEE model rather than to the FMM. Overall, with an average bias seldom exceeding 40%, the performance of ORCHIDEE-FM is deemed reliable to use it as a new modelling tool in the study of the effects of interactions between forest management and climate on biomass stocks of forests across a range of scales from plot to country.     

Simulating soil organic matter with CQESTR (v. 2.0): Model description and validation against long-term experiments across North America

Soil carbon (C) models are important tools for examining complex interactions between climate, crop and soil management practices, and to evaluate the long-term effects of management practices on C-storage potential in soils. CQESTR is a process-based carbon balance model that relates crop residue additions and crop and soil management to soil organic matter (SOM) accretion or loss. This model was developed for national use in U.S and calibrated initially in the Pacific Northwest. Our objectives were: (i) to revise the model, making it more applicable for wider geographic areas including potential international application, by modifying the thermal effect and incorporating soil texture and drainage effects, and (ii) to recalibrate and validate it for an extended range of soil properties and climate conditions. The current version of CQESTR (v. 2.0) is presented with the algorithms necessary to simulate SOM at field scale. Input data for SOM calculation include crop rotation, aboveground and belowground biomass additions, tillage, weather, and the nitrogen content of crop residues and any organic amendments. The model was validated with long-term data from across North America. Regression analysis of 306 pairs of predicted and measured SOM data under diverse climate, soil texture and drainage classes, and agronomic practices at 13 agricultural sites having a range of SOM (7.3–57.9 g SOM kg⁻¹), resulted in a linear relationship with an r² of 0.95 (P < 0.0001) and a 95% confidence interval of 4.3 g SOM kg⁻¹. Using the same data the version 1.0 of CQESTR had an r² of 0.71 with a 95% confidence interval of 5.5 g SOM kg⁻¹. The model can be used as a tool to predict and evaluate SOM changes from various management practices and offers the potential to estimate C accretion required for C credits.     

Relationships between nutrients and macroalgal biomass in a brazilian coastal lagoon: the impact of a lock construction

The Piratininga Lagoon is a coastal, choked, and brackish ecosystem in SE Brazil, where uncontrolled discharge of domestic sewage led to eutrophication, increasing massive proliferation of benthic macroalgae, and decrease of the lagoon mean depth. In 1992, a dam was constructed by the local Municipality at its tidal channel aiming at stabilising its water level. Main physico-chemical parameters, together with macroalgae biomass, nutrient concentrations in the water column, particulate organic carbon, and chlorophyll a were recorded monthly at 4 sampling stations within the Piratininga lagoon from April 1994 to April 1995. The data, compared with “before-lock” existing studies, show that nutrients and chlorophyll a concentrations significantly increased after the lock construction. Based on the functioning of the ecosystem, we propose to harvest the algal mats before their decomposition period in order to partly remove the nutrient stocks from the lagoon and the future ecosystem modelling to predict the impact of natural and anthropogenic eutrophication.     

Relationships between nutrients and macroalgal biomass in a brazilian coastal lagoon: the impact of a lock construction

The Piratininga Lagoon is a coastal, choked, and brackish ecosystem in SE Brazil, where uncontrolled discharge of domestic sewage led to eutrophication, increasing massive proliferation of benthic macroalgae, and decrease of the lagoon mean depth. In 1992, a dam was constructed by the local Municipality at its tidal channel aiming at stabilising its water level. Main physico-chemical parameters, together with macroalgae biomass, nutrient concentrations in the water column, particulate organic carbon, and chlorophyll a were recorded monthly at 4 sampling stations within the Piratininga lagoon from April 1994 to April 1995. The data, compared with “before-lock” existing studies, show that nutrients and chlorophyll a concentrations significantly increased after the lock construction. Based on the functioning of the ecosystem, we propose to harvest the algal mats before their decomposition period in order to partly remove the nutrient stocks from the lagoon and the future ecosystem modelling to predict the impact of natural and anthropogenic eutrophication.     

It's a dirty job but someone has to do it: The role of marine benthic macrofauna in organic matter turnover and nutrient recycling to the water column

Benthic macrofaunal populations through their feeding, bioturbation, burrow construction and sediment irrigation activities have profound influences on organic matter inputs to marine sediments (biodeposition) and on the vertical distribution of deposited organic matter within the sediment. These effects in turn influence the rates and pathways of organic matter mineralisation, and element cycles. Similarly, bioturbation, burrow construction and burrow irrigation are major determinants of sediment-water column fluxes of oxygen and nutrients.

In this review, I discuss the influences of the different benthic macrofaunal feeding (functional) groups on mineralisation processes and sediment-water column fluxes of particulate and dissolved nutrients. How these effects influence diagenic processes, the balance between aerobic and anaerobic processes, and the redox status of the surficial sediments. Finally, I discuss some of the limitations of the predominantly laboratory techniques which have been used to study “macrofaunal effects” and how this hinders the inclusion of the effects in quantitative sediment biogeochemical models.     

It's a dirty job but someone has to do it: The role of marine benthic macrofauna in organic matter turnover and nutrient recycling to the water column

Benthic macrofaunal populations through their feeding, bioturbation, burrow construction and sediment irrigation activities have profound influences on organic matter inputs to marine sediments (biodeposition) and on the vertical distribution of deposited organic matter within the sediment. These effects in turn influence the rates and pathways of organic matter mineralisation, and element cycles. Similarly, bioturbation, burrow construction and burrow irrigation are major determinants of sediment-water column fluxes of oxygen and nutrients.

In this review, I discuss the influences of the different benthic macrofaunal feeding (functional) groups on mineralisation processes and sediment-water column fluxes of particulate and dissolved nutrients. How these effects influence diagenic processes, the balance between aerobic and anaerobic processes, and the redox status of the surficial sediments. Finally, I discuss some of the limitations of the predominantly laboratory techniques which have been used to study “macrofaunal effects” and how this hinders the inclusion of the effects in quantitative sediment biogeochemical models.     

Recruitment and attrition of associated plants under a shading crop canopy: Model selection and calibration

Associated plant and animal diversity provides ecosystem services within crop production systems. The importance of the maintenance or restoration of diversity is therefore increasingly acknowledged. Here we study the population dynamics of associated annual plants (‘weeds’) during the growth of a crop in a season and introduce a minimal model to characterize the recruitment and attrition of the associated plants under the influence of shading by the crop. A mechanistically based, logistic, light interception model was parameterized with light interception measurements in two single crops (barley and rye) and in mixtures of these cereals with peas. Population dynamics data were collected for the annuals Papaver rhoeas, Centaurea cyanus, Chrysanthemum segetum, and Misopates orontium. A minimal population dynamics model was identified for each annual plant species, using system identification techniques as model selection and calibration.