Testing ecological models: the meaning of validation

The ecological literature reveals considerable confusion about the meaning of validation in the context of simulation models. The confusion arises as much from semantic and philosophical considerations as from the selection of validation procedures. Validation is not a procedure for testing scientific theory or for certifying the ‘truth’ of current scientific understanding, nor is it a required activity of every modelling project. Validation means that a model is acceptable for its intended use because it meets specified performance requirements.Before validation is undertaken, (1) the purpose of the model, (2) the performance criteria, and (3) the model context must be specified. The validation process can be decomposed into several components: (1) operation, (2) theory, and (3) data. Important concepts needed to understand the model evaluation process are verification, calibration, validation, credibility, and qualification. These terms are defined in a limited technical sense applicable to the evaluation of simulation models, and not as general philosophical concepts. Different tests and standards are applied to the operational, theoretical, and data components. The operational and data components can be validated; the theoretical component cannot.The most common problem with ecological and environmental models is failure to state what the validation criteria are. Criteria must be explicitly stated because there are no universal standards for selecting what test procedures or criteria to use for validation. A test based on comparison of simulated versus observed data is generally included whenever possible. Because the objective and subjective components of validation are not mutually exclusive, disagreements over the meaning of validation can only be resolved by establishing a convention.     

沈阳市ADMS-Urban大气扩散模型的验证及应用

利用沈阳市 1999年度污染源排放清单和全年逐时气象数据验证了针对总悬浮颗粒物和二氧化硫的ADMS -Urban大气扩散模型 ,并将该模型应用到沈阳市大气环境容量测算和环境空气质量数值预报工作     

Effects of calibration on L-THIA GIS runoff and pollutant estimation

Urbanization can result in alteration of a watershed's hydrologic response and water quality. To simulate hydrologic and water quality impacts of land use changes, the Long-Term Hydrologic Impact Assessment (L-THIA) system has been used. The L-THIA system estimates pollutant loading based on direct runoff quantity and land use based pollutant coefficients. The accurate estimation of direct runoff is important in assessing water quality impacts of land use changes. An automated program was developed to calibrate the L-THIA model using the millions of curve number (CN) combinations associated with land uses and hydrologic soil groups. L-THIA calibration for the Little Eagle Creek (LEC) watershed near Indianapolis, Indiana was performed using land use data for 1991 and daily rainfall data for six months of 1991 (January 1-June 30) to minimize errors associated with use of different temporal land use data and rainfall data. For the calibration period, the Nash-Sutcliffe coefficient was 0.60 for estimated and observed direct runoff. The calibrated CN values were used for validation of the model for the same year (July 1-December 31), and the Nash-Sutcliffe coefficient was 0.60 for estimated and observed direct runoff. The Nash-Sutcliffe coefficient was 0.52 for January 1, 1991 to December 31, 1991 using uncalibrated CN values. As shown in this study, the use of better input parameters for the L-THIA model can improve accuracy. The effects on direct runoff and pollutant estimation of the calibrated CN values in the L-THIA model were investigated for the LEC. Following calibration, the estimated average annual direct runoff for the LEC watershed increased by 34%, total nitrogen by 24%, total phosphorus by 22%, and total lead by 43%. This study demonstrates that the L-THIA model should be calibrated and validated prior to application in a particular watershed to more accurately assess the effects of land use changes on hydrology and water quality.     

Analysis of a Farquhar-von Caemmerer-Berry leaf-level photosynthetic rate model for Populus tremuloides in the context of modeling and measurement limitations

The balance of mechanistic detail with mathematical simplicity contributes to the broad use of the Farquhar, von Caemmerer and Berry (FvCB) photosynthetic rate model. Here the FvCB model was coupled with a stomatal conductance model to form an [A,g_s] model, and parameterized for mature Populus tremuloides leaves under varying CO₂ and temperature levels. Data were selected to be within typical forest light, CO₂ and temperature ranges, reducing artifacts associated with data collected at extreme values. The error between model-predicted photosynthetic rate (A) and A data was measured in three ways and found to be up to three times greater for each of two independent data sets than for a base-line evaluation using parameterization data. The evaluation methods used here apply to comparisons of model validation results among data sets varying in number and distribution of data, as well as to performance comparisons of [A,g_s] models differing in internal-process components.     

Development and field validation of an indicator to assess the relative mobility and risk of pesticides in the Lourens River catchment,South Africa

A GIS based pesticide risk indicator that integrates exposure variables (i.e. pesticide application, geographic, physicochemical and crop data) and toxicity endpoints (using species sensitivity distributions) was developed to estimate the Predicted Relative Exposure (PREX) and Predicted Relative Risk (PRRI) of applied pesticides to aquatic ecosystem health in the Lourens River catchment, Western Cape, South Africa. Samples were collected weekly at five sites from the beginning of the spraying season (October) till the beginning of the rainy season (April) and were semi quantitatively analysed for relevant pesticides applied according to the local farmers spraying programme. Monitoring data indicate that physicochemical data obtained from international databases are reliable indicators of pesticide behaviour in the Western Cape of South Africa. Sensitivity analysis identified K_OC as the most important parameter influencing predictions of pesticide loading derived from runoff. A comparison to monitoring data showed that the PREX successfully identified hotspot sites, gave a reasonable estimation of the relative contamination potential of different pesticides at a site and identified important routes of exposure (i.e. runoff or spray drift) of different pesticides at different sites. All pesticides detected during a monitored runoff event, were indicated as being more associated with runoff than spray drift by the PREX. The PRRI identified azinphos-methyl and chlorpyrifos as high risk pesticides towards the aquatic ecosystem. These results contribute to providing increased confidence in the use of risk indicator applications and, in particular, could lead to improved utilisation of limited resources for monitoring and management in resource constrained countries.     

Modelling Tracer Dispersion from Landfills

Several wind tunnel experiments of tracer dispersion from reduced-scale landfill models are presented in this paper. Different experimental set-ups, hot-wire anemometry, particle image velocimetry and tracer concentration measurements were used for the characterisation of flow and dispersion phenomena nearby the models. The main aim of these experiments is to build an extensive experimental data set useful for model validation purposes. To demonstrate the potentiality of the experimental data set, a validation exercise on several mathematical models was performed by means of a statistical technique. The experiments highlighted an increase in pollutant ground level concentrations immediately downwind from the landfill because of induced turbulence and mean flow deflection. This phenomenon turns out to be predominant for the dispersion process. Tests with a different set-up showed an important dependence of the dispersion phenomena from the landfill height and highlighted how complex orographic conditions downwind of the landfill do not affect significantly the dispersion behaviour. Validation exercises were useful for model calibration, improving code reliability, as well as evaluating performances. The Van Ulden model proved to give the most encouraging results.     

Modeling human-induced climatic change: A summary for environmental managers

The rapid increase in atmospheric concentrations of greenhouse gases has caused concern because of their potential to alter the earth's radiation budget and disrupt current climate patterns While there are many uncertainties associated with use of general circulation models (GCMs), GCMs are currently the best available technology to project changes in climate associated with elevated gas concentrations. Results indicate increases in global temperature and changes in global precipitation patterns are likely as a result of doubled CO₂. GCMs are not reliable for use at the regional scale because local scale processes and geography are not taken into account. Comparison of results from five GCMs in three regions of the United States indicate high variability across regions and among models depending on season and climate variable. Statistical methods of scaling model output and nesting finer resolution models in global models are two techniques that may improve projections. Despite the many limitations in GCMs, they are useful tools to explore climate-earth system dynamics when used in conjunction with water resource and ecosystem models. A variety of water resource models showed significant alteration of regional hydrology when run with both GCM-generated and hypothetical climate scenarios, regardless of region or model complexity. Similarly, ecological models demonstrate the sensitivity of ecosystem production, nutrient dynamics, and distribution to changes in climate and CO₂ levels. We recommend the use of GCM-based scenarios in conjunction with water resource and ecosystem models to guide environmental management and policy in a “no-regrets” framework or as part of a precautionary approach to natural resource protection.     

Emission modelling and validation of VOCs' source strengths in air-conditioned office premises

An emission model for indoor volatile organic compounds (VOCs) based on mass balance considerations has been presented and validated under steady state conditions. Comparison were made for the measured and predicted concentrations of 37 selected VOCs and TVOC through a case intervention study on the filters of the ventilation system in a new commercial air-conditioned office building. The intervention involved replacing media filters with electronic and carbon filtration. TVOC and 37 compounds selected for their health and comfort impact, representation of major chemical classes that occur in indoor air and their utility as markers of pollution sources were studied. The concentration levels predicted by the model were compared with actual measurements. Twenty-five target compounds and the TVOC were adequately described by the model where the measured concentrations were in agreement with the predicted concentrations. Modeling of the remaining 12 compounds was found to be affected by the emission rates that were occupant related.     

Human Volunteer Kinematics in Rear-End Sled Collisions

Validation of new crash test dummies for rear-end collision testing requires human response data from pertinent test situations. Eleven human volunteers were exposed to 23 low-speed rear impacts to determine human response in well-defined test seats, and to quantify repeatability, variability and the effect of seat design on human response.

The results showed vertical motion of the volunteers’ H-point caused by ramping up along the seat, and an upward motion of the volunteers’ torso and head. The latter was caused by a combination of ramping up along the seatback and straightening of the thoracic kyphosis. During the first 100 ms, the volunteers flexed their necks. Thereafter, the volunteers extended their necks. These new data have proven to be useful in validation of rear-impact dummies.