Testing the generalization of artificial neural networks with cross-validation and independent-validation in modelling rice tillering dynamics

Neural networks (NN) rely on the inner structure of available data sets rather than on comprehension of the modeled processes between inputs and outputs. Therefore, neural networks have been regarded as highly empirical models with limited extrapolation capability to situations outside the range of the training and validation data sets. In this study, the generalization ability of neural networks in predicting rice tillering dynamics was tested and several techniques inducing the generalization ability of neural networks were compared. We compared the performance of cross-validated neural networks with independent-validated neural networks and found that neural networks were able to extrapolate and predict tillering dynamics if the data were within the range of inputs of the training set. An inadequate training set resulted in overfitting of available data and neural networks that were not generalizable. The training set size required to enable a neural network to generalize and predict rice tillering dynamics was found to be at least 9 times as many training patterns for each weight. When a large number of variables are included in the input vector of a neural network with inadequate amounts of training data, we strongly recommend that the dimension of the input vector is reduced using principle component analysis (PCA), correspondence analysis (CA) or similar techniques to decrease the number of weights before the training procedure to improve the generalization ability of the NN. If the amount of training data still is not sufficient after the dimension of the input vector is reduced, regularization techniques, such as early stopping, jittering, and especially the embedment of estimated results by a theoretical model into the training set, should be used to improve the generalization ability of the neural network. The generalization of neural networks presents a wide spectrum of problems, and the proposed approaches are not confined strictly to modelling rice tillering dynamics but can be applied to other agricultural and ecological systems.     

Pesticide prediction in ground water in North Carolina domestic wells using artificial neural networks

This paper illustrates the application of artificial neural networks (ANN) for prediction of pesticide occurrence in rural domestic wells from the available limited information. Among the three ANN models (a feed-forward back propagation [BP], a radial basis function [RBF] and an adaptive neural network-based fuzzy inference system [ANFIS]) employed for this investigation, the BP neural network was found to be superior to RBF and ANFIS type networks for the detection of pesticide occurrences in wells. For improved model prediction efficiency, optimization of network structure (e.g., number of hidden layers and number of nodes in each hidden layer) and spread (the width of the radial basis function) are important for BP and RBF type of network, respectively. A four layer BP network with a 3:2 neurons ratio of the first hidden layer to the second hidden layer produced better prediction performance efficiencies in terms of the pesticide detection efficiency (E_f), the root mean square error (RMSE), and the correlation coefficient (R) and the overall E_f of the BP neural network was found greater than 85%. Sensitivity analysis was performed to measure the relative importance of one input parameter over the other in pesticide occurrence in wells. It was shown in terms of the prediction efficiencies (E_f, RMSE, and R) of a four-layer BP neural network that the time of sample collection (TSC; month of the year), the depths of wells, and pesticide travel times (PTT) were more important parameters in the prediction of the pesticide occurrences in rural domestic wells. This means that the wells having shallow ground water table are more vulnerable to pesticide occurrence.     

Prediction of sulphur removal with Acidithiobacillus sp. using artificial neural networks

Artificial neural network (ANN) model was used to predict the extent of sulphur removal from three types of coal using native cultures of Acidithiobacillus ferrooxidans. The type of coal, initial pH, pulp density, particle size, residence time, media composition and initial sulphur content of coal were fed as input to the network. The output of the model was sulphur removal. The resulting ANN showed satisfactory prediction of sulphur removal percentages with mean absolute deviations varying from 0.003 to 0.5. A three layer feed forward neural network model consisting of an input layer, one hidden layer and an output layer was found to give satisfactory results. Although the number of data sets were limited, the parity plot shows that the model estimations for the test set was good.     

Modelling algal blooms using vector autoregressive model with exogenous variables and long memory filter

Algal blooms (ABs), which commonly occur in urbanised coastal marine environments worldwide, often result in hypoxia and even fish kills. Understanding the mechanism and providing accurate prediction of ABs’ formation and occurrence is of foremost importance in relation to the protection of sensitive marine resources. In this paper, a multivariate time series model, namely the vector autoregressive model with exogenous variables (VARX) and the long memory filter is proposed to model and predict ABs. To evaluate the effectiveness of this VARX model, both daily and 2-h field monitoring data of chlorophyll fluorescence (CHL), dissolved oxygen (DO), total inorganic nitrogen (TIN), water temperature (TEMP), solar radiation (SR) and wind speed (WS) obtained at Kat O, Hong Kong, between February 2000 and March 2003 were employed. Unlike the other data driven approaches, this VARX model not only provides more interpretable effects of specific lags of environmental factors, but also sheds light on the feedback effects of AB on these variables. In general, daily CHL measurements up to 4 days can provide crucial information for predicting algal dynamics, while the VARX model is able to explicitly reveal ecological relationships between CHL and other environmental factors. In addition, the application of long-memory filter can further extract patterns of seasonal variations which is thought to be correspondent to the variation of algal species in Hong Kong water. With a view to providing an early warning signal of AB to fishermen and regulatory authorities, an alarming system was developed based on the VARX model; it could achieve 83% correct prediction of AB occurrences with a lead time of 2.5 days. Concerning the forecast performance of the VARX model, daily forecasting performance is comparatively better than that of artificial neural network models.     

城市给水管网余氯的预测模型

选择余氯为研究对象,以南方某市给水管网水质监测的数据为基础,使用线性回归和非线性神经网络(ANN)方法建立模型,找到了一种利用在线监测数据和人工监测数据实时预测管网余氯的方法。通过建立给水管网水质模型,可以由监测系统动态回传的数据来实时的预测下一天人工点的水质。模拟的结果显示ANN模型比线性回归模型有更好的预测能力,预测的平均相对误差:ANN模型为14.9%,线性回归模型为25.8%。使用ANN模型可以实现实时预测。     

Artificial neural network modeling of the river water quality—A case study

The paper describes the training, validation and application of artificial neural network (ANN) models for computing the dissolved oxygen (DO) and biochemical oxygen demand (BOD) levels in the Gomti river (India). Two ANN models were identified, validated and tested for the computation of DO and BOD concentrations in the Gomti river water. Both the models employed eleven input water quality variables measured in river water over a period of 10 years each month at eight different sites. The performance of the ANN models was assessed through the coefficient of determination (R²) (square of the correlation coefficient), root mean square error (RMSE) and bias computed from the measured and model computed values of the dependent variables. Goodness of the model fit to the data was also evaluated through the relationship between the residuals and model computed values of DO and BOD. The model computed values of DO and BOD by both the ANN models were in close agreement with their respective measured values in the river water. Relative importance and contribution of the input variables to the model output was evaluated through the partitioning approach. The identified ANN models can be used as tools for the computation of water quality parameters.     

Applications of artificial neural networks for patterning and predicting aquatic insect species richness in running waters

Two artificial neural networks (ANNs), unsupervised and supervised learning algorithms, were applied to suggest practical approaches for the analysis of ecological data. Four major aquatic insect orders (Ephemeroptera, Plecoptera, Trichoptera, and Coleoptera, i.e. EPTC), and four environmental variables (elevation, stream order, distance from the source, and water temperature) were used to implement the models. The data were collected and measured at 155 sampling sites on streams of the Adour–Garonne drainage basin (South-western France). The modelling procedure was carried out following two steps. First, a self-organizing map (SOM), an unsupervised ANN, was applied to classify sampling sites using EPTC richness. Second, a backpropagation algorithm (BP), a supervised ANN, was applied to predict EPTC richness using a set of four environmental variables. The trained SOM classified sampling sites according to a gradient of EPTC richness, and the groups obtained corresponded to geographic regions of the drainage basin and characteristics of their environmental variables. The SOM showed its convenience to analyze relationships among sampling sites, biological attributes, and environmental variables. After accounting for the relationships in data sets, the BP used to predict the EPTC richness with a set of four environmental variables showed a high accuracy (r=0.91 and r=0.61 for training and test data sets respectively). The prediction of EPTC richness is thus a valuable tool to assess disturbances in given areas: by knowing what the EPTC richness should be, we can determine the degree to which disturbances have altered it. The results suggested that methodologies successively using two different neural networks are helpful to understand ecological data through ordination first, and then to predict target variables.     

Modelling algal blooms in the Dutch coastal waters by integrated numerical and fuzzy cellular automata approaches

In this paper, an integrated numerical and fuzzy cellular automata model was developed to predict possible algal blooms in Dutch coastal waters basing on the irradiance, nutrients and neighbourhood conditions. The numerical module used Delft3D-WAQ to compute the abiotic conditions, and fuzzy cellular automata approach was applied to predict the algal biomass that was indicated by chlorophyll a concentration. The simulated results of year 1995 were compared with that from BLOOM II model, and the advantages, disadvantages as well as future improvement were presented. In general, through this study, it is seen that the integrated modelling deserves more research inputs because: (1) the hydrodynamic processes and nutrients concentrations can be simulated in details by numerical method; (2) the irregular and sparse water quality and biological data, and the empirical knowledge from experts can be explored by the fuzzy logic technique; (3) the spatial heterogeneity, local interactions and the emerge of patchiness could be well captured through the cellular automata paradigm.     

Modelling the effects of green macroalgae blooms on the population dynamics of Cyathura carinata (Crustacea: Isopoda) in an eutrophied estuary

A population dynamics model was developed to simulate the effects of benthic macroalgae blooms (mostly Enteromorpha spp.) on the productivity of Cyathura carinata (Crustacea: Isopoda), a possible keystone species in the benthic communities of the Mondego estuary. The model describes C. carinata population dynamics, as well as the relationships between Enteromorpha biomass, Enteromorpha decaying rates, organic matter content in the sediments and detritus consumption by C. carinata, a detritic feeder. Model results support the idea that seasonal blooms of Enteromorpha determine a significant increase of organic matter content in the sediments, due to macroalgae decay, which initially contributes to enhance C. carinata consumption and growth rates, determining a significant increase in the biomass. Nevertheless, later, following the algae bloom, C. carinata biomass decreases, and reaches its lowest value, close to 0, when the algae crash. This effect is probably related with strong anoxic conditions, especially during night, due to high algal decomposition rates. In accordance with the model, migration of new individuals from adjacent areas must occur in order to recolonise the area affected by the algae bloom. Therefore, it seems reasonable to conclude that macroalgae blooms that are limited in space may favour C. carinata populations, but extensive blooms affecting the whole area of distribution of this species will determine its disappearance.     

Determining factors that influence the dispersal of a pelagic species: A comparison between artificial neural networks and evolutionary algorithms

Because of increasing transport and trade there is a growing threat of marine invasive species being introduced into regions where they do not presently occur. So that the impacts of such species can be mitigated, it is important to predict how individuals, particularly passive dispersers are transported and dispersed in the ocean as well as in coastal regions so that new incursions of potential invasive species are rapidly detected and origins identified. Such predictions also support strategic monitoring, containment and/or eradication programs. To determine factors influencing a passive disperser, around coastal New Zealand, data from the genus Physalia (Cnidaria: Siphonophora) were used. Oceanographic data on wave height and wind direction and records of occurrences of Physalia on swimming beaches throughout the summer season were used to create models using artificial neural networks (ANNs) and Na?ve Bayesian Classifier (NBC). First, however, redundant and irrelevant data were removed using feature selection of a subset of variables. Two methods for feature selection were compared, one based on the multilayer perceptron and another based on an evolutionary algorithm. The models indicated that New Zealand appears to have two independent systems driven by currents and oceanographic variables that are responsible for the redistribution of Physalia from north of New Zealand and from the Tasman Sea to their subsequent presence in coastal waters. One system is centred in the east coast of northern New Zealand and the other involves a dynamic system that encompasses four other regions on both coasts of the country. Interestingly, the models confirm, molecular data obtained from Physalia in a previous study that identified a similar distribution of systems around New Zealand coastal waters. Additionally, this study demonstrates that the modelling methods used could generate valid hypotheses from noisy and complicated data in a system about which there is little previous knowledge.     

Modelling the dynamics of coral reef macroalgae using a Bayesian belief network approach

Macroalgae are a major benthic component of coral reefs and their dynamics influence the resilience of coral reefs to disturbance. However, the relative importance of physical and ecological processes in driving macroalgal dynamics is poorly understood. Here we develop a Bayesian belief network (BBN) model to integrate many of these processes and predict the growth of coral reef macroalgae. Bayesian belief networks use probabilistic relationships rather than deterministic rules to quantify the cause and effect assumptions. The model was developed using both new empirical data and quantified relationships elicited from previous studies. We demonstrate the efficacy of the BBN to predict the dynamics of a common Caribbean macroalgal genus Dictyota. Predictions of the model have an average accuracy of 55% (implying that 55% of the predicted categories of Dictyota cover were assigned to the correct class). Sensitivity analysis suggested that macroalgal dynamics were primarily driven by top–down processes of grazing rather than bottom–up nutrification. BBNs provide a useful framework for modelling complex systems, identifying gaps in our scientific understanding and communicating the complexities of the associated uncertainties in an explicit manner to stakeholders. We anticipate that accuracies will improve as new data are added to the model.     

Prediction of longitudinal dispersion coefficients in natural rivers using artificial neural network

An artificial neural network (ANN) model is developed for predicting the longitudinal dispersion coefficient in natural rivers. The model uses few rivers’ hydraulic and geometric characteristics, that are readily available, as the model input, and the target output is the longitudinal dispersion coefficient (K). For performance evaluation of the model, using published field data, predictions by the developed ANN model are compared with those of other reported important models. Based on various performance indices, it is concluded that the new model predicts the longitudinal dispersion coefficient more accurately. Sensitive analysis performed on input parameters indicates stream width, flow depth, stream sinuosity, flow velocity, and shear velocity to be the most influencing parameters for accurate prediction of the longitudinal dispersion coefficient.     

The application of artificial neural networks to flow and phosphorus dynamics in small streams on the Boreal Plain,with emphasis on the role of wetlands

Historically, management strategies in Canada's boreal forest have focused on forest polygons and terrestrial biodiversity to address ecological considerations in forest management. The Forest Watershed and Riparian Disturbance (FORWARD) project examines the problem from a watershed perspective rather than a forest polygon viewpoint. The main objective of this study was to devise an artificial neural network (ANN) modeling tool that can predict flow and total phosphorus (TP) concentration for ungauged watersheds (where daily flow is not monitored). This dictates that all inputs should be easily accessed via a public domain database, like the Environment Canada weather database, without the need to install flow gauges in each modeled watershed. Daily flow and TP concentration for two of the project watersheds were modeled using ANNs. The two watersheds (1A Creek, 5.1 km² and Willow Creek, 15.6 km²) were chosen to reflect variations in wetland area and composition in the study area. Flow was modeled with a feed-forward multilayer perceptron ANN trained with the error back-propagation algorithm. Simulated values for flow were then used, as inputs, to model TP concentration using the same neural networks algorithm. One hidden layer with three slabs; each operating with a different activation function was utilized to simulate the conceptual differences between base flow, snowmelt, and storm events. Time domain analysis was conducted to identify possible model time-lagged inputs reflecting the time dependency of the modeled variables. Spectral analysis was used to address data hystereses. Our results highlight the capabilities of ANN in modeling complex ecosystems and highly correlated variables. Results also indicated that more research towards the phosphorus dynamics in wetlands is required to better represent the impact of wetland area and composition on the water-phase phosphorus in ANN modeling.     

Annual cycles of phytoplankton community-structure and bloom dynamics in the Neuse River Estuary, North Carolina

The spatiotemporal distributions of major phytoplankton taxa were quantified to estimate the relative contribution of different microalgal groups to biomass and bloom dynamics in the eutrophic Neuse River Estuary, North Carolina, USA. Biweekly water samples and ambient physical and chemical data were examined at sites along a salinity gradient from January 1994 through December 1996. Chemosystematic photopigments (chlorophylls and carotenoids) were identified and quantified using high-performance liquid chromatography (HPLC). A recently-developed factor-analysis procedure (CHEMTAX) was used to partition the algal group-specific chlorophyll a (chl a) concentrations based on photopigment concentrations. Results were spatially and temporally integrated to determine the ecosystem-level dynamics of phytoplankton community-constituents. Seasonal patterns of phytoplankton community-composition changes were observed over the 3 yr. Dinoflagellates reached maximum abundance in the late winter to early spring (January to March), followed by a spring diatom bloom (May to July). Cyanobacteria were more prevalent during summer months and made a large contribution to phytoplankton biomass, possibly in response to nutrient-enriched freshwater discharge. Cryptomonad blooms were not associated with a particular season, and varied from year to year. Chlorophyte abundance was low, but occasional blooms occurred during spring and summer. Over the 3 yr period, the total contribution of each algal group, in terms of chl a, was evenly balanced, with each contributing nearly 20% of the total chl a. Cryptomonad, chlorophyte, and cyanobacterial dynamics did not exhibit regular seasonal bloom patterns. High dissolved inorganic-nitrogen loading during the summer months promoted major blooms of cryptomonads, chlorophytes, and cyanobacteria. Received: 12 September 1997 / Accepted: 12 December 1997     

A simulation model for Japanese sardine (Sardinops melanostictus) migrations in the western North Pacific

A two-dimensional individual-based model coupled with fish bioenergetics was developed to simulate migration and growth of Japanese sardine (Sardinops melanostictus) in the western North Pacific. In the model, fish movement is controlled by feeding and spawning migrations with passive transport by simulated ocean current. Feeding migration was assumed to be governed by search for local optimal habitats, which is estimated by the spatial distribution of net growth rate of a sardine bioenergetics model. The forage density is one of the most important factors which determines the geographical distributions of Japanese sardine during their feeding migrations. Spawning migration was modeled by an artificial neural network (ANN) with an input layer composed of five neurons that receive environmental information (surface temperature, temperature change experienced, current speed, day length and distance from land). Once the weight of the ANN was determined, the fish movement was solved by combining with the feeding migration model. To obtain the weights of the ANN, three experiments were conducted in which (1) the ANN was trained with back propagation (BP) method with optimum training data, (2) genetic algorithm (GA) was used to adjust the weights and (3) the weights of the ANN were decided by the GA with BP, respectively. BP is a supervised learning technique for training ANNs. GA is a search technique used in computing to find approximate solutions, such as optimization of parameters. Condition factor of sardine in the model is used as a factor of optimization in the GA works. The methods using only BP or GA did not work to search the appropriate weights in the ANN for spawning migration. In the third method, which is a combined approach of GA with BP, the model reproduced the most realistic spawning migration of Japanese sardine. The changes in temperature and day length are important factors for the orientation cues of Japanese sardine according to the sensitivity analysis of the weights of the ANN.     

Optimization and modeling of coagulation-flocculation to remove algae and organic matter from surface water by response surface methodology

Charge neutralization and sweep flocculation were the major mechanisms. Effect of process parameters was investigated. Optimal coagulation conditions were studied by response surface methodology. ANN models presented more robust and accurate prediction than RSM. Seasonal algal blooms of Lake Yangcheng highlight the necessity to develop an effective and optimal water treatment process to enhance the removal of algae and dissolved organic matter (DOM). In the present study, the coagulation performance for the removal of algae, turbidity, dissolved organic carbon (DOC) and ultraviolet absorbance at 254 nm (UV₂₅₄) was investigated systematically by central composite design (CCD) using response surface methodology (RSM). The regression models were developed to illustrate the relationships between coagulation performance and experimental variables. Analysis of variance (ANOVA) was performed to test the significance of the response surface models. It can be concluded that the major mechanisms of coagulation to remove algae and DOM were charge neutralization and sweep flocculation at a pH range of 4.66–6.34. The optimal coagulation conditions with coagulant dosage of 7.57 mg Al/L, pH of 5.42 and initial algal cell density of 3.83 × 10⁶ cell/mL led to removal of 96.76%, 97.64%, 40.23% and 30.12% in term of cell density, turbidity, DOC and UV₂₅₄ absorbance, respectively, which were in good agreement with the validation experimental results. A comparison between the modeling results derived through both ANOVA and artificial neural networks (ANN) based on experimental data showed a high correlation coefficient, which indicated that the models were significant and fitted well with experimental results. The results proposed a valuable reference for the treatment of algae-laden surface water in practical application by the optimal coagulation-flocculation process.     

Neural network modeling of dissolved oxygen in the Gru?a reservoir, Serbia

The objective of this study is to develop a feedforward neural network (FNN) model to predict the dissolved oxygen in the Gru?a Reservoir, Serbia. The neural network model was developed using experimental data which are collected during a three years. The input variables of the neural network are: water pH, water temperature, chloride, total phosphate, nitrites, nitrates, ammonia, iron, manganese and electrical conductivity. Sensitivity analysis is used to determine the influence of input variables on the dependent variable. The most effective inputs are determined as pH and temperature, while nitrates, chloride and total phosphate are found to be least effective parameters. The Levenberg-Marquardt algorithm is used to train the FNN. The optimal FNN architecture was determined. The FNN architecture having 15 hidden neurons gives the best choice. Results of FNN models have been compared with the measured data on the basis of correlation coefficient (r), mean absolute error (MAE) and mean square error (MSE). Comparing the modelled values by FNN with the experimental data indicates that neural network model provides accurate results.     

Optimization of an artificial neural network for identifying fishing set positions from VMS data: An example from the Peruvian anchovy purse seine fishery

The spatial behavior of numerous fishing fleets is nowadays well documented thanks to satellite Vessel Monitoring Systems (VMS). Vessel positions are recorded on a frequent and regular basis which opens promising perspectives for improving fishing effort estimation and management. However, no specific information is provided on whether the vessel is fishing or not. To answer that question, existing works on VMS data usually apply simple criteria (e.g. threshold on speed). Those simple criteria generally focus in detecting true positives (a true fishing set detected as a fishing set); conversely, estimation errors are given no attention. For our case study, the Peruvian anchovy fishery, those criteria overestimate the total number of fishing sets by 182%. To overcome this problem an artificial neural network (ANN) approach is presented here. In order to set both the optimal parameterization and use “rules” for this ANN, we perform an extensive sensitivity analysis on the optimization of (1) the internal structure and training algorithm of the ANN and (2) the “rules” used for choosing both the relative size and the composition of the databases (DBs) used for training and inferring with the ANN. The “optimized” ANN greatly improves the estimates of the number and location of fishing events. For our case study, ANN reduces the total estimation error on the number of fishing sets to 1% (in average) and obtains 76% of true positives. This spatially explicit information on effort, provided with error estimation, should greatly reduce misleading interpretations of catch per unit effort and thus significantly improve the adaptive management of fisheries. While fitted on Peruvian anchovy fishery data, this type of neural network approach has wider potential and could be implemented in any fishery relying on both VMS and at-sea observer data. In order to increase the accuracy of the ANN results, we also suggest some criteria for improving sampling design by at-sea observers and VMS data.