Estimating parameters of hidden Markov models based on marked individuals: use of robust design data

Development and use of multistate mark-recapture models, which provide estimates of parameters of Markov processes in the face of imperfect detection, have become common over the last 20 years. Recently, estimating parameters of hidden Markov models, where the state of an individual can be uncertain even when it is detected, has received attention. Previous work has shown that ignoring state uncertainty biases estimates of survival and state transition probabilities, thereby reducing the power to detect effects. Efforts to adjust for state uncertainty have included special cases and a general framework for a single sample per period of interest. We provide a flexible framework for adjusting for state uncertainty in multistate models, while utilizing multiple sampling occasions per period of interest to increase precision and remove parameter redundancy. These models also produce direct estimates of state structure for each primary period, even for the case where there is just one sampling occasion. We apply our model to expected-value data, and to data from a study of Florida manatees, to provide examples of the improvement in precision due to secondary capture occasions. We have also implemented these models in program MARK. This general framework could also be used by practitioners to consider constrained models of particular interest, or to model the relationship between within-primary-period parameters (e.g., state structure) and between-primary-period parameters (e.g., state transition probabilities).     

Feasibility and stability in randomly assembled Lotka-Volterra models

For a Lotka-Volterra model to represent a viable ecosystem it's nontrivial equilibrium must be feasible. If m is the number of species, it is shown that in a set of randomly assembled Lotka-Volterra models, the fraction of models with a feasible equilibrium is some function of m which behaves like 2^?m. Moreover a subset of Lotka-Volterra models, each of which has a feasible equilibrium, has the same stability property as a set of linear models which is assembled randomly in the same manner. This contradicts a recent claim that a Lotka-Volterra model with a feasible equilibrium tends to be stable. Thus for two reasons the probability that a Lotka-Volterra model represents a viable and stable ecosystem decreases rapidly with the number of species. This supports the theme developed by May that stability in model ecosystems decreases with diversity.     

Analysis of behavioral changes of zebrafish (Danio rerio) in response to formaldehyde using Self-organizing map and a hidden Markov model

Two computational methods were applied to classification of movement patterns of zebrafish (Danio rerio) to elucidate Markov processes in behavioral changes before and after treatment of formaldehyde (0.1 mg/L) in semi-natural conditions. The complex data of the movement tracks were initially classified by the Self-organizing map (SOM) to present different behavioral states of test individuals. Transition probabilities between behavioral states were further evaluated to fit Markov processes by using the hidden Markov model (HMM). Emission transition probability was also obtained from the observed variables (i.e., speed) for training with the HMM. Experimental transition and emission probability matrices were successfully estimated with the HMM for recognizing sequences of behavioral states with accuracy rates in acceptable ranges at central and boundary zones before (77.3-81.2%) and after (70.1-76.5%) treatment. A heuristic algorithm and a Markov model were efficiently combined to analyze movement patterns and could be a means of in situ behavioral monitoring tool.     

Effects of Tour Boats on the Behavior of Bottlenose Dolphins: Using Markov Chains to Model Anthropogenic Impacts

Abstract:   Nature-based tourism activities have been developing over the last decade, but it is still difficult to manage these activities sustainably. This sector is increasingly focusing on whales and dolphins in coastal communities, but the exact effects of these tourism activities are unclear. Markov chain modeling may help researchers assess the effects of tourism activities on the behavioral budget of small cetaceans. Matrix models have been used widely in population ecology to provide successful management guidelines. From June 2000 to August 2001, I collected information on the behavioral state of bottlenose dolphin (  Tursiops spp.) schools from a population residing in Doubtful Sound, Fiordland, New Zealand. In addition, I recorded the occurrence of boat and dolphin interactions. I then calculated the transition probabilities of passing from one behavior to another by using a first-order, time-discrete Markov chain model. Behavioral transitions during which a boat-dolphin interaction occurred were compiled in an "impact" chain. All other transitions were tallied in a control chain. I then quantified the effect of boat-dolphin interactions during behavioral transitions by comparing the behavioral transition probabilities of both chains. Socializing and resting behaviors were disrupted by interactions with boats to a level that raises concern. Both the duration of bouts and the total amount of time spent in both these behavioral states were substantially decreased. Dolphins were significantly more likely to be traveling after an interaction with a boat. However, the overall behavioral budget of the population was not significantly affected. Therefore, the bottlenose dolphin population seems to be able to sustain the present level of boat interactions because of its low intensity. More effort is needed to develop prognosis analyses in order to understand how the effect of boat interactions on dolphins changes with variations in intensity.     

Combining multistate capture-recapture data with tag recoveries to estimate demographic parameters

Matrix population models that allow an animal to occupy more than one state over time are important tools for population and evolutionary ecologists. Definition of state can vary, including location for metapopulation models and breeding state for life history models. For populations whose members can be marked and subsequently reencountered, multistate mark-recapture models are available to estimate the survival and transition probabilities needed to construct population models. Multistate models have proved extremely useful in this context, but they often require a substantial amount of data and restrict estimation of transition probabilities to those areas or states subjected to formal sampling effort. At the same time, for many species, there are considerable tag recovery data provided by the public that could be modeled in order to increase precision and to extend inference to a greater number of areas or states. Here we present a statistical model for combining multistate capture-recapture data (e.g., from a breeding ground study) with multistate tag recovery data (e.g., from wintering grounds). We use this method to analyze data from a study of Canada Geese (Branta canadensis) in the Atlantic Flyway of North America. Our analysis produced marginal improvement in precision, due to relatively few recoveries, but we demonstrate how precision could be further improved with increases in the probability that a retrieved tag is reported.     

Multistate modeling of habitat dynamics: factors affecting Florida scrub transition probabilities

Many ecosystems are influenced by disturbances that create specific successional states and habitat structures that species need to persist. Estimating transition probabilities between habitat states and modeling the factors that influence such transitions have many applications for investigating and managing disturbance-prone ecosystems. We identify the correspondence between multistate capture-recapture models and Markov models of habitat dynamics. We exploit this correspondence by fitting and comparing competing models of different ecological covariates affecting habitat transition probabilities in Florida scrub and flatwoods, a habitat important to many unique plants and animals. We subdivided a large scrub and flatwoods ecosystem along central Florida's Atlantic coast into 10-ha grid cells, which approximated average territory size of the threatened Florida Scrub-Jay (Aphelocoma coerulescens), a management indicator species. We used 1.0-m resolution aerial imagery for 1994, 1999, and 2004 to classify grid cells into four habitat quality states that were directly related to Florida Scrub-Jay source-sink dynamics and management decision making. Results showed that static site features related to fire propagation (vegetation type, edges) and temporally varying disturbances (fires, mechanical cutting) best explained transition probabilities. Results indicated that much of the scrub and flatwoods ecosystem was resistant to moving from a degraded state to a desired state without mechanical cutting, an expensive restoration tool. We used habitat models parameterized with the estimated transition probabilities to investigate the consequences of alternative management scenarios on future habitat dynamics. We recommend this multistate modeling approach as being broadly applicable for studying ecosystem, land cover, or habitat dynamics. The approach provides maximum-likelihood estimates of transition parameters, including precision measures, and can be used to assess evidence among competing ecological models that describe system dynamics.     

Fitting complex population models by combining particle filters with Markov chain Monte Carlo

We show how a recent framework combining Markov chain Monte Carlo (MCMC) with particle filters (PFMCMC) may be used to estimate population state-space models. With the purpose of utilizing the strengths of each method, PFMCMC explores hidden states by particle filters, while process and observation parameters are estimated using an MCMC algorithm. PFMCMC is exemplified by analyzing time series data on a red kangaroo (Macropus rufus) population in New South Wales, Australia, using MCMC over model parameters based on an adaptive Metropolis-Hastings algorithm. We fit three population models to these data; a density-dependent logistic diffusion model with environmental variance, an unregulated stochastic exponential growth model, and a random-walk model. Bayes factors and posterior model probabilities show that there is little support for density dependence and that the random-walk model is the most parsimonious model. The particle filter Metropolis-Hastings algorithm is a brute-force method that may be used to fit a range of complex population models. Implementation is straightforward and less involved than standard MCMC for many models, and marginal densities for model selection can be obtained with little additional effort. The cost is mainly computational, resulting in long running times that may be improved by parallelizing the algorithm.     

A Markovian model for ecosystem flow analysis

Nutrient flow through ecosystems is modeled as a discrete Markov chain whose transition probabilities are stationary or time inhomogenous according to whether a steady state or dynamic ecosystem is modeled, respectively. Expected residence time and number of intercompartmental transfers for a nutrient within a set of compartments are derived. Variances of these random variables are also considered. A measure for ecosystem resource recycling is given as a weighted sum of probabilities.     

An application of Markov Decision Programming to ecologico-cybernetics—A decision method applied to the antifrost of winter wheat

Modern ecologico-cybernetic principle is of importance to decreasing damages in relation to agricultural productions. As an illustration of this, the authors studied some of the questions about the optimal policy of choosing actions for the antifrost measures of winter wheat by means of the finite-stage model of Markov Decision Programming (MDP). The related data came from the investigation results of the investigated region in the northeastern part of Henan, China. First, the authors give the states, ecologico-cybernetic action measures, transition probabilities and reward values in relation to these antifrost action measures of winter wheat crop. Second, the authors describe the principles and computational procedures of the ecologico-cybernetic decision-makings based on the finite-stage model of MDP. Third, a simple table applicable to the decision-making practice is given. Finally, we evaluate the results of this study, point out their shortcomings and suggest that this method is applicable to the other fields in relation to decreasing damage ecologico-cybernetics.     

A single-chain-based multidimensional Markov chain model for subsurface characterization

Multidimensional Markov chain models in geosciences were often built on multiple chains, one in each direction, and assumed these 1-D chains to be independent of each other. Thus, unwanted transitions (i.e., transitions of multiple chains to the same location with unequal states) inevitably occur and have to be excluded in estimating the states at unobserved locations. This consequently may result in unreliable estimates, such as underestimation of small classes (i.e., classes with smaller than average areas) in simulated realizations. This paper presents a single-chain-based multidimensional Markov chain model for estimation (i.e., prediction and conditional stochastic simulation) of spatial distribution of subsurface formations with borehole data. The model assumes that a single Markov chain moves in a lattice space, interacting with its nearest known neighbors through different transition probability rules in different cardinal directions. The conditional probability distribution of the Markov chain at the location to be estimated is formulated in an explicit form by following the Bayes’ Theorem and the conditional independence of sparse data in cardinal directions. Since no unwanted transitions are involved, the model can estimate all classes fairly. Transiogram models (i.e., 1-D continuous Markov transition probability diagrams) are used to provide transition probability input with needed lags to generalize the model. Therefore, conditional simulation can be conducted directly and efficiently. The model provides an alternative for heterogeneity characterization of subsurface formations.

General methods for sensitivity analysis of equilibrium dynamics in patch occupancy models

Sensitivity analysis is a useful tool for the study of ecological models that has many potential applications for patch occupancy modeling. Drawing from the rich foundation of existing methods for Markov chain models, I demonstrate new methods for sensitivity analysis of the equilibrium state dynamics of occupancy models. Estimates from three previous studies are used to illustrate the utility of the sensitivity calculations: a joint occupancy model for a prey species, its predators, and habitat used by both; occurrence dynamics from a well-known metapopulation study of three butterfly species; and Golden Eagle occupancy and reproductive dynamics. I show how to deal efficiently with multistate models and how to calculate sensitivities involving derived state variables and lower-level parameters. In addition, I extend methods to incorporate environmental variation by allowing for spatial and temporal variability in transition probabilities. The approach used here is concise and general and can fully account for environmental variability in transition parameters. The methods can be used to improve inferences in occupancy studies by quantifying the effects of underlying parameters, aiding prediction of future system states, and identifying priorities for sampling effort.     

Modeling survival and mark loss in molting animals: recapture, dead recoveries, and exuvia recoveries

Capture-mark-recapture (CMR) analyses aim primarily at estimating relevant life history parameters, despite the fact that some individuals are not always recaptured, even if alive on the study site. Applying such approaches to species with a complex life cycle, such as insects, remains challenging because each change of stage tends to cause mark loss through molting. We developed a multistate model based on three exclusive events ("dead", "surviving and molting", and "surviving and staying in the same larval stage") to estimate probabilities of survival and mark loss. Estimates of biologically relevant parameters were derived from those of the probabilities of transition between these states. The model was applied to data from radio-tracking diodes glued on grasshoppers. The estimates of recapture probabilities decreased throughout the season for animals remaining alive, while the detection of dead animals and lost diodes was exhaustive. The survival probability was higher for larvae than for adults (0.98 vs. 0.96), and mark loss was stronger in larvae than in adults (0.09 vs. 0.06). We show that the survival rate of a species with a high rate of mark loss can be estimated using multistate models, provided that marks can be recovered after being lost. These models are flexible enough to test for several effects that potentially affect survival and mark loss probabilities.     

Effects of surrounding land use and water depth on seagrass dynamics relative to a catastrophic algal bloom

Seagrasses are the foundation of many coastal ecosystems and are in global decline because of anthropogenic impacts. For the Indian River Lagoon (Florida, U.S.A.), we developed competing multistate statistical models to quantify how environmental factors (surrounding land use, water depth, and time [year]) influenced the variability of seagrass state dynamics from 2003 to 2014 while accounting for time‐specific detection probabilities that quantified our ability to determine seagrass state at particular locations and times. We classified seagrass states (presence or absence) at 764 points with geographic information system maps for years when seagrass maps were available and with aerial photographs when seagrass maps were not available. We used 4 categories (all conservation, mostly conservation, mostly urban, urban) to describe surrounding land use within sections of lagoonal waters, usually demarcated by land features that constricted these waters. The best models predicted that surrounding land use, depth, and year would affect transition and detection probabilities. Sections of the lagoon bordered by urban areas had the least stable seagrass beds and lowest detection probabilities, especially after a catastrophic seagrass die‐off linked to an algal bloom. Sections of the lagoon bordered by conservation lands had the most stable seagrass beds, which supports watershed conservation efforts. Our results show that a multistate approach can empirically estimate state‐transition probabilities as functions of environmental factors while accounting for state‐dependent differences in seagrass detection probabilities as part of the overall statistical inference procedure.     

A retrospective Markovian model for ecosystem resource flow

Historic ecosystem resource flow is modeled as a retrospective discrete Markov chain to obtain the expected values and variances of compartmental residence times and numbers of intercompartmental transfers that compartmental standing crops have experienced since their latest entry into the ecosystem. The transition probabilities of the proposed retrospective Markovian model are either stationary or time inhomogeneous according to whether a steady state or non-steady state ecosystem, respectively, is analyzed.     

Effects of tour boats on dolphin activity examined with sensitivity analysis of Markov chains

In Patagonia, Argentina, watching dolphins, especially dusky dolphins (Lagenorhynchus obscurus), is a new tourist activity. Feeding time decreases and time to return to feeding after feeding is abandoned and time it takes a group of dolphins to feed increase in the presence of boats. Such effects on feeding behavior may exert energetic costs on dolphins and thus reduce an individual's survival and reproductive capacity or maybe associated with shifts in distribution. We sought to predict which behavioral changes modify the activity pattern of dolphins the most. We modeled behavioral sequences of dusky dolphins with Markov chains. We calculated transition probabilities from one activity to another and arranged them in a stochastic matrix model. The proportion of time dolphins dedicated to a given activity (activity budget) and the time it took a dolphin to resume that activity after it had been abandoned (recurrence time) were calculated. We used a sensitivity analysis of Markov chains to calculate the sensitivity of the time budget and the activity-resumption time to changes in behavioral transition probabilities. Feeding-time budget was most sensitive to changes in the probability of dolphins switching from traveling to feeding behavior and of maintaining feeding behavior. Thus, an increase in these probabilities would be associated with the largest reduction in the time dedicated to feeding. A reduction in the probability of changing from traveling to feeding would also be associated with the largest increases in the time it takes dolphins to resume feeding. To approach dolphins when they are traveling would not affect behavior less because presence of the boat may keep dolphins from returning to feeding. Our results may help operators of dolphin-watching vessels minimize negative effects on dolphins.     

Quantifying parameter uncertainty in a coral reef model using Metropolis-Coupled Markov Chain Monte Carlo

Coral reefs are threatened ecosystems, so it is important to have predictive models of their dynamics. Most current models of coral reefs fall into two categories. The first is simple heuristic models which provide an abstract understanding of the possible behaviour of reefs in general, but do not describe real reefs. The second is complex simulations whose parameters are obtained from a range of sources such as literature estimates. We cannot estimate the parameters of these models from a single data set, and we have little idea of the uncertainty in their predictions.We have developed a compromise between these two extremes, which is complex enough to describe real reef data, but simple enough that we can estimate parameters for a specific reef from a time series. In previous work, we fitted this model to a long-term data set from Heron Island, Australia, using maximum likelihood methods. To evaluate predictions from this model, we need estimates of the uncertainty in our parameters. Here, we obtain such estimates using Bayesian Metropolis-Coupled Markov Chain Monte Carlo. We do this for versions of the model in which corals are aggregated into a single state variable (the three-state model), and in which corals are separated into four state variables (the six-state model), in order to determine the appropriate level of aggregation. We also estimate the posterior distribution of predicted trajectories in each case.In both cases, the fitted trajectories were close to the observed data, but we had doubts about the biological plausibility of some parameter estimates. We suggest that informative prior distributions incorporating expert knowledge may resolve this problem. In the six-state model, the posterior distribution of state frequencies after 40 years contained two divergent community types, one dominated by free space and soft corals, and one dominated by acroporid, pocilloporid, and massive corals. The three-state model predicts only a single community type. We conclude that the three-state model hides too much biological heterogeneity, but we need more data if we are to obtain reliable predictions from the six-state model. It is likely that there will be similarly large, but currently unevaluated, uncertainty in the predictions of other coral reef models, many of which are much more complex and harder to fit to real data.     

Hidden Markov models for circular and linear-circular time series

We introduce a new class of circular time series based on hidden Markov models. These are compared with existing models, their properties are outlined and issues relating to parameter estimation are discussed. The new models conveniently describe multi-modal circular time series as dependent mixtures of circular distributions. Two examples from biology and meteorology are used to illustrate the theory. Finally, we introduce a hidden Markov model for bivariate linear-circular time series and use it to describe larval movement of the fly Drosophila. Received: September 2003 / Revised: March 2004     

Using accelerometer,high sample rate GPS and magnetometer data to develop a cattle movement and behaviour model

The study described in this paper developed a model of animal movement, which explicitly recognised each individual as the central unit of measure. The model was developed by learning from a real dataset that measured and calculated, for individual cows in a herd, their linear and angular positions and directional and angular speeds. Two learning algorithms were implemented: a Hidden Markov model (HMM) and a long-term prediction algorithm. It is shown that a HMM can be used to describe the animal's movement and state transition behaviour within several “stay” areas where cows remained for long periods. Model parameters were estimated for hidden behaviour states such as relocating, foraging and bedding. For cows’ movement between the “stay” areas a long-term prediction algorithm was implemented. By combining these two algorithms it was possible to develop a successful model, which achieved similar results to the animal behaviour data collected. This modelling methodology could easily be applied to interactions of other animal species.     

Mapping species distributions in one dimension by non-homogeneous hidden Markov models: the case of freshwater pearl mussels in the River Dee

The investigation of species distributions in rivers involves data which are inherently sequential and unlikely to be fully independent. To take these characteristics into account, we develop a Bayesian hierarchical model for mapping the distribution of freshwater pearl mussels in the River Dee (Scotland). At the top of the hierarchy the likelihood is used to describe the sequence of sites in which mussels were observed or not. Given that false observations can occur, and that “not observed” means both that the species was not present and that it was not observed, a Markov prior is introduced at the second level of the hierarchy to represent the sequence of sites in which mussels are estimated to occur. The Markov prior allows modelling the spatial dependency between neighbouring sites. A third level in the hierarchy is given by the representation of the transition probabilities of the Markov chain in terms of site-specific explanatory variables, through a logistic regression. The selection of the explanatory variables which influence the Markov process is performed by means of a simulation-based procedure, in the complex case of association between covariates. Four features were found to be associated with reduced chance of finding a local mussel population: tributaries, bridges, dredging, and waste water treatment works. These results complement the results of a previous study, providing new evidence for the causes of the deterioration of a highly threatened species.     

A spatial model of growth and competition strategies in coral communities

A discrete spatial simulation model is developed to investigate the type and intensity of biological and physical factors influencing the structure of coral communities. The model represents reproduction, growth, and interspecific competition by coral colonies in terms of “ownership” of space in a plot of reef habitat. Using data for several eastern Pacific coral species, the model reproduces observed changes in species composition and diversity during coral community development. Model results suggest that during early successional stages, or in areas that are frequently disturbed, larval colonization and rapid growth are more important than dominance achieved by extracoelenteric digestion or by growing over another coral in acquiring and maintaining possession of reef substrate. In mature communities that remain undisturbed, dominance is the best competitive strategy. Although the model was developed to study natural and man-induced changes in the community dynamics of coral reefs, it could be adapted to study other sessile organisms where spatial pattern is an important influence on the frequency and outcome of biological interactions.