Conservation decision-making in large state spaces

When looking for the best course of management decisions to efficiently conserve metapopulation systems, a classic approach in the ecology literature is to model the optimisation problem as a Markov decision process and find an optimal control policy using exact stochastic dynamic programming techniques. Stochastic dynamic programming is an iterative procedure that seeks to optimise a value function at each timestep by evaluating the benefits of each of the actions in each state of the system defined in the Markov decision process.Although stochastic dynamic programming methods provide an optimal solution to conservation management questions in a stochastic world, their applicability in metapopulation problems has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. The high computational requirements of stochastic dynamic programming methods mean that only simple metapopulation management problems can be analysed. In this paper we overcome the complexity burden of exact stochastic dynamic programming methods and present the benefits of an on-line sparse sampling algorithm proposed by Kearns, Mansour and Ng (2002). The algorithm is particularly attractive for problems with large state spaces as the running time is independent of the size of the state space of the problem. This appealing improvement is achieved at a cost: the solutions found are no longer guaranteed to be optimal.We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation problem where the management objective is to maximise the number of occupied patches over the management time horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the optimal solution to the results of the on-line sparse sampling algorithm for a simple 3-waterhole case. We find that three look-ahead steps minimises the error between the optimal solution and the approximation algorithm. This paper introduces a new algorithm to conservation management that provides a way to avoid the effects of the curse of dimensionality. The work has the potential to allow us to approximate solutions to much more complex metapopulation management problems in the future.