Revisiting a statistical shortcoming when fitting the Langmuir model to sorption data

The Langmuir model is commonly used for describing the sorption behavior of reactive solutes to surfaces and is often fit to sorption data using nonlinear least squares regression. An important assumption of least squares regression is that the predictor variable is error free. In the case of sorption data, this assumption is not valid, and therefore the potential for parameter bias exists. Although alternative regression methods exist that either explicitly account for error in the predictor variable (Model II regression) or minimize the error in the predictor variable, these methods are not commonly used. Therefore, this paper more fully explores the differences in fitted parameters and model fits between these different data fitting methods by fitting P sorption data collected on 26 different soil samples using three different regression methods. For a majority of soils tested in this study, the differences in model fits between the three regression methods were not statistically significant. Statistical differences were observed in over a third of the soils, however, suggesting that errors in the predictor variable may be large enough to produce biased parameter estimates. These results suggest that multiple regression methods should be used when fitting the Langmuir model to sorption data to better assess the potential impact of error on model fits.