Testing model accuracy measures according to the EU directives—examples using the chemical transport model REM-CALGRID

The paper presents a comprehensive model evaluation focusing on the meaning and shortcomings of accuracy measures used to determine model quality according to European Union (EU) directives on air quality. European wide simulations employing the chemical transport model REM-CALGRID for the year 2002 were compared with O₃, NO₂, SO₂ and PM₁₀ observations of the German measurement network.The EU model quality objective, which is based on maximum relative errors, tends to penalise (i) the overestimation of very low measured concentrations in the case of annual averages and (ii) the underestimation of extremely high measured concentrations in the case of short-term values. As a more robust alternative, a model accuracy measure is presented, which corresponds to the allowed number of exceedances of the corresponding short-term air quality limit values.The influence of the spatial heterogeneity of the observations in relation to the spatial resolution of the model is investigated by spatial averaging of observation data. Because of this heterogeneity, any model with a 25 km resolution would fail to simulate about 20% of all NO₂ and SO₂ stations and 5–10% of all O₃ and PM₁₀ stations in Germany according to the EU model quality objectives for short-term averages.