We present and show applications of two new test statistics for deciding if one ARIMA model provides significantly better h-step-ahead forecasts than another, as measured by the difference of approximations to their asymptotic mean square forecast errors. The two statistics differ in the variance estimate whose square root is the statistic's denominator. Both variance estimates are consistent even when the ARMA components of the models considered are incorrect. Our principal statistic's variance estimate accounts for parameter estimation. Our simpler statistic's variance estimate treats parameters as fixed. The broad consistency properties of these estimates yield improvements to what are known as tests of Diebold and Mariano (1995) type. These are tests whose variance estimates treat parameters as fixed and are generally not consistent in our context. We describe how the new test statistics can be calculated algebraically for any pair of ARIMA models with the same differencing operator. Our size and power studies demonstrate their superiority over the Diebold-Mariano statistic. The power study and the empirical study also reveal that, in comparison to treating estimated parameters as fixed, accounting for parameter estimation can increase power and can yield more plausible model selections for some time series in standard textbooks.
Diebold-Mariano tests; Incorrect models; Misspecified models; Model selection;
Parameter estimation effects; Time series
David F. Findley was the Senior Mathematical Statistican for Time Series
and is now retired from U. S. Census Bureau.
This symbol indicates a link to a non-government web site. Our linking to these sites does not constitute an endorsement of any products, services or the information found on them. Once you link to another site you are subject to the policies of the new site.