Estimating multivariate ARIMA models: when is close not good enough?

The purpose of this study is to examine the forecasting abilities of the same multivariate autoregressive model estimated using two methods. The first method is the "exact method" used by the SCA System from Scientific Computing Associates. The second method is an approximation method as implemented in the MTS system by Automatic Forecasting Systems, Inc. ; The two methods were used to estimate a five-series multivariate autoregressive model for the Quenouille series on hog numbers, hog prices, corn prices, corn supply, and farm wage rates. The 82 observations were arbitrarily divided into two periods: the first 60 observations were used to estimate the models; then forecasts for one through eight years ahead were calculated for each possible point in the remaining 22 observations. The root mean square error (RMSE) using the SCA-estimated parameters was smaller than the RMSE using the MTS-estimated parameters for 38 of the 40 possible values (five variables by eight forecast horizons) and tied for one point. The average increase in the RMSE when using the MTS parameters was approximately 9 percent. Using the SCA parameters for forecasting provided smaller mean absolute error (MAE) for 35 of the 40 values, with the average increase from using the MTS parameters being approximately 5.6 percent. Using the SCA parameters provided smaller mean errors (ME) for 39 of the 40 values, with the average increase from using the MTS parameters being approximately .023. Thus, the SCA estimation method is shown to provide better forecasts than the MTS method for this one example.