Model uncertainty and forecast accuracy

In time-series analysis, a model is rarely pre-specified but rather is typically formulated in an iterative, interactive way using the given time-series data. Unfortunately the properties of the fitted model, and the forecasts from it, are generally calculated as if the model were known in the first place. This is theoretically incorrect, as least squares theory, for example, does not apply when the same data are used to formulates and fit a model. Ignoring prior model selection leads to biases, not only in estimates of model parameters but also in the subsequent construction of prediction intervals. The latter are typically too narrow, partly because they do not allow for model uncertainty. Empirical results also suggest that more complicated models tend to give a better fit but poorer ex-ante forecasts. The reasons behind these phenomena are reviewed. When comparing different forecasting models, the BIC is preferred to the AIC for identifying a model on the basis of within-sample fit, but out-of-sample forecasting accuracy provides the real test. Alternative approaches to forecasting, which avoid conditioning on a single model, include Bayesian model averaging and using a forecasting method which is not model-based but which is designed to be adaptable and robust.     

Combination of Forecasts across Estimation Windows: An Application to Air Travel Demand

This paper applies combining forecasts of air travel demand generated from the same model but over different estimation windows. The combination approach used resorts to Pesaran and Pick (Journal of Business Economics and Statistics 2011; 29 : 307–318), but the empirical application is extended in several ways. The forecasts are based on a seasonal Box–Jenkins model (SARIMA), which is adequate to forecast monthly air travel demand with distinct seasonal patterns at the largest German airport: Frankfurt am Main. Furthermore, forecasts with forecast horizons from 1 to 12 months ahead, which are based on different average estimation windows, expanding windows and single rolling windows, are compared with baseline forecasts based on an expanding window of the observations after a structural break. The forecast exercise shows that the average window forecasts mostly outperform the alternative single window forecasts. Copyright © 2016 John Wiley & Sons, Ltd.     

Sources of error in forecasts and expectations: U.K. economic models, 1984–8

This paper presents a comparative analysis of the sources of error in forecasts for the UK economy published over a recent four-year period by four independent groups. This analysis rests on the archiving at the ESRC Macroeconomic Modelling Bureau of the original forecasts together with all their accompanying assumptions and adjustments. A method of decomposing observed forecast errors so as to distinguish the contributions of forecaster and model is set out; the impact of future expectations treated in a ‘model-consistent’ or ‘rational’ manner is specifically considered. The results show that the forecaster's adjustments make a substantial contribution to forecast performance, a good part of which comes from adjustments that bring the model on track at the start of the forecast period. The published ex-ante forecasts are usually superior to pure model-based ex-post forecasts, whose performance indicates some misspecification of the underlying models.     

Forecasting volatility in commodity markets

This paper uses recent advances in time-series modeling to derive long-horizon forecasts of commodity price volatility which incorporate investors' expectations of volatility. Our results are promising. We compare several different forecasts of commodity price volatility, which we divide into three categories: (1) forecasts using only expectations derived from options prices; (2) forecasts using only time-series modeling; and (3) forecasts which combine market expectations and time-series methods. The forecasts in (1) and (2) are used extensively in the literature, while those in (3) are new in this paper. On comparing these different forecasts, we find that our proposed forecasts from category (3) outperform both market expectations forecasts and time-series forecasts. This result holds both in and out of sample for virtually all commodities considered.     

Assessing the historical accuracy of regional economic forecasts

This study addresses for the first time systematic evaluation of a widely used class of forecasts, regional economic forecasts. Ex ante regional structural equation model forecasts are analysed for 19 metropolitan areas. One- to ten-quarter-ahead forecasts are considered and the seven-year sample spans a complete business cycle. Counter to previous speculation in the literature, (1) dependency on macroeconomic forecasting model inputs does not substantially erode accuracy relative to univariate extrapolative methodologies and (2) stochastic time series models do not on average, yield more accurate regional economic predictions than structural models. Similar to findings in other studies, clear preferences among extrapolative methodologies do not emerge. Most general conclusions, however, are subject to caveats based on step-length effects and region-specific effects.     

Linear and Non-linear (Non-)Forecastability of High-frequency Exchange Rates

This paper forecasts Daily Sterling exchange rate returns using various naive, linear and non-linear univariate time-series models. The accuracy of the forecasts is evaluated using mean squared error and sign prediction criteria. These show only a very modest improvement over forecasts generated by a random walk model. The Pesaran–Timmerman test and a comparison with forecasts generated artificially shows that even the best models have no evidence of market timing ability.©1997 John Wiley & Sons, Ltd.     

Step-ahead spot price densities using daily synchronously reported prices and wind forecasts

This paper uses non-linear methodologies to follow the synchronously reported relationship between the Nordic/Baltic electric daily spot auction prices and geographical relevant wind forecasts in MWh from early 2013 to 2020. It is a well-known market (auctions) microstructure fact that the daily wind forecasts are information available to the market before the daily auction bid deadline at 11 a.m. The main objective is therefore to establish conditional and marginal step ahead spot price density forecast using a stochastic representation of the lagged, synchronously reported and stationary spot price and wind forecast movements. Using an upward expansion path applying the Schwarz (Bayesian information criterion [BIC]) criterion and a battery of residual test statistics, an optimal maximum likelihood process density is suggested. The optimal specification reports a significant negative covariance between the daily price and wind forecast movements. Conditional on bivariate lags from the SNP information and using the known market information for wind forecast movements at t₁, the paper establishes one-step-ahead bivariate and marginal day-ahead spot price movement densities. The result shows that wind forecasts significantly influence the synchronously reported spot price densities (means and volatilities). The paper reports day-ahead bivariate and marginal densities for spot price movements conditional on several very plausible price and wind forecast movements. The paper suggests day-ahead spot price predictions from conditional and synchronously reported wind forecasts movements. The information should increase market participants spot market insight and consequently make spot price predictions more accurate and the confidence interval considerably narrower.     

Judgemental revision of sales forecasts: The relative performance of judgementally revised versus non-revised forecasts

The judgemental revision of sales forecasts is an issue which is receiving increasing attention in the forecasting literature. This paper compares the performance of forecasts after revision by managers with that of the forecasts which were accepted by them without revision. The data set consists of sales forecasting data from an industrial company, spanning six quarterly periods and relating to some 900 individual products. The findings show that, in general, the improvements made by managers bring the forecast errors of revised forecasts more into line with non-revised forecasts, but the change is often marginal, and the best result is equivalence between revised and non-revised forecasts.     

Quarterly liverpool model with current partial information: An exercise in forecasting

This paper applies an algorithm for the solution of partial current information in rational expectation models to the quarterly Liverpool macroeconomic model of the U.K. The algorithm is shown to produce marginally superior results in forecasts both in ex-post and ex-ante forecasts and can be viewed as an additional tool for the forecaster's kit-bag.     

Forecasting the federal budget with time-series models

The stochastic properties of conventionally denned federal expenditures and revenues are examined, and cointegration is found. Alternative time-series models-univariate ARIMA models, vector autoregressions in levels and differences, and an error correction model-are specified and estimated using quarterly data from 1955:1 through 1979:4. Updated forecasts for up to three years beyond the sample period are evaluated against actual expenditures, revenues and the deficit. The vector autoregression in levels shows evidence of nonstationarity, which leads to strong biases in the forecasts. The remaining models produce forecasts that are satisfactory by the mean squared error criterion, and the magnitudes of biases at the longer horizons are significantly smaller than those of the official forecasts.     

Experience in combining subjective and quantitative forecasts of open market rates

This paper presents a method of combining subjective information from open-market operators with results from a time-series forecasting model. Empirical results of forecasts for interest rates of bank reserves are presented.     

Perspectives on forecasting research in accounting and finance

This paper offers some perspectives on forecasting research in accounting and finance. It is maintained that many common areas of forecasting research exist. Yet, most research has focused upon a particular (Box-Jenkins) technique and a particular (reported earnings) variable, virtually neglecting numerous other relevant forecasting research topics. This symposium issue includes papers which address several of these neglected research topics. The eight papers constituting the issue are classified into three categories: (1) univariate time-series modelling; (2) multivariate time-series modelling; and (3) comparison of experts' forecasts with those of statistical models. Following a summary of the papers, some suggestions for future research are offered.     

Forecasting for business planning: A case study of IBM product sales

This is a case study of a closely managed product. Its purpose is to determine whether time-series methods can be appropriate for business planning. By appropriate, we mean two things: whether these methods can model and estimate the special events or features that are often present in sales data; and whether they can forecast accurately enough one, two and four quarters ahead to be useful for business planning. We use two time-series methods, Box-Jenkins modeling and Holt-Winters adaptive forecasting, to obtain forecasts of shipments of a closely managed product. We show how Box-Jenkins transfer-function models can account for the special events in the data. We develop criteria for choosing a final model which differ from the usual methods and are specifically directed towards maximizing the accuracy of next-quarter, next-half-year and next-full-year forecasts. We find that the best Box-Jenkins models give forecasts which are clearly better than those obtained from Holt-Winters forecast functions, and are also better than the judgmental forecasts of IBM's own planners. In conclusion, we judge that Box-Jenkins models can be appropriate for business planning, in particular for determining at the end of the year baseline business-as-usual annual and monthly forecasts for the next year, and in mid-year for resetting the remaining monthly forecasts.     

An empirical investigation of combinations of economic forecasts

This paper examines the effects of combining three econometric and three times-series forecasts of growth and inflation in the U.K. If forecasts are unbiased then a combination exploiting this fact will be more efficient than an unrestricted combination. Ex post econometric forecasts may be biased but ex ante they are unbiased. The results of the study are that a restricted linear combination of the econometric forecasts is superior to an unrestricted combination and also to the unweighted mean of the forecasts. However, it is not preferred to the best of the individual forecasts.     

Forecasting load-duration curves

A new method is proposed for forecasting electricity load-duration curves. The approach first forecasts the load curve and then uses the resulting predictive densities to forecast the load-duration curve. A virtue of this procedure is that both load curves and load-duration curves can be predicted using the same model, and confidence intervals can be generated for both predictions. The procedure is applied to the problem of predicting New Zealand electricity consumption. A structural time-series model is used to forecast the load curve based on half-hourly data. The model is tailored to handle effects such as daylight savings, holidays and weekends, as well as trend, annual, weekly and daily cycles. Time-series methods, including Kalman filtering, smoothing and prediction, are used to fit the model and to achieve the desired forecasts of the load-duration curve.     

Analysis of many short time sequences: Forecast improvements achieved by shrinkage

Credibility models in actuarial science deal with multiple short time series where each series represents claim amounts of different insurance groups. Commonly used credibility models imply shrinkage of group-specific estimates towards their average. In this paper we model the claim size yu in group i and at time t as the sum of three independent components: y_it = μ_r + δ_i + ?_it. The first component, μt = μ_t?1 + m_t, represents time-varying levels that are common to all groups. The second component, δ_i, represents random group offsets that are the same in all periods, and the third component represents independent measurement errors. In this paper we show how to obtain forecasts from this model and we discuss the nature of the forecasts, with particular emphasis on shrinkage. We also assess the forecast improvements that can be expected from such a model. Finally, we discuss an extension of the above model which also allows the group offsets to change over time. We assume that the offsets for different groups follow independent random walks.     

An integrated bayesian vector auto regression and error correction model for forecasting electricity consumption and prices

The analysis and forecasting of electricity consumption and prices has received considerable attention over the past forty years. In the 1950s and 1960s most of these forecasts and analyses were generated by simultaneous equation econometric models. Beginning in the 1970s, there was a shift in the modeling of economic variables from the structural equations approach with strong identifying restrictions towards a joint time-series model with very few restrictions. One such model is the vector auto regression (VAR) model. It was soon discovered that the unrestricted VAR models do not forecast well. The Bayesian vector auto regression (BVAR) approach as well the error correction model (ECM) and models based on the theory of co integration have been offered as alternatives to the simple VAR model. This paper argues that the BVAF., ECM, and co integration models are simply VAR models with various restrictions placed on the coefficients. Based on this notion of a restricted VAR model, a four-step procedure for specifying VAR forecasting models is presented and then applied to monthly data on US electricity consumption and prices.     

On the Predictive Content of Autoregression Residuals: A Semiparametric,Copula‐Based Approach to Time Series Prediction

This paper proposes an adjustment of linear autoregressive conditional mean forecasts that exploits the predictive content of uncorrelated model residuals. The adjustment is motivated by non‐Gaussian characteristics of model residuals, and implemented in a semiparametric fashion by means of conditional moments of simulated bivariate distributions. A pseudo ex ante forecasting comparison is conducted for a set of 494 macroeconomic time series recently collected by Dees et al. (Journal of Applied Econometrics 2007; 22: 1–38). In total, 10,374 time series realizations are contrasted against competing short‐, medium‐ and longer‐term purely autoregressive and adjusted predictors. With regard to all forecast horizons, the adjusted predictions consistently outperform conditionally Gaussian forecasts according to cross‐sectional mean group evaluation of absolute forecast errors and directional accuracy. Copyright © 2012 John Wiley & Sons, Ltd.     

ARIMA models of the price level: An assessment of the multilevel adaptive learning process in the USA

This paper estimates the ARIMA processes for the observed and expected price level corresponding to the three-level adaptive expectations model proposed by Jacobs and Jones (1980). These univariate processes are then compared with the best-fit ARIMA model. The results indicate that the best-fit model for the observed price level is a restricted version of the two-level adaptive learning process specified in terms of prices, suggesting a simple adaptive rule in the inflation rate. A comparison of the time-series forecasts from the best-fit model with the mean responses to the ASA-NBER survey shows no significant difference in their accuracy. The time-series forecasts are, however, conditionally efficient. The best-fit ARIMA model for expected prices measured by the ASA-NBER consensus forecasts does not correspond to any version of the Jacobs and Jones model.     

N-step combinations of forecasts

While there is general agreement that a linear combination of forecasts can outperform the individual forecasts, there is controversy about the appropriateness of the combination method to be used in a given situation. Hence, in any given application it may be more beneficial to combine different sets of combined forecasts rather than picking one of them. This paper introduces the concept of N-step combinations of forecasts which involves combining the combined forecasts obtained from different combination procedures used at the preceding step. Using quarterly GNP data, evidence supporting the increase in the accuracy of the one-period-ahead ex-ante forecasts as the combination step increases is provided. The MSE, MAE, MAPE and their corresponding standard deviations are used to evaluate the accuracy of the forecasts obtained.