Sources of error in forecasts and expectations: U.K. economic models, 1984–8

This paper presents a comparative analysis of the sources of error in forecasts for the UK economy published over a recent four-year period by four independent groups. This analysis rests on the archiving at the ESRC Macroeconomic Modelling Bureau of the original forecasts together with all their accompanying assumptions and adjustments. A method of decomposing observed forecast errors so as to distinguish the contributions of forecaster and model is set out; the impact of future expectations treated in a ‘model-consistent’ or ‘rational’ manner is specifically considered. The results show that the forecaster's adjustments make a substantial contribution to forecast performance, a good part of which comes from adjustments that bring the model on track at the start of the forecast period. The published ex-ante forecasts are usually superior to pure model-based ex-post forecasts, whose performance indicates some misspecification of the underlying models.     

Factor forecasts for the UK

Data are now readily available for a very large number of macroeconomic variables that are potentially useful when forecasting. We argue that recent developments in the theory of dynamic factor models enable such large data sets to be summarized by relatively few estimated factors, which can then be used to improve forecast accuracy. In this paper we construct a large macroeconomic data set for the UK, with about 80 variables, model it using a dynamic factor model, and compare the resulting forecasts with those from a set of standard time‐series models. We find that just six factors are sufficient to explain 50% of the variability of all the variables in the data set. These factors, which can be shown to be related to key variables in the economy, and their use leads to considerable improvements upon standard time‐series benchmarks in terms of forecasting performance. Copyright © 2005 John Wiley & Sons, Ltd.     

Does Disagreement Amongst Forecasters Have Predictive Value?

In the present study we examine the predictive power of disagreement amongst forecasters. In our empirical work, we find that in some situations this variable can signal upcoming structural and temporal changes in an economic process and in the predictive power of the survey forecasts. We examine a variety of macroeconomic variables, and we use different measurements for the degree of disagreement, together with measures for location of the survey data and autoregressive components. Forecasts from simple linear models and forecasts from Markov regime‐switching models with constant and with time‐varying transition probabilities are constructed in real time and compared on forecast accuracy. Copyright © 2015 John Wiley & Sons, Ltd.     

Theory and practice of multivariate arma forecasting

We compare univariate and multivariate forecasts based on ARMA models. In theory we cannot do worse by using a multivariate model instead of a univariate one, but we can risk getting no improvement. Conditions for no improvements are discussed as well as cases where large improvements occur. The effect of estimated parameters is examined and found to be small granted that a good method of estimation is used. However, multivariate models could be very sensitive to structural changes. This is illustrated via an example involving monetary data, where the multivariate forecasts perform considerably worse than the univariate ones. This seems to put a limitation on the use of multivariate ARMA forecasting models.     

Unbiasedness,efficiency and the combination of economic forecasts

This paper considers the problem of determining whether forecasts are unbiased and examines the implications this has for combining different forecasts. The practical issues of how economic forecasts might be combined are discussed. There is an empirical illustration of the procedures in which the properties of UK forecasts from the London Business School, the National Institute, the Henley Centre for Forecasting, Phillips and Drew and the OECD are examined.     

Updating the forecast function of ARIMA models and the link with DLMs

This paper shows that the whole forecast function of ARIMA time series models, and not just the eventual forecast function, may be updated each time an observation is received. The paper also shows that the coefficients in the updating equations for the forecast function may be expressed in exactly the same form as the Kalman filter updating equations for canonical time series DLMs. Moreover, the adaptive factors in the updating equations are shown to be a simple function of the ARIMA model parameters. Copyright © 1999 John Wiley & Sons, Ltd.     

Merging monthly and quarterly forecasts: Experience with mqem

Forecasts from quarterly econometric models are typically revised on a monthly basis to reflect the information in current economic data. The revision process usually involves setting targets for the quarterly values of endogenous variables for which monthly observations are available and then altering the intercept terms in the quarterly forecasting model to achieve the target values. A formal statistical approach to the use of monthly data to update quarterly forecasts is described and the procedure is applied to the Michigan Quarterly Econometric Model of the US Economy. The procedure is evaluated in terms of both ex post and ex ante forecasting performance. The ex ante results for 1986 and 1987 indicate that the method is quite promising. With a few notable exceptions, the formal procedure produces forecasts of GNP growth that are very close to the published ex ante forecasts.     

Forecasting contemporaneous aggregates and the combination of forecasts: The case of the U.K. monetary aggregates

A number of papers in recent years have investigated the problems of forecasting contemporaneously aggregated time series and of combining alternative forecasts of a time series. This paper considers the integration of both approaches within the example of assessing the forecasting performance of models for two of the U.K. monetary aggregates, £M3 and MO. It is found that forecasts from a time series model for aggregate £M3 are superior to aggregated forecasts from individual models fitted to either the components or counterparts of £M3 and that an even better forecast is obtained by forming a linear combination of the three alternatives. For MO, however, aggregated forecasts from its components prove superior to either the forecast from the aggregate itself or from a linear combination of the two.     

ARIMA models of the price level: An assessment of the multilevel adaptive learning process in the USA

This paper estimates the ARIMA processes for the observed and expected price level corresponding to the three-level adaptive expectations model proposed by Jacobs and Jones (1980). These univariate processes are then compared with the best-fit ARIMA model. The results indicate that the best-fit model for the observed price level is a restricted version of the two-level adaptive learning process specified in terms of prices, suggesting a simple adaptive rule in the inflation rate. A comparison of the time-series forecasts from the best-fit model with the mean responses to the ASA-NBER survey shows no significant difference in their accuracy. The time-series forecasts are, however, conditionally efficient. The best-fit ARIMA model for expected prices measured by the ASA-NBER consensus forecasts does not correspond to any version of the Jacobs and Jones model.     

System-based weights versus series-specific weights in the combination of forecasts

System-based combination weights for series r/step-length h incorporate relative accuracy information from other forecast step-lengths for r and from other series for step-length h. Such weights are examined utilizing the West and Fullerton (1996) data set-4275 ex ante employment forecasts from structural simultaneous equation econometric models for 19 metropolitan areas at 10 quarterly step-lengths and a parallel set of 4275 ARIMA forecasts. The system-based weights yielded combined forecasts of higher average accuracy and lower risk of large inaccuracy than seven alternative strategies: (1) averaging; (2) relative MSE weights; (3) outperformance (per cent best) weights; (4) Bates and Granger (1969) optimal weights with a convexity constraint imposed; (5) unconstrained optimal weights; (6) select a ‘best’ method (ex ante) by series and; (7) experiment in the Bischoff (1989) sense and select either method (2) or (6) based on the outcome of e experiment. Accuracy gains of the system-based combination were concentrated at step-lengths two to five. Although alternative (5) was generally outperformed, none of the six other alternatives was systematically most accurate when evaluated relative to each other. This contrasts with Bischoff's (1989) results that held promise for an empirically applicable guideline to determine whether or not to combine.     

Forecasts of inflation from var models

Why are forecasts of inflation from VAR models so much worse than their forecasts of real variables? This paper documents that relatively poor performance, and finds that the price equation of a VAR model fitted to US post-war data is poorly specified. Statistical work by other authors has found that coefficients in such price equations may not be constant. Based on specific monetary actions, two changes in monetary policy regimes are proposed. Accounting for those two shifts yields significantly more accurate forecasts and lessens the evidence of misspecification.     

Comparing ex-ante forecasts from a sem and var model: An application to the italian economy

The predictive performance of a large-scale structural econometric model (SEM) of the Italian economy the Prometeia model is compared in this paper with a vector autoregressive (VAR) model estimated for a selection of six main variables of interest. The paper concentrates on the quarterly ex-ante forecasts of GDP growth rate and the annual forecasts of GDP growth and inflation rate, over the period 1980-85. It concludes that no forecaster is systematically better than the other. In particular, the VAR model outperforms the SEM in short-run forecasts, suggesting that, for the latter, more careful attention should be addressed to questions of dynamic specification. On the other hand, for longer intervals, the SEM forecasts are more accurate than the VAR forecasts, in that they can benefit from the judgemental interventions of the model users and the model can pick up the non-linearities of the economy which cannot be captured by the VAR. Given the different kinds of information that can be extracted from the two approaches, it seems more reasonable to consider them as complementary rather than alternative tools for modelling and forecasting. Therefore, rather than attempting to establish the superiority of one type of model over the other, this kind of comparisons should be seen as a useful diagnostic tool for detecting types of model misspecification.     

Forecast comparisons of four models of U.S. interest rates

In this paper we compare the out of sample forecasts from four alternative interest rate models based on expanding information sets. The random walk model is the most restrictive. The univariate time series model allows for a richer dynamic pattern and more conditioning information on own rates. The multivariate time series model permits a flexible dynamic pattern with own- and cross-series information. Finally, the forecasts from the MPS econometric model depend on the full model structure and information set. In theory, more information is preferred to less. In practice, complicated misspecified models can perform much worse than simple (also probably misspecified) models. For forecasts evaluated over the volatile 1970s the multivariate time series model forecasts are considerably better than those from simpler models which use less conditioning information, as well as forecasts from the MPS model which uses substantially more conditioning information but also imposes ‘structural’ economic restrictions.     

Conservatism and consensus-seeking among economic forecasters

This paper uses the track records of a panel of US economic forecasters participating in a consensus forecasting service to test for conservatism and consensus-seeking behaviour. The tests are based on a particular method-of-moments estimator, designed to allow for the heteroscedasticity and serial correlation which is inevitably present in errors from repeated forecasts for fixed target dates. Most forecasters prove to be conservative. When revising forecasts they give too much weight to their own past forecasts. Surprisingly, forecasters are not consensus-seeking but ‘variety-seeking’. When revising forecasts, they give too little weight to the known forecasts of other forecasters.     

Bayesian consensus forecasts of macroeconomic variables

Economists, like other forecasters, share knowledge, data and theories in common. Consequently, their forecast errors are likely to be highly dependent. This paper reports on an empirical study of 16 macroeconomic forecasters. Composite forecasts are computed using a sequential weighting scheme that takes dependence into account; these are compared to a simple average and median forecasts. A within-sample composite is also calculated. Both these methods perform significantly better than the average or median of the forecasts. This improvement in accuracy is apparently because the dependence between the forecasters' errors is so high that the optimal composite forecasts sometimes lie outside the range of the individual forecasts.     

The performance of non‐linear exchange rate models: a forecasting comparison

In recent years there has been a considerable development in modelling non‐linearities and asymmetries in economic and financial variables. The aim of the current paper is to compare the forecasting performance of different models for the returns of three of the most traded exchange rates in terms of the US dollar, namely the French franc (FF/$), the German mark (DM/$) and the Japanese yen (Y/$). The relative performance of non‐linear models of the SETAR, STAR and GARCH types is contrasted with their linear counterparts. The results show that if attention is restricted to mean square forecast errors, the performance of the models, when distinguishable, tends to favour the linear models. The forecast performance of the models is evaluated also conditional on the regime at the forecast origin and on density forecasts. This analysis produces more evidence of forecasting gains from non‐linear models. Copyright © 2002 John Wiley & Sons, Ltd.     

Forecasting from non-linear models in practice

If a simple non-linear autoregressive time-series model is suggested for a series, it is not straightforward to produce multi-step forecasts from it. Several alternative theoretical approaches are discussed and then compared with a simulation study only for the two-step case. It is suggested that fitting a new model for each forecast horizon may be a satisfactory strategy.     

Some practical aspects of forecasting in organizations

This paper presents the writer's experience, over a period of 25 years, in analysing organizational systems and, in particular, concentrates on the overall forecasting activity. The paper first looks at the relationship between forecasting and decision taking–with emphasis on the fact that forecasting is a means to aid decision taking and not an end in itself. It states that there are many types of forecasting problems, each requiring different methods of treatment. The paper then discusses attitudes which are emerging about the relative advantages of different forecasting techniques. It suggests a model building process which requires‘experience’and‘craftsmanship’, extensive practical application, frequent interaction between theory and practice and a methodology that eventually leads to models that contain no detectable inadequacies. Furthermore, it argues that although models which forecast a time series from its past history have a very important role to play, for effective policy making it is necessary to augment the model by introducing policy variables, again in a systematic not an ‘ad hoc’ manner. Finally, the paper discusses how forecasting systems can be introduced into the management process in the first place and how they should be monitored and updated when found wanting.     

A BVAR model for the connecticut economy

A Bayesian vector autoregressive (BVAR) model is developed for the Connecticut economy to forecast the unemployment rate, nonagricultural employment, real personal income, and housing permits authorized. The model includes both national and state variables. The Bayesian prior is selected on the basis of the accuracy of the out-of-sample forecasts. We find that a loose prior generally produces more accurate forecasts. The out-of-sample accuracy of the BVAR forecasts is also compared with that of forecasts from an unrestricted VAR model and of benchmark forecasts generated from univariate ARIMA models. The BVAR model generally produces the most accurate short- and long-term out-of-sample forecasts for 1988 through 1992. It also correctly predicts the direction of change.     

Real‐Time Forecasts of Inflation: The Role of Financial Variables

We present a mixed‐frequency model for daily forecasts of euro area inflation. The model combines a monthly index of core inflation with daily data from financial markets; estimates are carried out with the MIDAS regression approach. The forecasting ability of the model in real time is compared with that of standard VARs and of daily quotes of economic derivatives on euro area inflation. We find that the inclusion of daily variables helps to reduce forecast errors with respect to models that consider only monthly variables. The mixed‐frequency model also displays superior predictive performance with respect to forecasts solely based on economic derivatives. Copyright © 2012 John Wiley & Sons, Ltd.