Bayesian consensus forecasts of macroeconomic variables

Economists, like other forecasters, share knowledge, data and theories in common. Consequently, their forecast errors are likely to be highly dependent. This paper reports on an empirical study of 16 macroeconomic forecasters. Composite forecasts are computed using a sequential weighting scheme that takes dependence into account; these are compared to a simple average and median forecasts. A within-sample composite is also calculated. Both these methods perform significantly better than the average or median of the forecasts. This improvement in accuracy is apparently because the dependence between the forecasters' errors is so high that the optimal composite forecasts sometimes lie outside the range of the individual forecasts.     

The combination of macroeconomic forecasts

The paper examines combined forecasts based on two components: forecasts produced by Chase Econometrics and those produced using the Box-Jenkins ARIMA technique. Six series of quarterly ex ante and simulated ex ante forecasts are used over 37 time periods and ten horizons. The forecasts are combined using seven different methods. The best combined forecasts, judged by average relative root-mean-square error, are superior to the Chase forecasts for three variables and inferior for two, though averaged over all six variables the Chase forecasts are slightly better. A two-step procedure produces forecasts for the last half of the sample which, on average, are slightly better than the Chase forecasts.     

The accuracy of individual and group forecasts from business outlook surveys

This paper reports on the accuracy of quarterly multiperiod predictions of inflation, real growth, unemployment and percentage changes in nominal GNP and two of its more volatile components. The survey data are highly differentiated; they cover 79 professional forecasters (mostly economists, analysts and corporate executives). Combining corresponding predictions from different sources can result in significant gains; thus the group mean forecasts are on the average over time more accurate than most of the corresponding sets of individual forecasts. But there is also a moderate degree of consistency in the relative performance of a sufficient number of the survey members, as evidenced in positive rank correlations among ratios of the individual to group root mean square errors.     

A test of the joint efficiency of macroeconomic forecasts using multivariate random forests

We contribute to recent research on the joint evaluation of the properties of macroeconomic forecasts in a multivariate setting. The specific property of forecasts that we are interested in is their joint efficiency. We study the joint efficiency of forecasts by means of multivariate random forests, which we use to model the links between forecast errors and predictor variables in a forecaster's information set. We then use permutation tests to study whether the Mahalanobis distance between the predicted forecast errors for the growth and inflation forecasts of four leading German economic research institutes and actual forecast errors is significantly smaller than under the null hypothesis of forecast efficiency. We reject joint efficiency in several cases, but also document heterogeneity across research institutes with regard to the joint efficiency of their forecasts.     

Unbiasedness,efficiency and the combination of economic forecasts

This paper considers the problem of determining whether forecasts are unbiased and examines the implications this has for combining different forecasts. The practical issues of how economic forecasts might be combined are discussed. There is an empirical illustration of the procedures in which the properties of UK forecasts from the London Business School, the National Institute, the Henley Centre for Forecasting, Phillips and Drew and the OECD are examined.     

The uses and abuses of ‘consensus’ forecasts

Clemen's (1989) review of the forecast-combining literature amply illustrates both the interest in and the importance of this subject. This article stresses the tautological properties of various consensus measures that assure their success relative to most individual forecasts. It confirms the finding of earlier studies that for each specific macroeconomic variable roughly one-third of individual forecasters are more accurate than a consensus. However, each individual does relatively poorly for some variable while the consensus, in contrast, necessarily never fails relative to most individuals. These results, like most previous studies, describe consensus measures that are synthetic constructs derived from a pre-existing set of individual forecasts. Strictly speaking, this contemporaneous consensus is not available to individual forecasters when their forecasts are made. A prior consensus measure, which is in their information sets, was relatively much less accurate than the contemporaneous measure. Nevertheless, a small subset of individual forecasters were generally inferior to the known, prior consensus forecast.     

Linear constraints and the efficiency of combined forecasts

Studies of combined forecasts have typically constrained the combining weights to sum to one and have not included a constant term in the combination. In a recent paper, Granger and Ramanathan (1984) have argued in favour of an unrestricted linear combination, including a constant term. This paper shows that for the purpose of prediction it may make sense to impose restrictions on the combining model because of potential increases in forecasting efficiency. Empirical results show that small gains in forecasting efficiency can be obtained by restricting the linear combination of GNP forecasts from four econometric models.     

System-based weights versus series-specific weights in the combination of forecasts

System-based combination weights for series r/step-length h incorporate relative accuracy information from other forecast step-lengths for r and from other series for step-length h. Such weights are examined utilizing the West and Fullerton (1996) data set-4275 ex ante employment forecasts from structural simultaneous equation econometric models for 19 metropolitan areas at 10 quarterly step-lengths and a parallel set of 4275 ARIMA forecasts. The system-based weights yielded combined forecasts of higher average accuracy and lower risk of large inaccuracy than seven alternative strategies: (1) averaging; (2) relative MSE weights; (3) outperformance (per cent best) weights; (4) Bates and Granger (1969) optimal weights with a convexity constraint imposed; (5) unconstrained optimal weights; (6) select a ‘best’ method (ex ante) by series and; (7) experiment in the Bischoff (1989) sense and select either method (2) or (6) based on the outcome of e experiment. Accuracy gains of the system-based combination were concentrated at step-lengths two to five. Although alternative (5) was generally outperformed, none of the six other alternatives was systematically most accurate when evaluated relative to each other. This contrasts with Bischoff's (1989) results that held promise for an empirically applicable guideline to determine whether or not to combine.     

Collinearity and the use of latent root regression for combining GNP forecasts

In combining economic forecasts a problem often faced is that the individual forecasts display some degree of dependence. We discuss latent root regression for combining collinear GNP forecasts. Our results indicate that latent root regression produces more efficient combining weight estimates (regression parameter estimates) than ordinary least squares estimation (OLS), although out-of-sample forecasting performance is comparable to OLS.     

A comparative evaluation of objective and subjective weather forecasts in the united states

This paper is concerned primarily with the evaluation and comparison of objective and subjective weather forecasts. Operational forecasts of three weather elements are considered: (1) probability forecasts of precipitation occurrence, (2) categorical (i.e. non-probabilistic) forecasts of maximum and minimum temperatures and (3) categorical forecasts of cloud amount. The objective forecasts are prepared by numerical-statistical procedures, whereas the subjective forecasts are based on the judgements of individual forecasters. In formulating the latter, the forecasters consult information from a variety of sources, including the objective forecasts themselves. The precipitation probability forecasts are found to be both reliable and skilful, and evaluation of the temperature/cloud amount forecasts reveals that they are quite accurate/skilful. Comparison of the objective and subjective forecasts of precipitation occurrence indicates that the latter are generally more skilful than the former for shorter lead times (e.g. 12–24 hours), whereas the two types of forecasts are of approximately equal skill for longer lead times (e.g. 36–48 hours). Similar results are obtained for the maximum and minimum temperature forecasts. Objective cloud amount forecasts are more skilful than subjective cloud amount forecasts for all lead times. Examination of trends in performance over the last decade reveals that both types of forecasts for all three elements increased in skill (or accuracy) over the period, with improvements in objective forecasts equalling or exceeding improvements in subjective forecasts. The role and impact of the objective forecasts in the subjective weather forecasting process are discussed in some detail. The need to conduct controlled experiments and other studies of this process, with particular reference to the assimilation of information from different sources, is emphasized. Important characteristics of the forecasting system in meteorology are identified, and they are used to describe similarities and differences between weather forecasting and forecasting in other fields. Acquisition of some of these characteristics may be beneficial to other forecasting systems.     

Experience in combining subjective and quantitative forecasts of open market rates

This paper presents a method of combining subjective information from open-market operators with results from a time-series forecasting model. Empirical results of forecasts for interest rates of bank reserves are presented.     

Linear constraints and the efficiency of combined forecasts

This note extends some recent results, achieved by Clemen, on constraining the weights of a combined forecast. There is a great potential for improving the ordinary least squares forecast by imposing linear restrictions, and it will be shown how this potential can be exhausted by using an F-test. The corresponding decision procedure leads to a pre-test forecast with good statistical properties.     

Evaluating probability forecasts in terms of refinement and strictly proper scoring rules

This note gives an easily verified necessary and sufficient condition for one probability forecaster to empirically outperform another one in terms of all strictly proper scoring rules. Copyright © 2006 John Wiley & Sons, Ltd.     

Monitoring and adjusting forecasts in the presence of additive outliers

The effect of an additive outlier upon the accuracy of forecasts derived from extrapolative methods is investigated. It is demonstrated that an outlier affects not only the accuracy of the forecasts at the time of occurrence but also subsequent forecasts. Methods to adjust for additive outliers are discussed. The results of the paper are illustrated with two examples.     

Linear combination of restrictions and forecasts in time series analysis

An important tool in time series analysis is that of combining information in an optimal way. Here we establish a basic combining rule of linear predictors and show that such problems as forecast updating, missing value estimation, restricted forecasting with binding constraints, analysis of outliers and temporal disaggregation can be viewed as problems of optimal linear combination of restrictions and forecasts. A compatibility test statistic is also provided as a companion tool to check that the linear restrictions are compatible with the forecasts generated from the historical data. Copyright © 2000 John Wiley & Sons, Ltd.     

Medium‐term forecasts of potential GDP and inflation using age structure information

Economic behaviour as well as economic resources of individuals vary with age. Swedish time series show that the age structure contains information correlated to medium‐term trends in growth and inflation. GDP gaps estimated by age structure regressions are closely related to conventional measures. Monetary policy is believed to affect inflation with a lag of 1 or 2 years. Projections of the population's age structure are comparatively reliable several years ahead and provide additional information to improve on 3–5 years‐ahead forecasts of potential GDP and inflation. Thus there is a potential scope for using age structure based forecasts as an aid to monetary policy formation. Copyright © 2004 John Wiley & Sons, Ltd.     

Portfolio managers' and novices' forecasts of risk and return: are there predictable forecast errors?

This study aims to investigate the individual behaviour that underlies the overreaction hypothesis by conducting a controlled experiment. Two areas that were not captured by previous research on the validity of the overreaction hypothesis are investigated. First, actual portfolio managers are employed as forecasters. Second a real‐world assessment task is given in the form of predicting the prices of stocks traded on the exchange on a real time basis. The purpose is to explore return expectations and risk perceptions of portfolio managers as well as financially unsophisticated investors by using point and interval forecasts provided for different forecast horizons in bull and bear markets. Contributions stem from three sources. (1) The use of financially sophisticated subjects for the first time in an experimental framework testing the overreaction hypothesis makes possible to control for the effect of expertise. (2) The use of different forecast horizons controls for the effect of forecast period. (3) The use of real‐time forecasts of specific stocks traded at the stock exchange, for the first time in an experimental framework testing the overreaction hypothesis enables to control for ecological validity. Discussions will be given as to the portfolio managers' versus naive investors' interpolating asset prices from past trends and hedging behaviour, due to their caution in projections of ranges for future prices. Copyright © 2002 John Wiley & Sons, Ltd.     

Sources of error in forecasts and expectations: U.K. economic models, 1984–8

This paper presents a comparative analysis of the sources of error in forecasts for the UK economy published over a recent four-year period by four independent groups. This analysis rests on the archiving at the ESRC Macroeconomic Modelling Bureau of the original forecasts together with all their accompanying assumptions and adjustments. A method of decomposing observed forecast errors so as to distinguish the contributions of forecaster and model is set out; the impact of future expectations treated in a ‘model-consistent’ or ‘rational’ manner is specifically considered. The results show that the forecaster's adjustments make a substantial contribution to forecast performance, a good part of which comes from adjustments that bring the model on track at the start of the forecast period. The published ex-ante forecasts are usually superior to pure model-based ex-post forecasts, whose performance indicates some misspecification of the underlying models.     

Forecasting contemporaneous aggregates and the combination of forecasts: The case of the U.K. monetary aggregates

A number of papers in recent years have investigated the problems of forecasting contemporaneously aggregated time series and of combining alternative forecasts of a time series. This paper considers the integration of both approaches within the example of assessing the forecasting performance of models for two of the U.K. monetary aggregates, £M3 and MO. It is found that forecasts from a time series model for aggregate £M3 are superior to aggregated forecasts from individual models fitted to either the components or counterparts of £M3 and that an even better forecast is obtained by forming a linear combination of the three alternatives. For MO, however, aggregated forecasts from its components prove superior to either the forecast from the aggregate itself or from a linear combination of the two.     

Sports forecasting: a comparison of the forecast accuracy of prediction markets,betting odds and tipsters

This article compares the forecast accuracy of different methods, namely prediction markets, tipsters and betting odds, and assesses the ability of prediction markets and tipsters to generate profits systematically in a betting market. We present the results of an empirical study that uses data from 678–837 games of three seasons of the German premier soccer league. Prediction markets and betting odds perform equally well in terms of forecasting accuracy, but both methods strongly outperform tipsters. A weighting‐based combination of the forecasts of these methods leads to a slightly higher forecast accuracy, whereas a rule‐based combination improves forecast accuracy substantially. However, none of the forecasts leads to systematic monetary gains in betting markets because of the high fees (25%) charged by the state‐owned bookmaker in Germany. Lower fees (e.g., approximately 12% or 0%) would provide systematic profits if punters exploited the information from prediction markets and bet only on a selected number of games. Copyright © 2008 John Wiley & Sons, Ltd.