Do experts' adjustments on model‐based SKU‐level forecasts improve forecast quality?

Model‐based SKU‐level forecasts are often adjusted by experts. In this paper we propose a statistical methodology to test whether these expert forecasts improve on model forecasts. Application of the methodology to a very large database concerning experts in 35 countries who adjust SKU‐level forecasts for pharmaceutical products in seven distinct categories leads to the general conclusion that expert forecasts are equally good at best, but are more often worse than model‐based forecasts. We explore whether this is due to experts putting too much weight on their contribution, and this indeed turns out to be the case. Copyright © 2009 John Wiley & Sons, Ltd.     

A comparative evaluation of objective and subjective weather forecasts in the united states

This paper is concerned primarily with the evaluation and comparison of objective and subjective weather forecasts. Operational forecasts of three weather elements are considered: (1) probability forecasts of precipitation occurrence, (2) categorical (i.e. non-probabilistic) forecasts of maximum and minimum temperatures and (3) categorical forecasts of cloud amount. The objective forecasts are prepared by numerical-statistical procedures, whereas the subjective forecasts are based on the judgements of individual forecasters. In formulating the latter, the forecasters consult information from a variety of sources, including the objective forecasts themselves. The precipitation probability forecasts are found to be both reliable and skilful, and evaluation of the temperature/cloud amount forecasts reveals that they are quite accurate/skilful. Comparison of the objective and subjective forecasts of precipitation occurrence indicates that the latter are generally more skilful than the former for shorter lead times (e.g. 12–24 hours), whereas the two types of forecasts are of approximately equal skill for longer lead times (e.g. 36–48 hours). Similar results are obtained for the maximum and minimum temperature forecasts. Objective cloud amount forecasts are more skilful than subjective cloud amount forecasts for all lead times. Examination of trends in performance over the last decade reveals that both types of forecasts for all three elements increased in skill (or accuracy) over the period, with improvements in objective forecasts equalling or exceeding improvements in subjective forecasts. The role and impact of the objective forecasts in the subjective weather forecasting process are discussed in some detail. The need to conduct controlled experiments and other studies of this process, with particular reference to the assimilation of information from different sources, is emphasized. Important characteristics of the forecasting system in meteorology are identified, and they are used to describe similarities and differences between weather forecasting and forecasting in other fields. Acquisition of some of these characteristics may be beneficial to other forecasting systems.     

Early Warning with Calibrated and Sharper Probabilistic Forecasts

Given a nonlinear model, a probabilistic forecast may be obtained by Monte Carlo simulations. At a given forecast horizon, Monte Carlo simulations yield sets of discrete forecasts, which can be converted to density forecasts. The resulting density forecasts will inevitably be downgraded by model misspecification. In order to enhance the quality of the density forecasts, one can mix them with the unconditional density. This paper examines the value of combining conditional density forecasts with the unconditional density. The findings have positive implications for issuing early warnings in different disciplines including economics and meteorology, but UK inflation forecasts are considered as an example. Copyright © 2012 John Wiley & Sons, Ltd.     

An empirical investigation of combinations of economic forecasts

This paper examines the effects of combining three econometric and three times-series forecasts of growth and inflation in the U.K. If forecasts are unbiased then a combination exploiting this fact will be more efficient than an unrestricted combination. Ex post econometric forecasts may be biased but ex ante they are unbiased. The results of the study are that a restricted linear combination of the econometric forecasts is superior to an unrestricted combination and also to the unweighted mean of the forecasts. However, it is not preferred to the best of the individual forecasts.     

A BVAR model for the connecticut economy

A Bayesian vector autoregressive (BVAR) model is developed for the Connecticut economy to forecast the unemployment rate, nonagricultural employment, real personal income, and housing permits authorized. The model includes both national and state variables. The Bayesian prior is selected on the basis of the accuracy of the out-of-sample forecasts. We find that a loose prior generally produces more accurate forecasts. The out-of-sample accuracy of the BVAR forecasts is also compared with that of forecasts from an unrestricted VAR model and of benchmark forecasts generated from univariate ARIMA models. The BVAR model generally produces the most accurate short- and long-term out-of-sample forecasts for 1988 through 1992. It also correctly predicts the direction of change.     

The Predictive Power of Survey‐Based Exchange Rate Forecasts: Is there a Role for Dispersion?

Previous research has shown that the consensus of individual exchange rate forecasts performs no better than many commonly used forecasting models in predicting future exchange rates. Studies on equity and bond markets have explored the effects of dispersion in forecasts on the predictive power of forecasts; however, no earlier paper has investigated such effects in the context of the foreign exchange market. This study explores the role of consensus forecast dispersion as a factor leading to bias and anchoring in exchange rate forecasts. Our analysis of five currency pairs reveals that consensus forecasts mostly appear to be unbiased predictors of exchange rates in the long run, but most are unable to pass tests for short‐run unbiasedness. In three of the five currencies examined it appears that forecasters should take greater account of reported forecast dispersion. Copyright © 2015 John Wiley & Sons, Ltd.     

Professional economic forecasts: Are they worth their costs?

Most economic forecast evaluations dating back 20 years show that professional forecasters add little to the forecasts generated by the simplest of models. Using various types of forecast error criteria, these evaluations usually conclude that the professional forecasts are little better than the no-change or ARIM A type forecast. It is our contention that this conclusion is mistaken because the conventional error criteria may not capture why forecasts are ma& or how they are used. Using forecast directional accuracy, the criterion which has been found to be highly correlated with profits in an interest rate setting, we find that professional GNP forecasts dominate the cheaper alternatives. Moreover, there appears to be no systematic relationship between this preferred criterion and the error measures used in previous studies.     

Information content of DSGE forecasts

This paper examines the question whether information is contained in forecasts from dynamic stochastic general equilibrium (DSGE) models beyond that contained in lagged values, which are extensively used in the models. Four sets of forecasts are examined. The results are encouraging for DSGE forecasts of real GDP. The results suggest that there is information in the DSGE forecasts not contained in forecasts based only on lagged values, and that there is no information in the lagged‐value forecasts not contained in the DSGE forecasts. The opposite is true for forecasts of the GDP deflator.     

ARIMA models of the price level: An assessment of the multilevel adaptive learning process in the USA

This paper estimates the ARIMA processes for the observed and expected price level corresponding to the three-level adaptive expectations model proposed by Jacobs and Jones (1980). These univariate processes are then compared with the best-fit ARIMA model. The results indicate that the best-fit model for the observed price level is a restricted version of the two-level adaptive learning process specified in terms of prices, suggesting a simple adaptive rule in the inflation rate. A comparison of the time-series forecasts from the best-fit model with the mean responses to the ASA-NBER survey shows no significant difference in their accuracy. The time-series forecasts are, however, conditionally efficient. The best-fit ARIMA model for expected prices measured by the ASA-NBER consensus forecasts does not correspond to any version of the Jacobs and Jones model.     

Testing for unreliable estimators and insignificant forecasts in combined forecasts

The reliability and precision of the weights used in combining individual forecasts, irrespective of the method of combination, is important in evaluating a combined forecast. The objective of this study is not to suggest the ‘best’ method of combining individual forecasts, but rather to propose exploratory procedures, that make use of all available sample information contained in the covariance matrix of individual forecast errors, to (1) detect if the weights used in combining forecasts are ‘reliable’ (and ‘stable’ if it is known that the covariance matrix of forecast errors is stationary over time) and (2) test for ‘insignificant’ individual forecasts used in forming a combined forecast. We present empirical applications using two-year sales and individual forecast data provided by a major consumer durables manufacturer to illustrate the feasibility of our proposed procedures.     

Collinearity and the use of latent root regression for combining GNP forecasts

In combining economic forecasts a problem often faced is that the individual forecasts display some degree of dependence. We discuss latent root regression for combining collinear GNP forecasts. Our results indicate that latent root regression produces more efficient combining weight estimates (regression parameter estimates) than ordinary least squares estimation (OLS), although out-of-sample forecasting performance is comparable to OLS.     

Conservatism and consensus-seeking among economic forecasters

This paper uses the track records of a panel of US economic forecasters participating in a consensus forecasting service to test for conservatism and consensus-seeking behaviour. The tests are based on a particular method-of-moments estimator, designed to allow for the heteroscedasticity and serial correlation which is inevitably present in errors from repeated forecasts for fixed target dates. Most forecasters prove to be conservative. When revising forecasts they give too much weight to their own past forecasts. Surprisingly, forecasters are not consensus-seeking but ‘variety-seeking’. When revising forecasts, they give too little weight to the known forecasts of other forecasters.     

Comparing ex-ante forecasts from a sem and var model: An application to the italian economy

The predictive performance of a large-scale structural econometric model (SEM) of the Italian economy the Prometeia model is compared in this paper with a vector autoregressive (VAR) model estimated for a selection of six main variables of interest. The paper concentrates on the quarterly ex-ante forecasts of GDP growth rate and the annual forecasts of GDP growth and inflation rate, over the period 1980-85. It concludes that no forecaster is systematically better than the other. In particular, the VAR model outperforms the SEM in short-run forecasts, suggesting that, for the latter, more careful attention should be addressed to questions of dynamic specification. On the other hand, for longer intervals, the SEM forecasts are more accurate than the VAR forecasts, in that they can benefit from the judgemental interventions of the model users and the model can pick up the non-linearities of the economy which cannot be captured by the VAR. Given the different kinds of information that can be extracted from the two approaches, it seems more reasonable to consider them as complementary rather than alternative tools for modelling and forecasting. Therefore, rather than attempting to establish the superiority of one type of model over the other, this kind of comparisons should be seen as a useful diagnostic tool for detecting types of model misspecification.     

Are economic forecasts valuable?

A previous study questioned whether the published forecasts of one forecasting organization were valuable to users. A forecast was considered valuable if it differed significantly from a naive model in the sense of predicting the direction of change. The results indicated that it was not possible to show that the one-quarter-ahead predictions were valuable. However, the current-quarter forecasts were valuable. This paper examines whether the results are robust. The forecasts of two additional organizations were examined and the time period was extended. The results are generally robust.     

Relative accuracy of judgemental and extrapolative methods in forecasting annual earnings

This paper identifies and analyses previously published studies on annual earnings forecasts. Comparisons of forecasts produced by management, analysts, and extrapolative techniques indicated that: (1) management forecasts were superior to professional analyst forecasts (the mean absolute percentage errors were 15.9 and 17.7, respectively, based on five studies using data from 1967–1974) and (2) judgemental forecasts (both management and analysts) were superior to extrapolation forecasts on 14 of 17 comparisons from 13 studies using data from 1964–1979 (the mean absolute percentage errors were 21.0 and 28.4 for judgement and extrapolation, respectively). These conclusions, based on recent research, differ from those reported in previous reviews, which commented on less than half of the studies identified here.     

Combining expert‐adjusted forecasts

It is well known that a combination of model‐based forecasts can improve upon each of the individual constituent forecasts. Most forecasts available in practice are, however, not purely based on econometric models but entail adjustments, where experts with domain‐specific knowledge modify the original model forecasts. There is much evidence that expert‐adjusted forecasts do not necessarily improve the pure model‐based forecasts. In this paper we show, however, that combined expert‐adjusted model forecasts can improve on combined model forecasts, in the case when the individual expert‐adjusted forecasts are not better than their associated model‐based forecasts. We discuss various implications of this finding.     

Professional forecasters' expectations,consistency, and international spillovers

This paper focuses on the expectation formation process of professional forecasters by relying on survey data on forecasts regarding gross domestic product growth, consumer price index inflation and 3-month interest rates for a broad set of countries. We examine the interrelation between macroeconomic forecasts and also the impact of uncertainty on forecasts by allowing for cross-country interdependencies and time variation in the coefficients. We find that professional forecasts are often in line with the Taylor rule and identify significant expectation spillovers from monetary policy in the USA.     

The uses and abuses of ‘consensus’ forecasts

Clemen's (1989) review of the forecast-combining literature amply illustrates both the interest in and the importance of this subject. This article stresses the tautological properties of various consensus measures that assure their success relative to most individual forecasts. It confirms the finding of earlier studies that for each specific macroeconomic variable roughly one-third of individual forecasters are more accurate than a consensus. However, each individual does relatively poorly for some variable while the consensus, in contrast, necessarily never fails relative to most individuals. These results, like most previous studies, describe consensus measures that are synthetic constructs derived from a pre-existing set of individual forecasts. Strictly speaking, this contemporaneous consensus is not available to individual forecasters when their forecasts are made. A prior consensus measure, which is in their information sets, was relatively much less accurate than the contemporaneous measure. Nevertheless, a small subset of individual forecasters were generally inferior to the known, prior consensus forecast.     

Estimation and Prediction Tests of Cash Flow Forecast Accuracy

Recently, analysts' cash flow forecasts have become widely available through financial information services. Cash flow information enables practitioners to better understand the real operating performance and financial stability of a company, particularly when earnings information is noisy and of low quality. However, research suggests that analysts' cash flow forecasts are less accurate and more dispersed than earnings forecasts. We thus investigate factors influencing cash flow forecast accuracy and build a practical model to distinguish more accurate from less accurate cash flow forecasters, using past cash flow forecast accuracy and analyst characteristics. We find significant power in our cash flow forecast accuracy prediction models. We also find that analysts develop cash flow‐specific forecasting expertise and knowhow, which are distinct from those that analysts acquire from forecasting earnings. In particular, cash flow‐specific information is more useful in identifying accurate cash flow forecasters than earnings‐specific information.Copyright © 2011 John Wiley & Sons, Ltd.     

Linear combination of forecasts with an intercept: A bayesian approach

The standard approach to combining n expert forecasts involves taking a weighted average. Granger and Ramanathan proposed introducing an intercept term and unnormalized weights. This paper deduces their proposal from Bayesian principles. We find that their formula is equivalent to taking a weighted average of the n expert forecasts plus the decision-maker's prior forecast.