Judgemental revision of sales forecasts: A longitudinal eetension

When quantitative models are used for short-term multi-item sales forecasts it is possible that the managers who use such forecasts may disagree with at least some of the estimates obtained, and wish to change them so that they become more consistent with their own (subjective) evaluation of the marketplace. This study reports on an analysis of the effectiveness of judgemental revision of sales forecasts over six quarterly forecasting periods. The results give general support for the practice of forecast manipulation as a means of improving forecasting accuracy. It is also observed that the effectiveness of revision activity varies across different time periods.     

Judgemental revision of sales forecasts: Effectiveness of forecast selection

When managers make revisions to sales forecasts initially generated by a rational quantitative model it is important that the particular forecasts selected for adjustment are those which would benefit most from the adjustment process (i.e. realize high errors). This study reports an empirical investigation on this issue, spanning six quarterly forecasting periods and incorporating forecasting data on over 850 products. The results show that the errors of the forecasts chosen for revision are, in general, higher than those which were not chosen. In addition, it is shown that managesrs tend to revise forecasts which are initially low, hence possibily introducing some degree of bias into the overall forecasts.     

Merging monthly and quarterly forecasts: Experience with mqem

Forecasts from quarterly econometric models are typically revised on a monthly basis to reflect the information in current economic data. The revision process usually involves setting targets for the quarterly values of endogenous variables for which monthly observations are available and then altering the intercept terms in the quarterly forecasting model to achieve the target values. A formal statistical approach to the use of monthly data to update quarterly forecasts is described and the procedure is applied to the Michigan Quarterly Econometric Model of the US Economy. The procedure is evaluated in terms of both ex post and ex ante forecasting performance. The ex ante results for 1986 and 1987 indicate that the method is quite promising. With a few notable exceptions, the formal procedure produces forecasts of GNP growth that are very close to the published ex ante forecasts.     

Evaluating forecasts: a look at aggregate bias and accuracy measures

In this paper an investigation is made of the properties and use of two aggregate measures of forecast bias and accuracy. These are metrics used in business to calculate aggregate forecasting performance for a family (group) of products. We find that the aggregate measures are not particularly informative if some of the one‐step‐ahead forecasts are biased. This is likely to be the case in practice if frequently employed forecasting methods are used to generate a large number of individual forecasts. In the paper, examples are constructed to illustrate some potential problems in the use of the metrics. We propose a simple graphical display of forecast bias and accuracy to supplement the information yielded by the accuracy measures. This support includes relevant boxplots of measures of individual forecasting success. This tool is simple but helpful as the graphic display has the potential to indicate forecast deterioration that can be masked by one or both of the aggregate metrics. The procedures are illustrated with data representing sales of food items. Copyright © 2005 John Wiley & Sons, Ltd.     

N-step combinations of forecasts

While there is general agreement that a linear combination of forecasts can outperform the individual forecasts, there is controversy about the appropriateness of the combination method to be used in a given situation. Hence, in any given application it may be more beneficial to combine different sets of combined forecasts rather than picking one of them. This paper introduces the concept of N-step combinations of forecasts which involves combining the combined forecasts obtained from different combination procedures used at the preceding step. Using quarterly GNP data, evidence supporting the increase in the accuracy of the one-period-ahead ex-ante forecasts as the combination step increases is provided. The MSE, MAE, MAPE and their corresponding standard deviations are used to evaluate the accuracy of the forecasts obtained.     

On the usefulness of macroeconomic forecasts as inputs to forecasting models

A forecasting model for y_t based on its relationship to exogenous variables (e.g. x?_t) must use x?_t, the forecast of x?_t. An example is given where commercially available x?_t's are sufficiently inaccurate that a univariate model for y_t appears preferable. For a variety of types of models inclusion of an exogenous variable x?_t is shown to worsen the y_t forecasts whenever x?_t must itself be forecast by x?_t and MSE (x?_t) > Var (x?_t). Tests with forecasts from a variety of sources indicate that, with a few notable exceptions, MSE (x?_t) > Var (x?_t) is common for macroeconomic forecasts more than a quarter or two ahead. Thus, either:

• (a) available medium range forecasts for many macroeconomic variables (e.g. the GNP growth rate) are not an improvement over the sample mean (so that such variables are not useful explanatory variables in forecasting models), and/or
• (b) the suboptimization involved in directly replacing x?_t by x?_t is a luxury that we cannot afford.

     

The combination of macroeconomic forecasts

The paper examines combined forecasts based on two components: forecasts produced by Chase Econometrics and those produced using the Box-Jenkins ARIMA technique. Six series of quarterly ex ante and simulated ex ante forecasts are used over 37 time periods and ten horizons. The forecasts are combined using seven different methods. The best combined forecasts, judged by average relative root-mean-square error, are superior to the Chase forecasts for three variables and inferior for two, though averaged over all six variables the Chase forecasts are slightly better. A two-step procedure produces forecasts for the last half of the sample which, on average, are slightly better than the Chase forecasts.     

Nonlinear identification of judgmental forecasts effects at SKU level

Prediction of demand is a key component within supply chain management. Improved accuracy in forecasts directly affects all levels of the supply chain, reducing stock costs and increasing customer satisfaction. In many application areas, demand prediction relies on statistical software which provides an initial forecast subsequently modified by the expert's judgment. This paper outlines a new methodology based on state‐dependent parameter (SDP) estimation techniques to identify the nonlinear behaviour of such managerial adjustments. This non‐parametric SDP estimate is used as a guideline to propose a nonlinear model that corrects the bias introduced by the managerial adjustments. One‐step‐ahead forecasts of stock‐keeping unit sales sampled monthly from a manufacturing company are utilized to test the proposed methodology. The results indicate that adjustments introduce a nonlinear pattern, undermining accuracy. This understanding can be used to enhance the design of the forecasting support system in order to help forecasters towards more efficient judgmental adjustments. Copyright © 2010 John Wiley & Sons, Ltd.     

Daily volatility forecasts: reassessing the performance of GARCH models

Volatility plays a key role in asset and portfolio management and derivatives pricing. As such, accurate measures and good forecasts of volatility are crucial for the implementation and evaluation of asset and derivative pricing models in addition to trading and hedging strategies. However, whilst GARCH models are able to capture the observed clustering effect in asset price volatility in‐sample, they appear to provide relatively poor out‐of‐sample forecasts. Recent research has suggested that this relative failure of GARCH models arises not from a failure of the model but a failure to specify correctly the ‘true volatility’ measure against which forecasting performance is measured. It is argued that the standard approach of using ex post daily squared returns as the measure of ‘true volatility’ includes a large noisy component. An alternative measure for ‘true volatility’ has therefore been suggested, based upon the cumulative squared returns from intra‐day data. This paper implements that technique and reports that, in a dataset of 17 daily exchange rate series, the GARCH model outperforms smoothing and moving average techniques which have been previously identified as providing superior volatility forecasts. Copyright © 2004 John Wiley & Sons, Ltd.     

Bayesian forecasts for cointegrated models

This paper investigates Bayesian forecasts for some cointegrated time series data. Suppose data are derived from some cointegrated model, but, an unrestricted vector autoregressive model, without including cointegrated conditions, is fitted; the implication of using an incorrect model will be investigated from the Bayesian forecasting viewpoint. For some special cointegrated data and under the diffuse prior assumption, it can be analytically proven that the posterior predictive distributions for both the true model and the fitted model are asymptotically the same for any future step. For a more general cointegrated model, examinations are performed via simulations. Some simulated results reveal that a reasonably unrestricted model will still provide a rather accurate forecast as long as the sample size is large enough or the forecasting period is not too far in the future. For a small sample size or for long‐term forecasting, more accurate forecasts are expected if the correct cointegrated model is actually applied. Copyright © 2002 John Wiley & Sons, Ltd.     

Combination of forecasts in electricity demand prediction

Accurate demand prediction is of great importance in the electricity supply industry. Electricity cannot be stored, and generating plant must be scheduled well in advance to meet future demand. Up to now, where online information about external conditions is unavailable, time series methods on the historical demand series have been used for short-term demand prediction. These have drawbacks, both in their sensitivity to changing weather conditions and in their poor modelling of the daily/weekly business cycles. To overcome these problems a framework has been constructed whereby forecasts from different prediction methods and different forecasting origins can be selected and combined, solely on the basis of recent forecasting performance, with no a priori assumptions of demand behaviour. This added flexibility in univariate forecasting provides a significant improvement in prediction accuracy.     

Choosing among competing econometric forecasts: Regression‐based forecast combination using model selection

Forecast combination based on a model selection approach is discussed and evaluated. In addition, a combination approach based on ex ante predictive ability is outlined. The model selection approach which we examine is based on the use of Schwarz (SIC) or the Akaike (AIC) Information Criteria. Monte Carlo experiments based on combination forecasts constructed using possibly (misspecified) models suggest that the SIC offers a potentially useful combination approach, and that further investigation is warranted. For example, combination forecasts from a simple averaging approach MSE‐dominate SIC combination forecasts less than 25% of the time in most cases, while other ‘standard’ combination approaches fare even worse. Alternative combination approaches are also compared by conducting forecasting experiments using nine US macroeconomic variables. In particular, artificial neural networks (ANN), linear models, and professional forecasts are used to form real‐time forecasts of the variables, and it is shown via a series of experiments that SIC, t‐statistic, and averaging combination approaches dominate various other combination approaches. An additional finding is that while ANN models may not MSE‐dominate simpler linear models, combinations of forecasts from these two models outperform either individual forecast, for a subset of the economic variables examined. Copyright © 2001 John Wiley & Sons, Ltd.     

An empirical investigation of combinations of economic forecasts

This paper examines the effects of combining three econometric and three times-series forecasts of growth and inflation in the U.K. If forecasts are unbiased then a combination exploiting this fact will be more efficient than an unrestricted combination. Ex post econometric forecasts may be biased but ex ante they are unbiased. The results of the study are that a restricted linear combination of the econometric forecasts is superior to an unrestricted combination and also to the unweighted mean of the forecasts. However, it is not preferred to the best of the individual forecasts.     

Assessing the historical accuracy of regional economic forecasts

This study addresses for the first time systematic evaluation of a widely used class of forecasts, regional economic forecasts. Ex ante regional structural equation model forecasts are analysed for 19 metropolitan areas. One- to ten-quarter-ahead forecasts are considered and the seven-year sample spans a complete business cycle. Counter to previous speculation in the literature, (1) dependency on macroeconomic forecasting model inputs does not substantially erode accuracy relative to univariate extrapolative methodologies and (2) stochastic time series models do not on average, yield more accurate regional economic predictions than structural models. Similar to findings in other studies, clear preferences among extrapolative methodologies do not emerge. Most general conclusions, however, are subject to caveats based on step-length effects and region-specific effects.     

Judgmental adjustment of earnings forecasts

Earnings forecasts have received a great deal of attention, much of which has centered on the comparative accuracy of judgmental and objective forecasting methods. Recently, studies have focused on the use of combinations of subjective and objective forecasts to improve forecast accuracy. This research offers an extension on this theme by subjectively modifying an objective forecast. Specifically, ARIMA forecasts are judgmentally adjusted by analysts using a structured approach based on Saaty's (1980) analytic hierarchy process. The results show that the accuracy of the unadjusted objective forecasts can be improved when judgmentally adjusted.     

Estimating the strength of expert judgement: The case of US mortality forecasts

The use of expert judgement is an important part of demographic forecasting. However, because judgement enters into the forecasting process in an informal way, it has been very difficult to assess its role relative to the analysis of past data. The use of targets in demographic forecasts permits us to embed the subjective forecasting process into a simple time-series regression model, in which expert judgement is incorporated via mixed estimation. The strength of expert judgement is denned, and estimated using the official forecasts of cause-specific mortality in the United States. We show that the weight given to judgement varies in an improbable manner by age. Overall, the weight given to judgement appears too high. An alternative approach to combining expert judgement and past data is suggested.     

Testing for unreliable estimators and insignificant forecasts in combined forecasts

The reliability and precision of the weights used in combining individual forecasts, irrespective of the method of combination, is important in evaluating a combined forecast. The objective of this study is not to suggest the ‘best’ method of combining individual forecasts, but rather to propose exploratory procedures, that make use of all available sample information contained in the covariance matrix of individual forecast errors, to (1) detect if the weights used in combining forecasts are ‘reliable’ (and ‘stable’ if it is known that the covariance matrix of forecast errors is stationary over time) and (2) test for ‘insignificant’ individual forecasts used in forming a combined forecast. We present empirical applications using two-year sales and individual forecast data provided by a major consumer durables manufacturer to illustrate the feasibility of our proposed procedures.     

Professional economic forecasts: Are they worth their costs?

Most economic forecast evaluations dating back 20 years show that professional forecasters add little to the forecasts generated by the simplest of models. Using various types of forecast error criteria, these evaluations usually conclude that the professional forecasts are little better than the no-change or ARIM A type forecast. It is our contention that this conclusion is mistaken because the conventional error criteria may not capture why forecasts are ma& or how they are used. Using forecast directional accuracy, the criterion which has been found to be highly correlated with profits in an interest rate setting, we find that professional GNP forecasts dominate the cheaper alternatives. Moreover, there appears to be no systematic relationship between this preferred criterion and the error measures used in previous studies.     

Algorithms for explaining forecast revisions

Forecasts are routinely revised, and these revisions are often the subject of informal analysis and discussion. This paper argues (1) that forecast revisions are analyzed because they help forecasters and forecast users to evaluate forecasts and forecasting procedures and (2) that these analyses can be sharpened by using the forecasting model to systematically express its forecast revision as the sum of components identified with specific subsets of new information, such as data revisions and forecast errors. An algorithm for this purpose is explained and illustrated.