Judgemental revision of sales forecasts: A longitudinal eetension

When quantitative models are used for short-term multi-item sales forecasts it is possible that the managers who use such forecasts may disagree with at least some of the estimates obtained, and wish to change them so that they become more consistent with their own (subjective) evaluation of the marketplace. This study reports on an analysis of the effectiveness of judgemental revision of sales forecasts over six quarterly forecasting periods. The results give general support for the practice of forecast manipulation as a means of improving forecasting accuracy. It is also observed that the effectiveness of revision activity varies across different time periods.     

Judgemental revision of sales forecasts: Effectiveness of forecast selection

When managers make revisions to sales forecasts initially generated by a rational quantitative model it is important that the particular forecasts selected for adjustment are those which would benefit most from the adjustment process (i.e. realize high errors). This study reports an empirical investigation on this issue, spanning six quarterly forecasting periods and incorporating forecasting data on over 850 products. The results show that the errors of the forecasts chosen for revision are, in general, higher than those which were not chosen. In addition, it is shown that managesrs tend to revise forecasts which are initially low, hence possibily introducing some degree of bias into the overall forecasts.     

The use of non-time series information in sales forecasting: A case study

The contribution of product and industry knowledge to the accuracy of sales forecasting was investigated by examining the company forecasts of a leading manufacturer and marketer of consumable products. The company forecasts of 18 products produced by a meeting of marketing, sales, and production personnel were compared with those generated by the same company personnel when denied specific product knowledge and with the forecasts of selected judgemental and statistical time series methods. Results indicated that product knowledge contributed significantly to forecast accuracy and that the forecast accuracy of company personnel who possessed industry forecasting knowledge (but not product knowledge) was not significantly different from the time series based methods. Furthermore, the company forecasts were more accurate than averages of the judgemental and statistical time series forecasts. These results point to the importance of specific product information to forecast accuracy and accordingly call into question the continuing strong emphasis on improving extrapolation techniques without consideration of the inclusion of non-time series knowledge.     

An exploration of some practical issues in the use of quantitative forecasting models

Extrapolative forecasting models have been available for many years and as most organizations have the need to regularly develop forecasts one might anticipate the widespread use of these models. The evidence in Australia indicates that computer based forecasting systems are not being widely used and in fact a number of established systems have been discarded, with the issue of forecast accuracy often being mentioned as a problem area. Two experiments are carried out to examine this issue by comparing judgemental and quantitative forecasts. Other problem areas mentioned as contributing to the abandonment of forecasting systems include the difficulty of manually reviewing the computer forecasts and the effort required to carefully massage the forecast database to remove extraordinary events.     

Subjective versus objective combining of forecasts: An experiment

The paper reports results of an experiment conducted to evaluate subjective versus objective combination of forecasts. The subjects were undergraduate students at Texas A&M. The students forecasted two different types of time series. The results found show that the subjective combination of forecasts improves their accuracy as compared with individual efforts. Four ex-ante weighting methods were also used to combine the forecasts. They all improve the accuracy of the forecasts. The best results, though, were from the subjective combination of forecasts.     

Subjective confidence in forecasts: A response to fischhoff and MacGregor

Here we evaluate the generalizability of calibration studies which have used general knowledge questions, and argue that on conceptual, methodological and empirical grounds the results have limited applicability to judgemental forecasting. We also review evidence which suggests that judgemental forecast probabilities are influenced by variables such as the desirability, imminence, time period and perceived controllability of the event to be forecast. As these variables do not apply to judgement in the domain of general knowledge, a need for research recognizing and exploring the psychological processes underlying uncertainty about the future is apparent.     

Towards a taxonomy of forecast error measures a factor-comparative investigation of forecast error dimensions

This study explores the nature of information conveyed by 14 error measures drawn from the literature, using real-life forecasting data from 691 individual product items over six quarterly periods. Principal components analysis is used to derive factor solutions that are subsequently compared for two forecasting methods, a version of Holt's exponential smoothing, and the random walk model (Naive 1). The results reveal four underlying forecast error dimensions that are stable across the two factor solutions. The potentially confounding influence of sales volume on the derived error dimensions is also explored via correlation analysis.     

The consistency,coherence and calibration of holistic,decomposed and recomposed judgemental probability forecasts

In this paper we make an empirical investigation of the relationship between the consistency, coherence and validity of probability judgements in a real-world forecasting context. Our results indicate that these measures of the adequacy of an individual's probability assessments are not closely related as we anticipated. Twenty-nine of our thirty-six subjects were better calibrated in point probabilities than in odds and our subjects were, in general more coherent using point probabilities than odds forecasts. Contrary to our expectations we found very little difference in forecasting response and performance between simple and compound holistic forecasts. This result is evidence against the ‘divide-and-conquer’ rationale underlying most applications of normative decision theory. In addition, our recompositions of marginal and conditional assessments into compound forecasts were no better calibrated or resolved than their holistic counterparts. These findings convey two implications for forecasting. First, untrained judgemental forecasters should use point probabilities in preference to odds. Second, judgemental forecasts of complex compound probabilities may be as well assessed holistically as they are using methods of decomposition and recomposition. In addition, our study provides a paradigm for further studies of the relationship between consistency, coherence and validity in judgemental probability forecasting.     

The accuracy of extrapolation (time series) methods: Results of a forecasting competition

In the last few decades many methods have become available for forecasting. As always, when alternatives exist, choices need to be made so that an appropriate forecasting method can be selected and used for the specific situation being considered. This paper reports the results of a forecasting competition that provides information to facilitate such choice. Seven experts in each of the 24 methods forecasted up to 1001 series for six up to eighteen time horizons. The results of the competition are presented in this paper whose purpose is to provide empirical evidence about differences found to exist among the various extrapolative (time series) methods used in the competition.     

The maximum and minimum of primary forecasts

The purpose of this paper is to suggest that the maximum (or minimum) of a number of primary forecasts may make a valuable addition to the forecasting accuracy of a combination of forecasts. Such forecasts are readily computable. Theoretical results are presented for two unbiased forecasts with correlated normally distributed errors, showing that the maximum (minimum) of two forecasts can have a smaller error variance than either of the primary forecasts and the forecast error can have low correlation with the primary errors. Empirical results are obtained for two different sets of forecasts available in the literature, and it is observed that a combination forecast including the maximum and/or minimum has attractive forecasting properties.     

The effectiveness of imprecise probability forecasts

In this paper we investigate the feasibility of algorithmically deriving precise probability forecasts from imprecise forecasts. We provide an empirical evaluation of precise probabilities that have been derived from two types of imprecise probability forecasts: probability intervals and probability intervals with second-order probability distributions. The minimum cross-entropy (MCE) principle is applied to the former to derive precise (i.e. additive) probabilities; expectation (EX) is used to derive precise probabilities in the latter case. Probability intervals that were constructed without second-order probabilities tended to be narrower than and contained in those that were amplified by second-order probabilities. Evidence that this narrowness is due to motivational bias is presented. Analysis of forecasters' mean Probability Scores for the derived precise probabilities indicates that it is possible to derive precise forecasts whose external correspondence is as good as directly assessed precise probability forecasts. The forecasts of the EX method, however, are more like the directly assessed precise forecasts than those of the MCE method.     

Sources of error in forecasts and expectations: U.K. economic models, 1984–8

This paper presents a comparative analysis of the sources of error in forecasts for the UK economy published over a recent four-year period by four independent groups. This analysis rests on the archiving at the ESRC Macroeconomic Modelling Bureau of the original forecasts together with all their accompanying assumptions and adjustments. A method of decomposing observed forecast errors so as to distinguish the contributions of forecaster and model is set out; the impact of future expectations treated in a ‘model-consistent’ or ‘rational’ manner is specifically considered. The results show that the forecaster's adjustments make a substantial contribution to forecast performance, a good part of which comes from adjustments that bring the model on track at the start of the forecast period. The published ex-ante forecasts are usually superior to pure model-based ex-post forecasts, whose performance indicates some misspecification of the underlying models.     

The uses and abuses of ‘consensus’ forecasts

Clemen's (1989) review of the forecast-combining literature amply illustrates both the interest in and the importance of this subject. This article stresses the tautological properties of various consensus measures that assure their success relative to most individual forecasts. It confirms the finding of earlier studies that for each specific macroeconomic variable roughly one-third of individual forecasters are more accurate than a consensus. However, each individual does relatively poorly for some variable while the consensus, in contrast, necessarily never fails relative to most individuals. These results, like most previous studies, describe consensus measures that are synthetic constructs derived from a pre-existing set of individual forecasts. Strictly speaking, this contemporaneous consensus is not available to individual forecasters when their forecasts are made. A prior consensus measure, which is in their information sets, was relatively much less accurate than the contemporaneous measure. Nevertheless, a small subset of individual forecasters were generally inferior to the known, prior consensus forecast.     

The impact of forecasting methodology on the accuracy of national population forecasts: evidence from the Netherlands and Czechoslovakia

"This study considers the accuracy of national population forecasts of the Netherlands and the Czechoslovak Socialist Republic.... We look at the demographic components employed in each forecast, the procedure to extrapolate fertility and the level at which assumptions for each component are formulated. Errors in total population size, fertility, mortality and foreign migration, and age structure are considered. We discuss trends in errors and methodology since 1950 and compare the situations in the two countries. The findings suggest that methodology has only a very limited impact on the accuracy of national population forecasts."     

Seven components of judgmental forecasting skill: Implications for research and the improvement of forecasts

A decomposition of the Brier skill score shows that the performance of judgmental forecasts depends on seven components: environmental predictability, fidelity of the information system, match between environment and forecaster, reliability of information acquisition, reliability of information processing, conditional bias, and unconditional bias. These components provide a framework for research on the forecasting process. Selected literature addressing each component is reviewed, and implications for improving judgmental forecasting are discussed.     

The accuracy of individual and group forecasts from business outlook surveys

This paper reports on the accuracy of quarterly multiperiod predictions of inflation, real growth, unemployment and percentage changes in nominal GNP and two of its more volatile components. The survey data are highly differentiated; they cover 79 professional forecasters (mostly economists, analysts and corporate executives). Combining corresponding predictions from different sources can result in significant gains; thus the group mean forecasts are on the average over time more accurate than most of the corresponding sets of individual forecasts. But there is also a moderate degree of consistency in the relative performance of a sufficient number of the survey members, as evidenced in positive rank correlations among ratios of the individual to group root mean square errors.     

Modifying quantitative forecasts of livestock production using expert judgments: An application to the australian lamb industry

The judgmental modification of quantitative forecasts has become increasingly adopted in the production of agricultural commodity outlook information. Such modifications allow current period information to be incorporated into the forecast value, and ensure that the forecast is realistic in the context of current industry trends. This paper investigates the potential value of this approach in production forecasting in the Australian lamb industry. Several individual and composite econometric models were used to forecast a lamb-slaughtering series with a selected forecast being given to a panel of lamb industry specialists for consideration and modification. The results demonstrate that this approach offers considerable accuracy advantages in the short-term forecasting of livestock market variables, such as slaughtering, whose values can be strongly influenced by current industry conditions.     

Forecasting volatility of emerging stock markets: linear versus non‐linear GARCH models

ARCH and GARCH models are substantially used for modelling volatility of time series data. It is proven by many studies that if variables are significantly skewed, linear versions of these models are not sufficient for both explaining the past volatility and forecasting the future volatility. In this paper, we compare the linear(GARCH(1,1)) and non‐linear(EGARCH) versions of GARCH model by using the monthly stock market returns of seven emerging countries from February 1988 to December 1996. We find that for emerging stock markets GARCH(1,1) model performs better than EGARCH model, even if stock market return series display skewed distributions. Copyright © 2000 John Wiley & Sons, Ltd.     

Predicting long-term earnings growth: Comparisons of expected return models,submartingales and value line analysts

This paper derives four–five year predictions of growth rates of accounting earnings per share implicit in four expected return models commonly used in financial research. A comparison of such growth rates with those produced and reported by Value Line analysts and those generated by a submartingale model revealed the following: two expected return models—the Sharpe–Lintner–Mossin model and the Black model—were significantly more accurate than the submartingale model, though not significantly more accurate than the other return models. However, the growth rate forecasts provided by Value Line significantly outperformed all the other models tested—none of which relied on the direct input of a security analyst.     

Estimating the strength of expert judgement: The case of US mortality forecasts

The use of expert judgement is an important part of demographic forecasting. However, because judgement enters into the forecasting process in an informal way, it has been very difficult to assess its role relative to the analysis of past data. The use of targets in demographic forecasts permits us to embed the subjective forecasting process into a simple time-series regression model, in which expert judgement is incorporated via mixed estimation. The strength of expert judgement is denned, and estimated using the official forecasts of cause-specific mortality in the United States. We show that the weight given to judgement varies in an improbable manner by age. Overall, the weight given to judgement appears too high. An alternative approach to combining expert judgement and past data is suggested.