Judgement in learning‐curve forecasting: a laboratory study

This study investigates whether human judgement can be of value to users of industrial learning curves, either alone or in conjunction with statistical models. In a laboratory setting, it compares the forecast accuracy of a statistical model and judgemental forecasts, contingent on three factors: the amount of data available prior to forecasting, the forecasting horizon, and the availability of a decision aid (projections from a fitted learning curve). The results indicate that human judgement was better than the curve forecasts overall. Despite their lack of field experience with learning curve use, 52 of the 79 subjects outperformed the curve on the set of 120 forecasts, based on mean absolute percentage error. Human performance was statistically superior to the model when few data points were available and when forecasting further into the future. These results indicate substantial potential for human judgement to improve predictive accuracy in the industrial learning‐curve context. Copyright © 1999 John Wiley & Sons, Ltd.     

Price forecasting and evaluation: An application in agriculture

Because of the high volatility of prices of agricultural commodities over the past decade, the importance of accurate price forecasting for decision makers has become even more acute. This paper reviews literature on forecasting and evaluation. An application with forecasting U.S. hog prices is presented which includes both economic and statistical evaluation measures. Seven forecasting approaches are described and their performances are examined over 24 quarters from 1976 to 1981. These methods include exponential smoothing, an autoregressive integrated moving average process, an econometric model, expert judgement, and a composite forecasting approach. The application gives results which support previous findings in the forecasting literature and suggests that forecasting methods can provide valuable information to the decision maker.     

Assessing time series models for forecasting international migration: Lessons from the United Kingdom

Migration is one of the most unpredictable demographic processes. The aim of this article is to provide a blueprint for assessing various possible forecasting approaches in order to help safeguard producers and users of official migration statistics against misguided forecasts. To achieve that, we first evaluate the various existing approaches to modelling and forecasting of international migration flows. Subsequently, we present an empirical comparison of ex post performance of various forecasting methods, applied to international migration to and from the United Kingdom. The overarching goal is to assess the uncertainty of forecasts produced by using different forecasting methods, both in terms of their errors (biases) and calibration of uncertainty. The empirical assessment, comparing the results of various forecasting models against past migration estimates, confirms the intuition about weak predictability of migration, but also highlights varying levels of forecast errors for different migration streams. There is no single forecasting approach that would be well suited for different flows. We therefore recommend adopting a tailored approach to forecasts, and applying a risk management framework to their results, taking into account the levels of uncertainty of the individual flows, as well as the differences in their potential societal impact.     

Prior Information and ARIMA Forecasting

Using the method of ARIMA forecasting with benchmarks developed in this paper, it is possible to obtain forecasts which take into account the historical information of a series, captured by an ARIMA model (Box and Jenkins, 1970), as well as partial prior information about the forecasts. Prior information takes the form of benchmarks. These originate from the advice of experts, from forecasts of an annual econometric model or simply from pessimistic, realistic or optimistic scenarios contemplated by the analyst of the current economic situation. The benchmarks may represent annual levels to be achieved, neighbourhoods to be reached for a given time period, movements to be displayed or more generally any linear criteria to be satisfied by the forecasted values. The forecaster may then exercise his current economic evaluation and judgement to the fullest extent in deriving forecasts, since the laboriousness experienced without a systematic method is avoided.     

Forecasting temporally aggregated vector ARMA processes

If interest centres on forecasting a temporally aggregated multiple time series and the generation process of the disaggregate series is a known vector ARMA (autoregressive moving average) process then forecasting the disaggregate series and temporally aggregating the forecasts is at least as efficient, under a mean squared error measure, as forecasting the aggregated series directly. Necessary and sufficient conditions for equality of the two forecasts are given. In practice the data generation process is usually unknown and has to be determined from the available data. Using asymptotic theory it is shown that also in this case aggregated forecasts from the disaggregate process will usually be superior to forecasts obtained from the aggregated process.     

Judgemental revision of sales forecasts: Effectiveness of forecast selection

When managers make revisions to sales forecasts initially generated by a rational quantitative model it is important that the particular forecasts selected for adjustment are those which would benefit most from the adjustment process (i.e. realize high errors). This study reports an empirical investigation on this issue, spanning six quarterly forecasting periods and incorporating forecasting data on over 850 products. The results show that the errors of the forecasts chosen for revision are, in general, higher than those which were not chosen. In addition, it is shown that managesrs tend to revise forecasts which are initially low, hence possibily introducing some degree of bias into the overall forecasts.     

Evaluating the role judgment plays in forecast accuracy

The majority of model-based forecasting efforts today rely on relatively simple techniques of estimation and the subjective adjustment of the model's results to produce forecasts. Published forecasts reflect to a great extent the judgment of the forecaster rather than what the model by itself has to say about the future. This paper examines the role judgment plays in the process of producing a macroeconometric forecast. The debate over the use of adjustment constants to alter the statistical results of a model is outlined and an empirical analysis of forecasts generated by the Michigan Quarterly Econometric Model of the US economy is presented using a unique data set which isolates the role of judgment in the forecasting process.     

Requirements for effective use of graphical methods in interactive forecasting

The construction of forecasts using interactive data analysis systems is greatly aided by the availability of graphical procedures. Data exploration, model identification and estimation, and interpretation of final forecasts are made considerably easier by the visual relay of information. This article discusses some recent developments in time series graphics designed to assist in the forecasting process. A discussion of requirerients for effective use of graphics in interactive forecasting is included as illustrated through an application of the Box-Jenkins methodology. Illustrations are included from the STATGRAPHICS system, a prototype implementation in APL.     

Conservatism and consensus-seeking among economic forecasters

This paper uses the track records of a panel of US economic forecasters participating in a consensus forecasting service to test for conservatism and consensus-seeking behaviour. The tests are based on a particular method-of-moments estimator, designed to allow for the heteroscedasticity and serial correlation which is inevitably present in errors from repeated forecasts for fixed target dates. Most forecasters prove to be conservative. When revising forecasts they give too much weight to their own past forecasts. Surprisingly, forecasters are not consensus-seeking but ‘variety-seeking’. When revising forecasts, they give too little weight to the known forecasts of other forecasters.     

A classifying procedure for signalling turning points

A Hidden Markov Model (HMM) is used to classify an out‐of‐sample observation vector into either of two regimes. This leads to a procedure for making probability forecasts for changes of regimes in a time series, i.e. for turning points. Instead of estimating past turning points using maximum likelihood, the model is estimated with respect to known past regimes. This makes it possible to perform feature extraction and estimation for different forecasting horizons. The inference aspect is emphasized by including a penalty for a wrong decision in the cost function. The method, here called a ‘Markov Bayesian Classifier (MBC)’, is tested by forecasting turning points in the Swedish and US economies, using leading data. Clear and early turning point signals are obtained, contrasting favourably with earlier HMM studies. Some theoretical arguments for this are given. Copyright © 2004 John Wiley & Sons, Ltd.     

Ex post and ex ante analysis of provisional data

In this paper we suggest a framework to assess the degree of reliability of provisional estimates as forecasts of final data, and we re‐examine the question of the most appropriate way in which available data should be used for ex ante forecasting in the presence of a data‐revision process. Various desirable properties for provisional data are suggested, as well as procedures for testing them, taking into account the possible non‐stationarity of economic variables. For illustration, the methodology is applied to assess the quality of the US M1 data production process and to derive a conditional model whose performance in forecasting is then tested against other alternatives based on simple transformations of provisional data or of past final data. Copyright © 1999 John Wiley & Sons, Ltd.     

Efficient use of information in the formation of subjective industry forecasts

The paper considers the use of information by a panel of expert industry forecasters, focusing on their information-processing biases. The panel forecasts construction output by sector up to three years ahead. It is found that the biases observed in laboratory experiments, particularly ‘anchoring’, are observable. The expectations are formed by adjusting the previous forecast to take new information into account. By analysing forecast errors it is concluded that the panel overweight recently released information and do not understand the dynamics of the industry. However, their forecasts, both short and long term, are better than an alternative econometric model, and combining the two sources of forecasts leads to a deterioration in forecast accuracy. The expert forecasts can be ‘de-biased’, and this leads to the conclusion that it is better to optimally process information sources than to combine (optimally) alternative forecasts.     

Judgemental revision of sales forecasts: The relative performance of judgementally revised versus non-revised forecasts

The judgemental revision of sales forecasts is an issue which is receiving increasing attention in the forecasting literature. This paper compares the performance of forecasts after revision by managers with that of the forecasts which were accepted by them without revision. The data set consists of sales forecasting data from an industrial company, spanning six quarterly periods and relating to some 900 individual products. The findings show that, in general, the improvements made by managers bring the forecast errors of revised forecasts more into line with non-revised forecasts, but the change is often marginal, and the best result is equivalence between revised and non-revised forecasts.     

Predictive ability and economic gains from volatility forecast combinations

The availability of numerous modeling approaches for volatility forecasting leads to model uncertainty for both researchers and practitioners. A large number of studies provide evidence in favor of combination methods for forecasting a variety of financial variables, but most of them are implemented on returns forecasting and evaluate their performance based solely on statistical evaluation criteria. In this paper, we combine various volatility forecasts based on different combination schemes and evaluate their performance in forecasting the volatility of the S&P 500 index. We use an exhaustive variety of combination methods to forecast volatility, ranging from simple techniques to time-varying techniques based on the past performance of the single models and regression techniques. We then evaluate the forecasting performance of single and combination volatility forecasts based on both statistical and economic loss functions. The empirical analysis in this paper yields an important conclusion. Although combination forecasts based on more complex methods perform better than the simple combinations and single models, there is no dominant combination technique that outperforms the rest in both statistical and economic terms.     

Overreaction in Survey Exchange Rate Forecasts

We use survey data on five bilateral exchange rates to provide empirical evidence of the fact that professional forecasters of foreign exchange rates behave irrationally, in the specific sense that they respond inaccurately to available information in the market when forming their predictions. In particular, we find systematic biases in the forecasts resulting in the overreaction of analysts to past information contained in the exchange rate dynamics: forecasters change their prediction more than it would be rational on the basis of past realized changes. In addition, forecasters are heterogeneous in their irrationality: low performers in previous periods show a more pronounced overreaction effect. This can be read as an indication of perpetration of past errors and continued inability to learn from the past. In the second part of the paper, we exploit the novel structure of our dataset, which consists of survey data extracted from the Bloomberg platform and readily available to anyone. This feature allows us to consider their own and others' past forecasts as part of the information set that analysts use in making their predictions. By using past forecasts as proxies for relevant macroeconomic variables, we find evidence that analysts fail to correctly process not only the information contained in the spot rate past dynamics but also the information in this broader set. We see this as confirmation of the existence of inefficiency and heterogeneity between low and high performers also when full information is available. Copyright © 2014 John Wiley & Sons, Ltd.     

Combining ordinal forecasts with an application in a financial market

The literature on combining forecasts has almost exclusively focused on combining point forecasts. The issues and methods of combining ordinal forecasts have not yet been fully explored, even though ordinal forecasting has many practical applications in business and social research. In this paper, we consider the case of forecasting the movement of the stock market which has three possible states (bullish, bearish and sluggish). Given the sample of states predicted by different forecasters, several statistical and operation research methods can be applied to determine the optimal weight assigned to each forecaster in combining the ordinal forecasts. The performance of these methods is examined using Hong Kong stock market forecasting data, and their accuracies are found to be better than the consensus method and individual forecasts.     

Forecasting Elections

In this paper we assess opinion polls, prediction markets, expert opinion and statistical modelling over a large number of US elections in order to determine which perform better in terms of forecasting outcomes. In line with existing literature, we bias‐correct opinion polls. We consider accuracy, bias and precision over different time horizons before an election, and we conclude that prediction markets appear to provide the most precise forecasts and are similar in terms of bias to opinion polls. We find that our statistical model struggles to provide competitive forecasts, while expert opinion appears to be of value. Finally we note that the forecast horizon matters; whereas prediction market forecasts tend to improve the nearer an election is, opinion polls appear to perform worse, while expert opinion performs consistently throughout. We thus contribute to the growing literature comparing election forecasts of polls and prediction markets. Copyright © 2015 John Wiley & Sons, Ltd.     

Tourism forecasting: A review of empirical research

Many publications on tourism forecasting have appeared during the past twenty years. The purpose of this article is to organize and summarize that scattered literature. General conclusions are also drawn from the studies to help those wishing to develop tourism forecasts of their own. The forecasting techniques discussed include time series models, econometric causal models, the gravity model and expert-opinion techniques. The major conclusions are that time series models are the simplest and least costly (and therefore most appropriate for practitioners); the gravity model is best suited to handle international tourism flows (and will be most useful to governments and tourism agencies); and expert-opinion methods are useful when data are unavailable. Further research is needed on the use of economic indicators in tourism forecasting, on the development of attractivity and emissiveness indexes for use in gravity and econometric models and on empirical comparisons among the different methods.     

Henderson‐Trending of Macroeconomic Variables and Forecasting Accuracy

Using a structural time‐series model, the forecasting accuracy of a wide range of macroeconomic variables is investigated. Specifically of importance is whether the Henderson moving‐average procedure distorts the underlying time‐series properties of the data for forecasting purposes. Given the weight of attention in the literature to the seasonal adjustment process used by various statistical agencies, this study hopes to address the dearth of literature on ‘trending’ procedures. Forecasts using both the trended and untrended series are generated. The forecasts are then made comparable by ‘detrending’ the trended forecasts, and comparing both series to the realised values. Forecasting accuracy is measured by a suite of common methods, and a test of significance of difference is applied to the respective root mean square errors. It is found that the Henderson procedure does not lead to deterioration in forecasting accuracy in Australian macroeconomic variables on most occasions, though the conclusions are very different between the one‐step‐ahead and multi‐step‐ahead forecasts. Copyright © 2011 John Wiley & Sons, Ltd.     

A smart automated macroeconometric forecasting system

A smart, automated forecasting system is a kind of expert system for generating forecasts wholly or partly without human intervention. A pilot-scale system is reported in this paper. The interventions are made by a rulebase which describes the actual intervention procedures of the London Business School Centre for Economic Forecasting on one of the major macroeconometric forecasting models for the U.K. economy. The rulebase is sufficiently general as to be applicable to any forecasting model. One difference of the pilot model reported in this paper from conventional models is that policy behaviour is described entirely by rules rather than equations. This allows the use of thresholds, floors, ceilings, and discontinuities.