The impact of forecasting methodology on the accuracy of national population forecasts: evidence from the Netherlands and Czechoslovakia

"This study considers the accuracy of national population forecasts of the Netherlands and the Czechoslovak Socialist Republic.... We look at the demographic components employed in each forecast, the procedure to extrapolate fertility and the level at which assumptions for each component are formulated. Errors in total population size, fertility, mortality and foreign migration, and age structure are considered. We discuss trends in errors and methodology since 1950 and compare the situations in the two countries. The findings suggest that methodology has only a very limited impact on the accuracy of national population forecasts."     

Monetary policy,composite leading economic indicators and predicting the 2001 recession

On 26 November 2001, the National Bureau of Economic Research announced that the US economy had officially entered into a recession in March 2001. This decision was a surprise and did not end all the conflicting opinions expressed by economists. This matter was finally settled in July 2002 after a revision to the 2001 real gross domestic product showed negative growth rates for its first three quarters. A series of political and economic events in the years 2000–01 have increased the amount of uncertainty in the state of the economy, which in turn has resulted in the production of less reliable economic indicators and forecasts. This paper evaluates the performance of two very reliable methodologies for predicting a downturn in the US economy using composite leading economic indicators (CLI) for the years 2000–01. It explores the impact of the monetary policy on CLI and on the overall economy and shows how the gradualness and uncertainty of this impact on the overall economy have affected the forecasts of these methodologies. It suggests that the overexposure of the CLI to the monetary policy tools and a strong, but less effective, expansionary money policy have been the major factors in deteriorating the predictions of these methodologies. To improve these forecasts, it has explored the inclusion of the CLI diffusion index as a prior in the Bayesian methodology. Copyright © 2004 John Wiley & Sons, Ltd.     

Causal forces: Structuring knowledge for time-series extrapolation

This paper examines a strategy for structuring one type of domain knowledge for use in extrapolation. It does so by representing information about causality and using this domain knowledge to select and combine forecasts. We use five categories to express causal impacts upon trends: growth, decay, supporting, opposing, and regressing. An identification of causal forces aided in the determination of weights for combining extrapolation forecasts. These weights improved average ex ante forecast accuracy when tested on 104 annual economic and demographic time series. Gains in accuracy were greatest when (1) the causal forces were clearly specified and (2) stronger causal effects were expected, as in longer-range forecasts. One rule suggested by this analysis was: ‘Do not extrapolate trends if they are contrary to causal forces.’ We tested this rule by comparing forecasts from a method that implicitly assumes supporting trends (Holt's exponential smoothing) with forecasts from the random walk. Use of the rule improved accuracy for 20 series where the trends were contrary; the MdAPE (Median Absolute Percentage Error) was 18% less for the random walk on 20 one-year ahead forecasts and 40% less for 20 six-year-ahead forecasts. We then applied the rule to four other data sets. Here, the MdAPE for the random walk forecasts was 17% less than Holt's error for 943 short-range forecasts and 43% less for 723 long-range forecasts. Our study suggests that the causal assumptions implicit in traditional extrapolation methods are inappropriate for many applications.     

Judgmental adjustment of earnings forecasts

Earnings forecasts have received a great deal of attention, much of which has centered on the comparative accuracy of judgmental and objective forecasting methods. Recently, studies have focused on the use of combinations of subjective and objective forecasts to improve forecast accuracy. This research offers an extension on this theme by subjectively modifying an objective forecast. Specifically, ARIMA forecasts are judgmentally adjusted by analysts using a structured approach based on Saaty's (1980) analytic hierarchy process. The results show that the accuracy of the unadjusted objective forecasts can be improved when judgmentally adjusted.     

Sources of error in forecasts and expectations: U.K. economic models, 1984–8

This paper presents a comparative analysis of the sources of error in forecasts for the UK economy published over a recent four-year period by four independent groups. This analysis rests on the archiving at the ESRC Macroeconomic Modelling Bureau of the original forecasts together with all their accompanying assumptions and adjustments. A method of decomposing observed forecast errors so as to distinguish the contributions of forecaster and model is set out; the impact of future expectations treated in a ‘model-consistent’ or ‘rational’ manner is specifically considered. The results show that the forecaster's adjustments make a substantial contribution to forecast performance, a good part of which comes from adjustments that bring the model on track at the start of the forecast period. The published ex-ante forecasts are usually superior to pure model-based ex-post forecasts, whose performance indicates some misspecification of the underlying models.     

Heterogeneous Forecast Adjustment

There is ample empirical evidence that expert‐adjusted model forecasts can be improved. One way to potential improvement concerns providing various forms of feedback to the sales forecasters. It is also often recognized that the experts (forecasters) might not constitute a homogeneous group. This paper provides a data‐based methodology to discern latent clusters of forecasters, and applies it to a fully new large database with data on expert‐adjusted forecasts, model forecasts and realizations. For the data at hand, two clusters can clearly be identified. Next, the consequences of having clusters are discussed. Copyright © 2016 John Wiley & Sons, Ltd.     

Short-run macroeconomic forecasting: The OECD performance

Forecasts for the seven major industrial countries, Canada, France, Germany, Italy, Japan, the United Kingdom and the United States, are published on a regular basis in the OECD's Economic Outlook. This paper analyses the accuracy of the OECD annual forecasts of output and price changes and of the current balance in the balance of payments. As a reference basis, the forecasts are compared with those generated by a naive model, a random walk process. The measures of forecasting accuracy used are the mean-absolute error, the root-mean-square error, the median-absolute error, and Theil's inequality coefficient. The OECD forecasts of real GNP changes are significantly superior to those generated by the random walk process; however, the OECD price and current balance forecasts are not significantly more accurate than those obtained from the naive model. The OECD's forecasting performance has neither improved nor deteriorated over time.     

Monitoring and adjusting forecasts in the presence of additive outliers

The effect of an additive outlier upon the accuracy of forecasts derived from extrapolative methods is investigated. It is demonstrated that an outlier affects not only the accuracy of the forecasts at the time of occurrence but also subsequent forecasts. Methods to adjust for additive outliers are discussed. The results of the paper are illustrated with two examples.     

Nonlinear identification of judgmental forecasts effects at SKU level

Prediction of demand is a key component within supply chain management. Improved accuracy in forecasts directly affects all levels of the supply chain, reducing stock costs and increasing customer satisfaction. In many application areas, demand prediction relies on statistical software which provides an initial forecast subsequently modified by the expert's judgment. This paper outlines a new methodology based on state‐dependent parameter (SDP) estimation techniques to identify the nonlinear behaviour of such managerial adjustments. This non‐parametric SDP estimate is used as a guideline to propose a nonlinear model that corrects the bias introduced by the managerial adjustments. One‐step‐ahead forecasts of stock‐keeping unit sales sampled monthly from a manufacturing company are utilized to test the proposed methodology. The results indicate that adjustments introduce a nonlinear pattern, undermining accuracy. This understanding can be used to enhance the design of the forecasting support system in order to help forecasters towards more efficient judgmental adjustments. Copyright © 2010 John Wiley & Sons, Ltd.     

Economic factors and the stock market: a new perspective

It has been widely accepted that many financial and economic variables are non‐linear, and neural networks can model flexible linear or non‐linear relationships among variables. The present paper deals with an important issue: Can the many studies in the finance literature evidencing predictability of stock returns by means of linear regression be improved by a neural network? We show that the predictive accuracy can be improved by a neural network, and the results largely hold out‐of‐sample. Both the neural network and linear forecasts show significant market timing ability. While the switching portfolio based on the linear forecasts outperforms the buy‐and‐hold market portfolio under all three transaction cost scenarios, the switching portfolio based on the neural network forecasts beats the market only if there is no transaction cost. Copyright © 1999 John Wiley & Sons, Ltd.     

The use of non-time series information in sales forecasting: A case study

The contribution of product and industry knowledge to the accuracy of sales forecasting was investigated by examining the company forecasts of a leading manufacturer and marketer of consumable products. The company forecasts of 18 products produced by a meeting of marketing, sales, and production personnel were compared with those generated by the same company personnel when denied specific product knowledge and with the forecasts of selected judgemental and statistical time series methods. Results indicated that product knowledge contributed significantly to forecast accuracy and that the forecast accuracy of company personnel who possessed industry forecasting knowledge (but not product knowledge) was not significantly different from the time series based methods. Furthermore, the company forecasts were more accurate than averages of the judgemental and statistical time series forecasts. These results point to the importance of specific product information to forecast accuracy and accordingly call into question the continuing strong emphasis on improving extrapolation techniques without consideration of the inclusion of non-time series knowledge.     

Assessing the historical accuracy of regional economic forecasts

This study addresses for the first time systematic evaluation of a widely used class of forecasts, regional economic forecasts. Ex ante regional structural equation model forecasts are analysed for 19 metropolitan areas. One- to ten-quarter-ahead forecasts are considered and the seven-year sample spans a complete business cycle. Counter to previous speculation in the literature, (1) dependency on macroeconomic forecasting model inputs does not substantially erode accuracy relative to univariate extrapolative methodologies and (2) stochastic time series models do not on average, yield more accurate regional economic predictions than structural models. Similar to findings in other studies, clear preferences among extrapolative methodologies do not emerge. Most general conclusions, however, are subject to caveats based on step-length effects and region-specific effects.     

Estimating private information usage amongst analysts: evidence from UK earnings forecasts

This paper introduces a methodology for estimating the likelihood of private information usage amongst earnings analysts. This is achieved by assuming that one group of analysts generate forecasts based on the underlying dynamics of earnings, while all other analysts are assumed to issue forecasts based on the prevailing consensus forecast. Given this behavioural dichotomy, we are able to derive (and estimate) a structural econometric model of forecast behaviour, which has implications regarding the determinants of analysts' private information endowments and forecast accuracy over the forecast horizon. Copyright © 2010 John Wiley & Sons, Ltd.     

Forecasting inflation using economic indicators: the case of France

In order to provide short‐run forecasts of headline and core HICP inflation for France, we assess the forecasting performance of a large set of economic indicators, individually and jointly, as well as using dynamic factor models. We run out‐of‐sample forecasts implementing the Stock and Watson (1999) methodology. We find that, according to usual statistical criteria, the combination of several indicators—in particular those derived from surveys—provides better results than factor models, even after pre‐selection of the variables included in the panel. However, factors included in VAR models exhibit more stable forecasting performance over time. Results for the HICP excluding unprocessed food and energy are very encouraging. Moreover, we show that the aggregation of forecasts on subcomponents exhibits the best performance for projecting total inflation and that it is robust to data snooping. Copyright © 2007 John Wiley & Sons, Ltd.     

Forecasting the 10‐year US Treasury rate

This study compares the performance of two forecasting models of the 10‐year Treasury rate: a random walk (RW) model and an augmented‐autoregressive (A‐A) model which utilizes the information in the expected inflation rate. For 1993–2008, the RW and A‐A forecasts (with different lead times and forecast horizons) are generally unbiased and accurately predict directional change under symmetric loss. However, the A‐A forecasts outperform the RW, suggesting that the expected inflation rate (as a leading indicator) helps improve forecast accuracy. This finding is important since bond market efficiency implies that the RW forecasts are optimal and cannot be improved. Copyright © 2009 John Wiley & Sons, Ltd.     

Do experts' adjustments on model‐based SKU‐level forecasts improve forecast quality?

Model‐based SKU‐level forecasts are often adjusted by experts. In this paper we propose a statistical methodology to test whether these expert forecasts improve on model forecasts. Application of the methodology to a very large database concerning experts in 35 countries who adjust SKU‐level forecasts for pharmaceutical products in seven distinct categories leads to the general conclusion that expert forecasts are equally good at best, but are more often worse than model‐based forecasts. We explore whether this is due to experts putting too much weight on their contribution, and this indeed turns out to be the case. Copyright © 2009 John Wiley & Sons, Ltd.     

Do IMF fiscal forecasts add value?

We used a panel of 29 advanced and emerging market countries to investigate whether the IMF's World Economic Outlook (WEO) fiscal forecasts add value in terms of forecast accuracy and information content, relative to private sector forecasts (from Consensus Economics). We find that: (i) WEO forecasts are not significantly less accurate than Consensus forecasts; (ii) WEO and Consensus forecasts tend to mutually encompass one another; and (iii) each source of forecasts appears to contain some information that is not embedded in the other source.     

Efficient use of information in the formation of subjective industry forecasts

The paper considers the use of information by a panel of expert industry forecasters, focusing on their information-processing biases. The panel forecasts construction output by sector up to three years ahead. It is found that the biases observed in laboratory experiments, particularly ‘anchoring’, are observable. The expectations are formed by adjusting the previous forecast to take new information into account. By analysing forecast errors it is concluded that the panel overweight recently released information and do not understand the dynamics of the industry. However, their forecasts, both short and long term, are better than an alternative econometric model, and combining the two sources of forecasts leads to a deterioration in forecast accuracy. The expert forecasts can be ‘de-biased’, and this leads to the conclusion that it is better to optimally process information sources than to combine (optimally) alternative forecasts.     

A BVAR model for the connecticut economy

A Bayesian vector autoregressive (BVAR) model is developed for the Connecticut economy to forecast the unemployment rate, nonagricultural employment, real personal income, and housing permits authorized. The model includes both national and state variables. The Bayesian prior is selected on the basis of the accuracy of the out-of-sample forecasts. We find that a loose prior generally produces more accurate forecasts. The out-of-sample accuracy of the BVAR forecasts is also compared with that of forecasts from an unrestricted VAR model and of benchmark forecasts generated from univariate ARIMA models. The BVAR model generally produces the most accurate short- and long-term out-of-sample forecasts for 1988 through 1992. It also correctly predicts the direction of change.     

ARMA Models and the Box–Jenkins Methodology

The purpose of this paper is to apply the Box–Jenkins methodology to ARIMA models and determine the reasons why in empirical tests it is found that the post-sample forecasting the accuracy of such models is generally worse than much simpler time series methods. The paper concludes that the major problem is the way of making the series stationary in its mean (i.e. the method of differencing) that has been proposed by Box and Jenkins. If alternative approaches are utilized to remove and extrapolate the trend in the data, ARMA models outperform the models selected through Box–Jenkins methodology. In addition, it is shown that using ARMA models to seasonally adjusted data slightly improves post-sample accuracies while simplifying the use of ARMA models. It is also confirmed that transformations slightly improve post-sample forecasting accuracy, particularly for long forecasting horizons. Finally, it is demonstrated that AR(1), AR(2) and ARMA(1,1) models can produce more accurate post-sample forecasts than those found through the application of Box–Jenkins methodology.© 1997 John Wiley & Sons, Ltd.