Testing for unreliable estimators and insignificant forecasts in combined forecasts

The reliability and precision of the weights used in combining individual forecasts, irrespective of the method of combination, is important in evaluating a combined forecast. The objective of this study is not to suggest the ‘best’ method of combining individual forecasts, but rather to propose exploratory procedures, that make use of all available sample information contained in the covariance matrix of individual forecast errors, to (1) detect if the weights used in combining forecasts are ‘reliable’ (and ‘stable’ if it is known that the covariance matrix of forecast errors is stationary over time) and (2) test for ‘insignificant’ individual forecasts used in forming a combined forecast. We present empirical applications using two-year sales and individual forecast data provided by a major consumer durables manufacturer to illustrate the feasibility of our proposed procedures.     

Unbiasedness,efficiency and the combination of economic forecasts

This paper considers the problem of determining whether forecasts are unbiased and examines the implications this has for combining different forecasts. The practical issues of how economic forecasts might be combined are discussed. There is an empirical illustration of the procedures in which the properties of UK forecasts from the London Business School, the National Institute, the Henley Centre for Forecasting, Phillips and Drew and the OECD are examined.     

An empirical investigation of combinations of economic forecasts

This paper examines the effects of combining three econometric and three times-series forecasts of growth and inflation in the U.K. If forecasts are unbiased then a combination exploiting this fact will be more efficient than an unrestricted combination. Ex post econometric forecasts may be biased but ex ante they are unbiased. The results of the study are that a restricted linear combination of the econometric forecasts is superior to an unrestricted combination and also to the unweighted mean of the forecasts. However, it is not preferred to the best of the individual forecasts.     

Time series forecasts and extra-model information

Often a forecaster has supplementary information (e.g. field reports or forecasts from another source) that cannot be included directly in a time series model. Especially interesting are cases where this information is given at time intervals that are different from those of the time series model forecasts. Previous authors have considered a numerical and a model-based statistical method for combining extra-model information of this type with ARIMA model forecasts. This paper extends both methods to vector ARMA model forecasts and dynamic regression (transfer function) model forecasts. It is also shown that a Lagrange multiplier numerical procedure arises as a special case of the model-based procedure. An empirical example is given.     

Linear combination of forecasts with an intercept: A bayesian approach

The standard approach to combining n expert forecasts involves taking a weighted average. Granger and Ramanathan proposed introducing an intercept term and unnormalized weights. This paper deduces their proposal from Bayesian principles. We find that their formula is equivalent to taking a weighted average of the n expert forecasts plus the decision-maker's prior forecast.     

To combine or not to combine? issues of combining forecasts

This paper addresses issues such as: Does it always pay to combine individual forecasts of a variable? Should one combine an unbiased forecast with one that is heavily biased? Should one use optimal weights as suggested by Bates and Granger over twenty years ago? A simple model which accounts for the main features of individual forecasts is put forward. Bayesian analysis of the model using noninformative and informative prior probability densities is provided which extends and generalizes results obtained by Winkler (1981) and compared with non-Bayesian methods of combining forecasts relying explicitly on a statistical model for the individual forecasts. It is shown that in some instances it is sensible to use a simple average of individual forecasts instead of using Bates and Granger type weights. Finally, model uncertainty is considered and the issue of combining different models for individual forecasts is addressed.     

Estimating the strength of expert judgement: The case of US mortality forecasts

The use of expert judgement is an important part of demographic forecasting. However, because judgement enters into the forecasting process in an informal way, it has been very difficult to assess its role relative to the analysis of past data. The use of targets in demographic forecasts permits us to embed the subjective forecasting process into a simple time-series regression model, in which expert judgement is incorporated via mixed estimation. The strength of expert judgement is denned, and estimated using the official forecasts of cause-specific mortality in the United States. We show that the weight given to judgement varies in an improbable manner by age. Overall, the weight given to judgement appears too high. An alternative approach to combining expert judgement and past data is suggested.     

Combining ordinal forecasts with an application in a financial market

The literature on combining forecasts has almost exclusively focused on combining point forecasts. The issues and methods of combining ordinal forecasts have not yet been fully explored, even though ordinal forecasting has many practical applications in business and social research. In this paper, we consider the case of forecasting the movement of the stock market which has three possible states (bullish, bearish and sluggish). Given the sample of states predicted by different forecasters, several statistical and operation research methods can be applied to determine the optimal weight assigned to each forecaster in combining the ordinal forecasts. The performance of these methods is examined using Hong Kong stock market forecasting data, and their accuracies are found to be better than the consensus method and individual forecasts.     

Spurious Forecasts?

P. C. B. Phillips (1998) demonstrated that deterministic trends are a valid representation of an otherwise stochastic trending mechanism; he remained skeptic, however, about the predictive power of such representations. In this paper we prove that forecasts built upon spurious regression may perform (asymptotically) as well as those issued from a correctly specified regression. We derive the order in probability of several in‐sample and out‐of‐sample predictability criteria ( test, root mean square error, Theil's U‐statistics and R²) using forecasts based upon a least squares‐estimated regression between independent variables generated by a variety of empirically relevant data‐generating processes. It is demonstrated that, when the variables are mean stationary or trend stationary, the order in probability of these criteria is the same whether the regression is spurious or not. Simulation experiments confirm our asymptotic results. Copyright © 2011 John Wiley & Sons, Ltd.     

Early Warning with Calibrated and Sharper Probabilistic Forecasts

Given a nonlinear model, a probabilistic forecast may be obtained by Monte Carlo simulations. At a given forecast horizon, Monte Carlo simulations yield sets of discrete forecasts, which can be converted to density forecasts. The resulting density forecasts will inevitably be downgraded by model misspecification. In order to enhance the quality of the density forecasts, one can mix them with the unconditional density. This paper examines the value of combining conditional density forecasts with the unconditional density. The findings have positive implications for issuing early warnings in different disciplines including economics and meteorology, but UK inflation forecasts are considered as an example. Copyright © 2012 John Wiley & Sons, Ltd.     

ARIMA models of the price level: An assessment of the multilevel adaptive learning process in the USA

This paper estimates the ARIMA processes for the observed and expected price level corresponding to the three-level adaptive expectations model proposed by Jacobs and Jones (1980). These univariate processes are then compared with the best-fit ARIMA model. The results indicate that the best-fit model for the observed price level is a restricted version of the two-level adaptive learning process specified in terms of prices, suggesting a simple adaptive rule in the inflation rate. A comparison of the time-series forecasts from the best-fit model with the mean responses to the ASA-NBER survey shows no significant difference in their accuracy. The time-series forecasts are, however, conditionally efficient. The best-fit ARIMA model for expected prices measured by the ASA-NBER consensus forecasts does not correspond to any version of the Jacobs and Jones model.     

Local to unity,long‐horizon forecasting thresholds for model selection in the AR(1)

This article introduces a novel framework for analysing long‐horizon forecasting of the near non‐stationary AR(1) model. Using the local to unity specification of the autoregressive parameter, I derive the asymptotic distributions of long‐horizon forecast errors both for the unrestricted AR(1), estimated using an ordinary least squares (OLS) regression, and for the random walk (RW). I then identify functions, relating local to unity ‘drift’ to forecast horizon, such that OLS and RW forecasts share the same expected square error. OLS forecasts are preferred on one side of these ‘forecasting thresholds’, while RW forecasts are preferred on the other. In addition to explaining the relative performance of forecasts from these two models, these thresholds prove useful in developing model selection criteria that help a forecaster reduce error. Copyright © 2004 John Wiley & Sons, Ltd.     

Real‐Time Forecasts of Inflation: The Role of Financial Variables

We present a mixed‐frequency model for daily forecasts of euro area inflation. The model combines a monthly index of core inflation with daily data from financial markets; estimates are carried out with the MIDAS regression approach. The forecasting ability of the model in real time is compared with that of standard VARs and of daily quotes of economic derivatives on euro area inflation. We find that the inclusion of daily variables helps to reduce forecast errors with respect to models that consider only monthly variables. The mixed‐frequency model also displays superior predictive performance with respect to forecasts solely based on economic derivatives. Copyright © 2012 John Wiley & Sons, Ltd.     

Linear combination of forecasts: A general Bayesian model

We provide a general Bayesian model for combining forecasts from experts (or forecasting models) who might be biased and correlated with each other. The combination procedure involves debiasing and then combining unbiased forecasts. We also provide a sequential method for learning about the forecasters' biases in the process of combining information from them.     

Do experts' adjustments on model‐based SKU‐level forecasts improve forecast quality?

Model‐based SKU‐level forecasts are often adjusted by experts. In this paper we propose a statistical methodology to test whether these expert forecasts improve on model forecasts. Application of the methodology to a very large database concerning experts in 35 countries who adjust SKU‐level forecasts for pharmaceutical products in seven distinct categories leads to the general conclusion that expert forecasts are equally good at best, but are more often worse than model‐based forecasts. We explore whether this is due to experts putting too much weight on their contribution, and this indeed turns out to be the case. Copyright © 2009 John Wiley & Sons, Ltd.     

N-step combinations of forecasts

While there is general agreement that a linear combination of forecasts can outperform the individual forecasts, there is controversy about the appropriateness of the combination method to be used in a given situation. Hence, in any given application it may be more beneficial to combine different sets of combined forecasts rather than picking one of them. This paper introduces the concept of N-step combinations of forecasts which involves combining the combined forecasts obtained from different combination procedures used at the preceding step. Using quarterly GNP data, evidence supporting the increase in the accuracy of the one-period-ahead ex-ante forecasts as the combination step increases is provided. The MSE, MAE, MAPE and their corresponding standard deviations are used to evaluate the accuracy of the forecasts obtained.     

Linear combination of restrictions and forecasts in time series analysis

An important tool in time series analysis is that of combining information in an optimal way. Here we establish a basic combining rule of linear predictors and show that such problems as forecast updating, missing value estimation, restricted forecasting with binding constraints, analysis of outliers and temporal disaggregation can be viewed as problems of optimal linear combination of restrictions and forecasts. A compatibility test statistic is also provided as a companion tool to check that the linear restrictions are compatible with the forecasts generated from the historical data. Copyright © 2000 John Wiley & Sons, Ltd.     

Forecasting the Term Structure of Interest Rates Using Integrated Nested Laplace Approximations

This article discusses the use of Bayesian methods for inference and forecasting in dynamic term structure models through integrated nested Laplace approximations (INLA). This method of analytical approximation allows accurate inferences for latent factors, parameters and forecasts in dynamic models with reduced computational cost. In the estimation of dynamic term structure models it also avoids some simplifications in the inference procedures, such as the inefficient two‐step ordinary least squares (OLS) estimation. The results obtained in the estimation of the dynamic Nelson–Siegel model indicate that this method performs more accurate out‐of‐sample forecasts compared to the methods of two‐stage estimation by OLS and also Bayesian estimation methods using Markov chain Monte Carlo (MCMC). These analytical approaches also allow efficient calculation of measures of model selection such as generalized cross‐validation and marginal likelihood, which may be computationally prohibitive in MCMC estimations. Copyright © 2014 John Wiley & Sons, Ltd.     

Linear constraints and the efficiency of combined forecasts

Studies of combined forecasts have typically constrained the combining weights to sum to one and have not included a constant term in the combination. In a recent paper, Granger and Ramanathan (1984) have argued in favour of an unrestricted linear combination, including a constant term. This paper shows that for the purpose of prediction it may make sense to impose restrictions on the combining model because of potential increases in forecasting efficiency. Empirical results show that small gains in forecasting efficiency can be obtained by restricting the linear combination of GNP forecasts from four econometric models.     

A BVAR model for the connecticut economy

A Bayesian vector autoregressive (BVAR) model is developed for the Connecticut economy to forecast the unemployment rate, nonagricultural employment, real personal income, and housing permits authorized. The model includes both national and state variables. The Bayesian prior is selected on the basis of the accuracy of the out-of-sample forecasts. We find that a loose prior generally produces more accurate forecasts. The out-of-sample accuracy of the BVAR forecasts is also compared with that of forecasts from an unrestricted VAR model and of benchmark forecasts generated from univariate ARIMA models. The BVAR model generally produces the most accurate short- and long-term out-of-sample forecasts for 1988 through 1992. It also correctly predicts the direction of change.