Decision‐Based Forecast Evaluation of UK Interest Rate Predictability

This paper illustrates the importance of density forecasting and forecast evaluation in portfolio decision making. The decision‐making environment is fully described for an investor seeking to optimally allocate her portfolio between long and short Treasury bills, over investment horizons of up to 2 years. We examine the impact of parameter uncertainty and predictability in bond returns on the investor's allocation and we describe how the forecasts are computed and used in this context. Both statistical and decision‐based criteria are used to assess the predictability of returns. Our results show sensitivity to the evaluation criterion used and, in the context of investment decision making under an economic value criterion, we find some potential gain for the investor from assuming predictability. Copyright © 2015 John Wiley & Sons, Ltd.     

Predicting the Distribution of Stock Returns: Model Formulation,Statistical Evaluation,VaR Analysis and Economic Significance

A large literature has investigated predictability of the conditional mean of low‐frequency stock returns by macroeconomic and financial variables; however, little is known about predictability of the conditional distribution. We look at one‐step‐ahead out‐of‐sample predictability of the conditional distribution of monthly US stock returns in relation to the macroeconomic and financial environment. Our methodological approach is innovative: we consider several specifications for the conditional density and combinations schemes. Our results are as follows: the entire density is predicted under combination schemes as applied to univariate GARCH models with Gaussian innovations; the Bayesian winner in relation to GARCH‐skewed‐t models is informative about the 5% value at risk; the average realised utility of a mean–variance investor is maximised under the Bayesian winner as applied to GARCH models with symmetric Student t innovations. Our results have two implications: the best prediction model depends on the evaluation criterion; and combination schemes outperform individual models. Copyright © 2015 John Wiley & Sons, Ltd.     

What can we learn from the return predictability over the business cycle?

In this paper, we forecast stock returns using time‐varying parameter (TVP) models with parameters driven by economic conditions. An in‐sample specification test shows significant variation in the parameters. Out‐of‐sample results suggest that the TVP models outperform their constant coefficient counterparts. We also find significant return predictability from both statistical and economic perspectives with the application of TVP models. The out‐of‐sample R² of an equal‐weighted combination of TVP models is as high as 2.672%, and the gains in the certainty equivalent return are 214.7 basis points. Further analysis indicates that the improvement in predictability comes from the use of information on economic conditions rather than simply from allowing the coefficients to vary with time.     

Forecasting aggregate market volatility: The role of good and bad uncertainties

We decompose economic uncertainty into "good" and "bad" components according to the sign of innovations. Our results indicate that bad uncertainty provides stronger predictive content regarding future market volatility than good uncertainty. The asymmetric models with good and bad uncertainties forecast market volatility in a better way than the symmetric models with overall uncertainty. The combination for asymmetric uncertainty models significantly outperforms the benchmark of autoregression, as well as the combination for symmetric models. The revealed volatility predictability is further demonstrated to be economically significant in the framework of portfolio allocation.     

Evaluating the predictive accuracy of volatility models

Standard statistical loss functions, such as mean‐squared error, are commonly used for evaluating financial volatility forecasts. In this paper, an alternative evaluation framework, based on probability scoring rules that can be more closely tailored to a forecast user's decision problem, is proposed. According to the decision at hand, the user specifies the economic events to be forecast, the scoring rule with which to evaluate these probability forecasts, and the subsets of the forecasts of particular interest. The volatility forecasts from a model are then transformed into probability forecasts of the relevant events and evaluated using the selected scoring rule and calibration tests. An empirical example using exchange rate data illustrates the framework and confirms that the choice of loss function directly affects the forecast evaluation results. Copyright © 2001 John Wiley & Sons, Ltd.     

Forecast Combination for Euro Area Inflation: A Cure in Times of Crisis?

The period of extraordinary volatility in euro area headline inflation starting in 2007 raised the question whether forecast combination methods can be used to hedge against bad forecast performance of single models during such periods and provide more robust forecasts. We investigate this issue for forecasts from a range of short‐term forecasting models. Our analysis shows that there is considerable variation of the relative performance of the different models over time. To take that into account we suggest employing performance‐based forecast combination methods—in particular, one with more weight on the recent forecast performance. We compare such an approach with equal forecast combination that has been found to outperform more sophisticated forecast combination methods in the past, and investigate whether it can improve forecast accuracy over the single best model. The time‐varying weights assign weights to the economic interpretations of the forecast stemming from different models. We also include a number of benchmark models in our analysis. The combination methods are evaluated for HICP headline inflation and HICP excluding food and energy. We investigate how forecast accuracy of the combination methods differs between pre‐crisis times, the period after the global financial crisis and the full evaluation period, including the global financial crisis with its extraordinary volatility in inflation. Overall, we find that forecast combination helps hedge against bad forecast performance and that performance‐based weighting outperforms simple averaging. Copyright © 2017 John Wiley & Sons, Ltd.     

Predicting Recessions with Leading Indicators: Model Averaging and Selection over the Business Cycle

Four methods of model selection—equally weighted forecasts, Bayesian model‐averaged forecasts, and two models produced by the machine‐learning algorithm boosting—are applied to the problem of predicting business cycle turning points with a set of common macroeconomic variables. The methods address a fundamental problem faced by forecasters: the most useful model is simple but makes use of all relevant indicators. The results indicate that successful models of recession condition on different economic indicators at different forecast horizons. Predictors that describe real economic activity provide the clearest signal of recession at very short horizons. In contrast, signals from housing and financial markets produce the best forecasts at longer forecast horizons. A real‐time forecast experiment explores the predictability of the 2001 and 2007 recessions. Copyright © 2015 John Wiley & Sons, Ltd.     

Forecasting the Outcome of Closed‐Door Decisions: Evidence from 500 Years of Betting on Papal Conclaves

Closed‐door decisions may be defined as decisions in which the outcome is determined by a limited number of decision‐makers and where the process is shrouded in at least some secrecy. In this paper, we examine the use of betting markets to forecast one particular closed‐door decision: the election of the pope. Within the context of 500 years of papal election betting, we employ a unique dataset of betting on the 2013 papal election to investigate how new public information is incorporated into the betting odds. Our results suggest that the market was generally unable to incorporate effectively such information. We venture some possible explanations for our findings and offer suggestions for further research into the prediction and predictability of other ‘closed‐door’ decisions. Copyright © 2015 John Wiley & Sons, Ltd.     

The accuracy of IMF and OECD forecasts for G7 countries

This paper analyses the size and nature of the errors in GDP forecasts in the G7 countries from 1971 to 1995. These GDP short‐term forecasts are produced by the Organization for Economic Cooperation and Development and by the International Monetary Fund, and published twice a year in the Economic Outlook and in the World Economic Outlook, respectively. The evaluation of the accuracy of the forecasts is based on the properties of the difference between the realization and the forecast. A forecast is considered to be accurate if it is unbiased and efficient. A forecast is unbiased if its average deviation from the outcome is zero, and it is efficient if it reflects all the information that is available at the time the forecast is made. Finally, we also examine tests of directional accuracy and offer a non‐parametric method of assessment. Copyright © 2000 John Wiley & Sons, Ltd.     

Predicting returns and volatility with macroeconomic variables: evidence from tests of encompassing

Tests of forecast encompassing are used to evaluate one‐step‐ahead forecasts of S&P Composite index returns and volatility. It is found that forecasts over the 1990s made from models that include macroeconomic variables tend to be encompassed by those made from a benchmark model which does not include macroeconomic variables. However, macroeconomic variables are found to add significant information to forecasts of returns and volatility over the 1970s. Often in empirical research on forecasting stock index returns and volatility, in‐sample information criteria are used to rank potential forecasting models. Here, none of the forecasting models for the 1970s that include macroeconomic variables are, on the basis of information criteria, preferred to the relevant benchmark specification. Thus, had investors used information criteria to choose between the models used for forecasting over the 1970s considered in this paper, the predictability that tests of encompassing reveal would not have been exploited. Copyright © 2005 John Wiley & Sons, Ltd.     

Yield curve forecast combinations based on bond portfolio performance

We propose an economically motivated forecast combination strategy in which model weights are related to portfolio returns obtained by a given forecast model. An empirical application based on an optimal mean–variance bond portfolio problem is used to highlight the advantages of the proposed approach with respect to combination methods based on statistical measures of forecast accuracy. We compute average net excess returns, standard deviation, and the Sharpe ratio of bond portfolios obtained with nine alternative yield curve specifications, as well as with 12 different forecast combination strategies. Return‐based forecast combination schemes clearly outperformed approaches based on statistical measures of forecast accuracy in terms of economic criteria. Moreover, return‐based approaches that dynamically select only the model with highest weight each period and discard all other models delivered even better results, evidencing not only the advantages of trimming forecast combinations but also the ability of the proposed approach to detect best‐performing models. To analyze the robustness of our results, different levels of risk aversion and a different dataset are considered.     

Comparing the DSGE model with the factor model: an out‐of‐sample forecasting experiment

In this paper, we put dynamic stochastic general equilibrium DSGE forecasts in competition with factor forecasts. We focus on these two models since they represent nicely the two opposing forecasting philosophies. The DSGE model on the one hand has a strong theoretical economic background; the factor model on the other hand is mainly data‐driven. We show that incorporating a large information set using factor analysis can indeed improve the short‐horizon predictive ability, as claimed by many researchers. The micro‐founded DSGE model can provide reasonable forecasts for US inflation, especially with growing forecast horizons. To a certain extent, our results are consistent with the prevailing view that simple time series models should be used in short‐horizon forecasting and structural models should be used in long‐horizon forecasting. Our paper compares both state‐of‐the‐art data‐driven and theory‐based modelling in a rigorous manner. Copyright © 2008 John Wiley & Sons, Ltd.     

Evaluating forecasts: a look at aggregate bias and accuracy measures

In this paper an investigation is made of the properties and use of two aggregate measures of forecast bias and accuracy. These are metrics used in business to calculate aggregate forecasting performance for a family (group) of products. We find that the aggregate measures are not particularly informative if some of the one‐step‐ahead forecasts are biased. This is likely to be the case in practice if frequently employed forecasting methods are used to generate a large number of individual forecasts. In the paper, examples are constructed to illustrate some potential problems in the use of the metrics. We propose a simple graphical display of forecast bias and accuracy to supplement the information yielded by the accuracy measures. This support includes relevant boxplots of measures of individual forecasting success. This tool is simple but helpful as the graphic display has the potential to indicate forecast deterioration that can be masked by one or both of the aggregate metrics. The procedures are illustrated with data representing sales of food items. Copyright © 2005 John Wiley & Sons, Ltd.     

Geopolitical risk and global financial cycle: Some forecasting experiments

In this study, we investigate the connection between geopolitical risk (GPR) and global financial cycle (GFCy) as well as whether the former has predictive value for the out-of-sample predictability of the latter. We utilize both the historical and recent GPR data and their variants, namely, GPR act covering all “acts” that constitute GPR such as war, nuclear invasion and terrorism, and GPR threat, which represents threats of these acts. We construct a predictive model that accommodates the salient features of the predicted and predictor series while the forecast evaluation is conducted for both in-sample and out-of-sample periods. Our findings reveal that a rise in GPR discourages investments in risky assets and by implication worsens GFCy. The impact is more severe after the global financial crisis (gfc), and the GPR threat exerts more adverse effect on GFCy compared with GPR act regardless of whether historical GPR or recent GPR is used. Meanwhile, the predictive model of GFCy that accommodates the GPR data outperforms the benchmark model that ignores it both in the in-sample and out-of-sample estimates albeit with improved forecast performance during the post-gfc period and at a longer forecast horizon. However, the recent GPR data, which are broader in scope, offer better forecast accuracy than the historical GPR data. Additional analyses involving the vulnerability of global economic conditions reveal similar outcomes as GFCy.     

Mincer–Zarnowitz quantile and expectile regressions for forecast evaluations under aysmmetric loss functions

Forecasts are pervasive in all areas of applications in business and daily life. Hence evaluating the accuracy of a forecast is important for both the generators and consumers of forecasts. There are two aspects in forecast evaluation: (a) measuring the accuracy of past forecasts using some summary statistics, and (b) testing the optimality properties of the forecasts through some diagnostic tests. On measuring the accuracy of a past forecast, this paper illustrates that the summary statistics used should match the loss function that was used to generate the forecast. If there is strong evidence that an asymmetric loss function has been used in the generation of a forecast, then a summary statistic that corresponds to that asymmetric loss function should be used in assessing the accuracy of the forecast instead of the popular root mean square error or mean absolute error. On testing the optimality of the forecasts, it is demonstrated how the quantile regressions set in the prediction–realization framework of Mincer and Zarnowitz (in J. Mincer (Ed.), Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance (pp. 14–20), 1969) can be used to recover the unknown parameter that controls the potentially asymmetric loss function used in generating the past forecasts. Finally, the prediction–realization framework is applied to the Federal Reserve's economic growth forecast and forecast sharing in a PC manufacturing supply chain. It is found that the Federal Reserve values overprediction approximately 1.5 times more costly than underprediction. It is also found that the PC manufacturer weighs positive forecast errors (under forecasts) about four times as costly as negative forecast errors (over forecasts).     

Bootstrap prediction bands for forecast paths from vector autoregressive models

The problem of forecasting from vector autoregressive models has attracted considerable attention in the literature. The most popular non‐Bayesian approaches use either asymptotic approximations or bootstrapping to evaluate the uncertainty associated with the forecast. The practice in the empirical literature has been to assess the uncertainty of multi‐step forecasts by connecting the intervals constructed for individual forecast periods. This paper proposes a bootstrap method of constructing prediction bands for forecast paths. The bands are constructed from forecast paths obtained in bootstrap replications using an optimization procedure to find the envelope of the most concentrated paths. From extensive Monte Carlo study, it is found that the proposed method provides more accurate assessment of predictive uncertainty from the vector autoregressive model than its competitors. Copyright © 2010 John Wiley & Sons, Ltd.     

Does a lot help a lot? Forecasting stock returns with pooling strategies in a data‐rich environment

A variety of recent studies provide a skeptical view on the predictability of stock returns. Empirical evidence shows that most prediction models suffer from a loss of information, model uncertainty, and structural instability by relying on low‐dimensional information sets. In this study, we evaluate the predictive ability of various lately refined forecasting strategies, which handle these issues by incorporating information from many potential predictor variables simultaneously. We investigate whether forecasting strategies that (i) combine information and (ii) combine individual forecasts are useful to predict US stock returns, that is, the market excess return, size, value, and the momentum premium. Our results show that methods combining information have remarkable in‐sample predictive ability. However, the out‐of‐sample performance suffers from highly volatile forecast errors. Forecast combinations face a better bias–efficiency trade‐off, yielding a consistently superior forecast performance for the market excess return and the size premium even after the 1970s.     

Foreign exchange market prediction with multiple classifiers

Foreign exchange market prediction is attractive and challenging. According to the efficient market and random walk hypotheses, market prices should follow a random walk pattern and thus should not be predictable with more than about 50% accuracy. In this article, we investigate the predictability of foreign exchange spot rates of the US dollar against the British pound to show that not all periods are equally random. We used the Hurst exponent to select a period with great predictability. Parameters for generating training patterns were determined heuristically by auto‐mutual information and false nearest‐neighbor methods. Some inductive machine‐learning classifiers—artificial neural network, decision tree, k‐nearest neighbor, and naïve Bayesian classifier—were then trained with these generated patterns. Through appropriate collaboration of these models, we achieved a prediction accuracy of up to 67%. Copyright © 2009 John Wiley & Sons, Ltd.     

Non-linear prediction of security returns with moving average rules

Over the years, investors and the technical analysts have devised hundreds of technical market indicators in an effort to forecast the trend of a security market. Recent literature provides evidence that these rules may provide positive profits after accounting for transaction costs. This clearly contradicts the theory of the efficient market hypothesis which states that security prices cannot be forecasted from their past values or other past variables. This paper uses the daily Dow Jones Industrial Average Index from January 1963 to June 1988 to examine the linear and non-linear predictability of stock market returns with buy—sell signals generated from the moving average rules with a band between the short and the long averages. Strong evidence of non-linear predictability is found in the stock market returns by using the past buy and sell signals of these rules.     

Economic factors and the stock market: a new perspective

It has been widely accepted that many financial and economic variables are non‐linear, and neural networks can model flexible linear or non‐linear relationships among variables. The present paper deals with an important issue: Can the many studies in the finance literature evidencing predictability of stock returns by means of linear regression be improved by a neural network? We show that the predictive accuracy can be improved by a neural network, and the results largely hold out‐of‐sample. Both the neural network and linear forecasts show significant market timing ability. While the switching portfolio based on the linear forecasts outperforms the buy‐and‐hold market portfolio under all three transaction cost scenarios, the switching portfolio based on the neural network forecasts beats the market only if there is no transaction cost. Copyright © 1999 John Wiley & Sons, Ltd.