Evaluating volatility and interval forecasts

A widely used approach to evaluating volatility forecasts uses a regression framework which measures the bias and variance of the forecast. We show that the associated test for bias is inappropriate before introducing a more suitable procedure which is based on the test for bias in a conditional mean forecast. Although volatility has been the most common measure of the variability in a financial time series, in many situations confidence interval forecasts are required. We consider the evaluation of interval forecasts and present a regression‐based procedure which uses quantile regression to assess quantile estimator bias and variance. We use exchange rate data to illustrate the proposal by evaluating seven quantile estimators, one of which is a new non‐parametric autoregressive conditional heteroscedasticity quantile estimator. The empirical analysis shows that the new evaluation procedure provides useful insight into the quality of quantile estimators. Copyright © 1999 John Wiley & Sons, Ltd.     

Quantile forecasting for credit risk management using possibly misspecified hidden Markov models

Recent models for credit risk management make use of hidden Markov models (HMMs). HMMs are used to forecast quantiles of corporate default rates. Little research has been done on the quality of such forecasts if the underlying HMM is potentially misspecified. In this paper, we focus on misspecification in the dynamics and dimension of the HMM. We consider both discrete‐ and continuous‐state HMMs. The differences are substantial. Underestimating the number of discrete states has an economically significant impact on forecast quality. Generally speaking, discrete models underestimate the high‐quantile default rate forecasts. Continuous‐state HMMs, however, vastly overestimate high quantiles if the true HMM has a discrete state space. In the reverse setting the biases are much smaller, though still substantial in economic terms. We illustrate the empirical differences using US default data. Copyright © 2008 John Wiley & Sons, Ltd.     

Out‐of‐sample equity premium prediction: A scenario analysis approach

We propose two methods of equity premium prediction with single and multiple predictors respectively and evaluate their out‐of‐sample performance using US stock data with 15 popular predictors for equity premium prediction. The first method defines three scenarios in terms of the expected returns under the scenarios and assumes a Markov chain governing the occurrence of the scenarios over time. It employs predictive quantile regressions of excess return on a predictor for three quantiles to estimate the occurrence of the scenarios over an in‐sample period and the transition probabilities of the Markov chain, predicts the expected returns under the scenarios, and generates an equity premium forecast by combining the predicted expected returns under three scenarios with the estimated transition probabilities. The second method generates an equity premium forecast by combining the individual forecasts from the first method across all predictors. For most of predictors, the first method outperforms the benchmark method of historical average and the traditional predictive linear regression with a single predictor both statistically and economically, and the second method consistently performs better than several competing methods used in the literature. The performance of our methods is further examined under different scenarios and economic conditions, and is robust for two different out‐of‐sample periods and specifications of the scenarios.     

Predictable Return Distributions

Using quantile regression this paper explores the predictability of the stock and bond return distributions as a function of economic state variables. The use of quantile regression allows us to examine specific parts of the return distribution such as the tails and the center, and for a sufficiently fine grid of quantiles we can trace out the entire distribution. A univariate quantile regression model is used to examine the marginal stock and bond return distributions, while a multivariate model is used to capture their joint distribution. An empirical analysis on US data shows that economic state variables predict the stock and bond return distributions in quite different ways in terms of, for example, location shifts, volatility and skewness. Comparing the different economic state variables in terms of their out‐of‐sample forecasting performance, the empirical analysis also shows that the relative accuracy of the state variables varies across the return distribution. Density forecasts based on an assumed normal distribution with forecasted mean and variance is compared to forecasts based on quantile estimates and, in general, the latter yields the best performance. Copyright © 2015 John Wiley & Sons, Ltd.     

A Quantile Regression Approach to Equity Premium Prediction

We propose a quantile regression approach to equity premium forecasting. Robust point forecasts are generated from a set of quantile forecasts using both fixed and time‐varying weighting schemes, thereby exploiting the entire distributional information associated with each predictor. Further gains are achieved by incorporating the forecast combination methodology into our quantile regression setting. Our approach using a time‐varying weighting scheme delivers statistically and economically significant out‐of‐sample forecasts relative to both the historical average benchmark and the combined predictive mean regression modeling approach. Copyright © 2014 John Wiley & Sons, Ltd.     

Mincer–Zarnowitz quantile and expectile regressions for forecast evaluations under aysmmetric loss functions

Forecasts are pervasive in all areas of applications in business and daily life. Hence evaluating the accuracy of a forecast is important for both the generators and consumers of forecasts. There are two aspects in forecast evaluation: (a) measuring the accuracy of past forecasts using some summary statistics, and (b) testing the optimality properties of the forecasts through some diagnostic tests. On measuring the accuracy of a past forecast, this paper illustrates that the summary statistics used should match the loss function that was used to generate the forecast. If there is strong evidence that an asymmetric loss function has been used in the generation of a forecast, then a summary statistic that corresponds to that asymmetric loss function should be used in assessing the accuracy of the forecast instead of the popular root mean square error or mean absolute error. On testing the optimality of the forecasts, it is demonstrated how the quantile regressions set in the prediction–realization framework of Mincer and Zarnowitz (in J. Mincer (Ed.), Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance (pp. 14–20), 1969) can be used to recover the unknown parameter that controls the potentially asymmetric loss function used in generating the past forecasts. Finally, the prediction–realization framework is applied to the Federal Reserve's economic growth forecast and forecast sharing in a PC manufacturing supply chain. It is found that the Federal Reserve values overprediction approximately 1.5 times more costly than underprediction. It is also found that the PC manufacturer weighs positive forecast errors (under forecasts) about four times as costly as negative forecast errors (over forecasts).     

Quantile estimators with orthogonal pinball loss function

To guarantee stable quantile estimations even for noisy data, a novel loss function and novel quantile estimators are developed, by introducing the effective concept of orthogonal loss considering the noise in both response and explanatory variables. In particular, the pinball loss used in classical quantile estimators is improved into novel orthogonal pinball loss (OPL) by replacing vertical loss by orthogonal loss. Accordingly, linear quantile regression (QR) and support vector machine quantile regression (SVMQR) can be respectively extended into novel OPL‐based QR and OPL‐based SVMQR models. The empirical study on 10 publicly available datasets statistically verifies the superiority of the two OPL‐based models over their respective original forms in terms of prediction accuracy and quantile property, especially for extreme quantiles. Furthermore, the novel OPL‐based SVMQR model with both OPL and artificial intelligence (AI) outperforms all benchmark models, which can be used as a promising quantile estimator, especially for noisy data.     

Point forecasting of intraday volume using Bayesian autoregressive conditional volume models

In this paper, we apply Bayesian inference to model and forecast intraday trading volume, using autoregressive conditional volume (ACV) models, and we evaluate the quality of volume point forecasts. In the empirical application, we focus on the analysis of both in‐ and out‐of‐sample performance of Bayesian ACV models estimated for 2‐minute trading volume data for stocks quoted on the Warsaw Stock Exchange in Poland. We calculate two types of point forecasts, using either expected values or medians of predictive distributions. We conclude that, in general, all considered models generate significantly biased forecasts. We also observe that the considered models significantly outperform such benchmarks as the naïve or rolling means forecasts. Moreover, in terms of root mean squared forecast errors, point predictions obtained within the ACV model with exponential distribution emerge superior compared to those calculated in structures with more general innovation distributions, although in many cases this characteristic turns out to be statistically insignificant. On the other hand, when comparing mean absolute forecast errors, the median forecasts obtained within the ACV models with Burr and generalized gamma distribution are found to be statistically better than other forecasts.     

Using CAViaR Models with Implied Volatility for Value‐at‐Risk Estimation

This paper proposes value‐at risk (VaR) estimation methods that are a synthesis of conditional autoregressive value at risk (CAViaR) time series models and implied volatility. The appeal of this proposal is that it merges information from the historical time series and the different information supplied by the market's expectation of risk. Forecast‐combining methods, with weights estimated using quantile regression, are considered. We also investigate plugging implied volatility into the CAViaR models—a procedure that has not been considered in the VaR area so far. Results for daily index returns indicate that the newly proposed methods are comparable or superior to individual methods, such as the standard CAViaR models and quantiles constructed from implied volatility and the empirical distribution of standardised residuals. We find that the implied volatility has more explanatory power as the focus moves further out into the left tail of the conditional distribution of S&P 500 daily returns. Copyright © 2012 John Wiley & Sons, Ltd.     

Evaluating predictive performance of value‐at‐risk models in emerging markets: a reality check

We investigate the predictive performance of various classes of value‐at‐risk (VaR) models in several dimensions—unfiltered versus filtered VaR models, parametric versus nonparametric distributions, conventional versus extreme value distributions, and quantile regression versus inverting the conditional distribution function. By using the reality check test of White (2000), we compare the predictive power of alternative VaR models in terms of the empirical coverage probability and the predictive quantile loss for the stock markets of five Asian economies that suffered from the 1997–1998 financial crisis. The results based on these two criteria are largely compatible and indicate some empirical regularities of risk forecasts. The Riskmetrics model behaves reasonably well in tranquil periods, while some extreme value theory (EVT)‐based models do better in the crisis period. Filtering often appears to be useful for some models, particularly for the EVT models, though it could be harmful for some other models. The CaViaR quantile regression models of Engle and Manganelli (2004) have shown some success in predicting the VaR risk measure for various periods, generally more stable than those that invert a distribution function. Overall, the forecasting performance of the VaR models considered varies over the three periods before, during and after the crisis. Copyright © 2006 John Wiley & Sons, Ltd.     

Quantile Double AR Time Series Models for Financial Returns

We develop a novel quantile double autoregressive model for modelling financial time series. This is done by specifying a generalized lambda distribution to the quantile function of the location‐scale double autoregressive model developed by Ling (2004, 2007). Parameter estimation uses Markov chain Monte Carlo Bayesian methods. A simulation technique is introduced for forecasting the conditional distribution of financial returns m periods ahead, and hence any for predictive quantities of interest. The application to forecasting value‐at‐risk at different time horizons and coverage probabilities for Dow Jones Industrial Average shows that our method works very well in practice. Copyright © 2013 John Wiley & Sons, Ltd.     

Exchange rate forecasting and the performance of currency portfolios

We examine the potential gains of using exchange rate forecast models and forecast combination methods in the management of currency portfolios for three exchange rates: the euro versus the US dollar, the British pound, and the Japanese yen. We use a battery of econometric specifications to evaluate whether optimal currency portfolios implied by trading strategies based on exchange rate forecasts outperform single currencies and the equally weighted portfolio. We assess the differences in profitability of optimal currency portfolios for different types of investor preferences, two trading strategies, mean squared error‐based composite forecasts, and different forecast horizons. Our results indicate that there are clear benefits of integrating exchange rate forecasts from state‐of‐the‐art econometric models in currency portfolios. These benefits vary across investor preferences and prediction horizons but are rather similar across trading strategies.     

Financial density forecasts: A comprehensive comparison of risk‐neutral and historical schemes

We investigate the forecasting ability of the most commonly used benchmarks in financial economics. We approach the usual caveats of probabilistic forecasts studies—small samples, limited models, and nonholistic validations—by performing a comprehensive comparison of 15 predictive schemes during a time period of over 21 years. All densities are evaluated in terms of their statistical consistency, local accuracy and forecasting errors. Using a new composite indicator, the integrated forecast score, we show that risk‐neutral densities outperform historical‐based predictions in terms of information content. We find that the variance gamma model generates the highest out‐of‐sample likelihood of observed prices and the lowest predictive errors, whereas the GARCH‐based GJR‐FHS delivers the most consistent forecasts across the entire density range. In contrast, lognormal densities, the Heston model, or the nonparametric Breeden–Litzenberger formula yield biased predictions and are rejected in statistical tests.     

Real‐time inflation forecast combination for time‐varying coefficient models

We use real‐time macroeconomic variables and combination forecasts with both time‐varying weights and equal weights to forecast inflation in the USA. The combination forecasts compare three sets of commonly used time‐varying coefficient autoregressive models: Gaussian distributed errors, errors with stochastic volatility, and errors with moving average stochastic volatility. Both point forecasts and density forecasts suggest that models combined by equal weights do not produce worse forecasts than those with time‐varying weights. We also find that variable selection, the allowance of time‐varying lag length choice, and the stochastic volatility specification significantly improve forecast performance over standard benchmarks. Finally, when compared with the Survey of Professional Forecasters, the results of the best combination model are found to be highly competitive during the 2007/08 financial crisis.     

Forecasting Australia's real house price index: A comparison of time series and machine learning methods

We employ 47 different algorithms to forecast Australian log real house prices and growth rates, and compare their ability to produce accurate out-of-sample predictions. The algorithms, which are specified in both single- and multi-equation frameworks, consist of traditional time series models, machine learning (ML) procedures, and deep learning neural networks. A method is adopted to compute iterated multistep forecasts from nonlinear ML specifications. While the rankings of forecast accuracy depend on the length of the forecast horizon, as well as on the choice of the dependent variable (log price or growth rate), a few generalizations can be made. For one- and two-quarter-ahead forecasts we find a large number of algorithms that outperform the random walk with drift benchmark. We also report several such outperformances at longer horizons of four and eight quarters, although these are not statistically significant at any conventional level. Six of the eight top forecasts (4 horizons × 2 dependent variables) are generated by the same algorithm, namely a linear support vector regressor (SVR). The other two highest ranked forecasts are produced as simple mean forecast combinations. Linear autoregressive moving average and vector autoregression models produce accurate olne-quarter-ahead predictions, while forecasts generated by deep learning nets rank well across medium and long forecast horizons.     

Forecasting Temperature Indices Density with Time‐Varying Long‐Memory Models

The hedging of weather risks has become extremely relevant in recent years, promoting the diffusion of weather‐derivative contracts. The pricing of such contracts requires the development of appropriate models for the prediction of the underlying weather variables. Within this framework, a commonly used specification is the ARFIMA‐GARCH. We provide a generalization of such a model, introducing time‐varying memory coefficients. Our model satisfies the empirical evidence of the changing memory level observed in average temperature series, and provides useful improvements in the forecasting, simulation, and pricing issues related to weather derivatives. We present an application related to the forecast and simulation of a temperature index density, which is then used for the pricing of weather options. Copyright © 2011 John Wiley & Sons, Ltd.     

The use of encompassing tests for forecast combinations

This paper proposes an algorithm that uses forecast encompassing tests for combining forecasts when there are a large number of forecasts that might enter the combination. The algorithm excludes a forecast from the combination if it is encompassed by another forecast. To assess the usefulness of this approach, an extensive empirical analysis is undertaken using a US macroeconomic dataset. The results are encouraging; the algorithm forecasts outperform benchmark model forecasts, in a mean square error (MSE) sense, in a majority of cases. The paper also compares the empirical performance of different approaches to forecast combination, and provides a rule‐of‐thumb cut‐off point for the thick‐modeling approach. Copyright © 2009 John Wiley & Sons, Ltd.     

Realized Volatility Forecast of Stock Index Under Structural Breaks

We investigate the realized volatility forecast of stock indices under the structural breaks. We utilize a pure multiple mean break model to identify the possibility of structural breaks in the daily realized volatility series by employing the intraday high‐frequency data of the Shanghai Stock Exchange Composite Index and the five sectoral stock indices in Chinese stock markets for the period 4 January 2000 to 30 December 2011. We then conduct both in‐sample tests and out‐of‐sample forecasts to examine the effects of structural breaks on the performance of ARFIMAX‐FIGARCH models for the realized volatility forecast by utilizing a variety of estimation window sizes designed to accommodate potential structural breaks. The results of the in‐sample tests show that there are multiple breaks in all realized volatility series. The results of the out‐of‐sample point forecasts indicate that the combination forecasts with time‐varying weights across individual forecast models estimated with different estimation windows perform well. In particular, nonlinear combination forecasts with the weights chosen based on a non‐parametric kernel regression and linear combination forecasts with the weights chosen based on the non‐negative restricted least squares and Schwarz information criterion appear to be the most accurate methods in point forecasting for realized volatility under structural breaks. We also conduct an interval forecast of the realized volatility for the combination approaches, and find that the interval forecast for nonlinear combination approaches with the weights chosen according to a non‐parametric kernel regression performs best among the competing models. Copyright © 2014 John Wiley & Sons, Ltd.     

Forecasting volatility

In this paper, we investigate the time series properties of S&P 100 volatility and the forecasting performance of different volatility models. We consider several nonparametric and parametric volatility measures, such as implied, realized and model‐based volatility, and show that these volatility processes exhibit an extremely slow mean‐reverting behavior and possible long memory. For this reason, we explicitly model the near‐unit root behavior of volatility and construct median unbiased forecasts by approximating the finite‐sample forecast distribution using bootstrap methods. Furthermore, we produce prediction intervals for the next‐period implied volatility that provide important information about the uncertainty surrounding the point forecasts. Finally, we apply intercept corrections to forecasts from misspecified models which dramatically improve the accuracy of the volatility forecasts. Copyright © 2006 John Wiley & Sons, Ltd.     

On the Predictive Content of Autoregression Residuals: A Semiparametric,Copula‐Based Approach to Time Series Prediction

This paper proposes an adjustment of linear autoregressive conditional mean forecasts that exploits the predictive content of uncorrelated model residuals. The adjustment is motivated by non‐Gaussian characteristics of model residuals, and implemented in a semiparametric fashion by means of conditional moments of simulated bivariate distributions. A pseudo ex ante forecasting comparison is conducted for a set of 494 macroeconomic time series recently collected by Dees et al. (Journal of Applied Econometrics 2007; 22: 1–38). In total, 10,374 time series realizations are contrasted against competing short‐, medium‐ and longer‐term purely autoregressive and adjusted predictors. With regard to all forecast horizons, the adjusted predictions consistently outperform conditionally Gaussian forecasts according to cross‐sectional mean group evaluation of absolute forecast errors and directional accuracy. Copyright © 2012 John Wiley & Sons, Ltd.