Economic and statistical measures of forecast accuracy

This paper argues in favour of a closer link between the decision and the forecast evaluation problems. Although the idea of using decision theory for forecast evaluation appears early in the dynamic stochastic programming literature, and has continued to be used with meteorological forecasts, it is hardly mentioned in standard academic texts on economic forecasting. Some of the main issues involved are illustrated in the context of a two‐state, two‐action decision problem as well as in a more general setting. Relationships between statistical and economic methods of forecast evaluation are discussed and links between the Kuipers score used as a measure of forecast accuracy in the meteorology literature and the market timing tests used in finance are established. An empirical application to the problem of stock market predictability is also provided, and the conditions under which such predictability could be explained in the presence of transaction costs are discussed. Copyright © 2000 John Wiley & Sons, Ltd.     

An intelligent model selection and forecasting system

In this paper we present an intelligent decision‐support system based on neural network technology for model selection and forecasting. While most of the literature on the application of neural networks in forecasting addresses the use of neural network technology as an alternative forecasting tool, limited research has focused on its use for selection of forecasting methods based on time‐series characteristics. In this research, a neural network‐based decision support system is presented as a method for forecast model selection. The neural network approach provides a framework for directly incorporating time‐series characteristics into the model‐selection phase. Using a neural network, a forecasting group is initially selected for a given data set, based on a set of time‐series characteristics. Then, using an additional neural network, a specific forecasting method is selected from a pool of three candidate methods. The results of training and testing of the networks are presented along with conclusions. Copyright © 1999 John Wiley & Sons, Ltd.     

Some practical aspects of forecasting in organizations

This paper presents the writer's experience, over a period of 25 years, in analysing organizational systems and, in particular, concentrates on the overall forecasting activity. The paper first looks at the relationship between forecasting and decision taking–with emphasis on the fact that forecasting is a means to aid decision taking and not an end in itself. It states that there are many types of forecasting problems, each requiring different methods of treatment. The paper then discusses attitudes which are emerging about the relative advantages of different forecasting techniques. It suggests a model building process which requires‘experience’and‘craftsmanship’, extensive practical application, frequent interaction between theory and practice and a methodology that eventually leads to models that contain no detectable inadequacies. Furthermore, it argues that although models which forecast a time series from its past history have a very important role to play, for effective policy making it is necessary to augment the model by introducing policy variables, again in a systematic not an ‘ad hoc’ manner. Finally, the paper discusses how forecasting systems can be introduced into the management process in the first place and how they should be monitored and updated when found wanting.     

Modeling of frequency containment reserve prices with econometrics and artificial intelligence

The forecasting of prices for electricity balancing reserve power can essentially improve the trading positions of market participants in competitive auctions. Having identified a lack of literature related to forecasting balancing reserve prices, we deploy approaches originating from econometrics and artificial intelligence and set up a forecasting framework based on autoregressive and exogenous factors. We use SARIMAX models as well as neural networks with different structures and forecast based on a rolling one-step forecast with reestimation of the models. It turns out that the naive forecast performs reasonably well but is outperformed by the more advanced models. In addition, neural network approaches outperform the econometric approach in terms of forecast quality, whereas for the further use of the generated models the econometric approach has advantages in terms of explaining price drivers. For the present application, more advanced configurations of the neural networks are not able to further improve the forecasting performance.     

Decision‐Based Forecast Evaluation of UK Interest Rate Predictability

This paper illustrates the importance of density forecasting and forecast evaluation in portfolio decision making. The decision‐making environment is fully described for an investor seeking to optimally allocate her portfolio between long and short Treasury bills, over investment horizons of up to 2 years. We examine the impact of parameter uncertainty and predictability in bond returns on the investor's allocation and we describe how the forecasts are computed and used in this context. Both statistical and decision‐based criteria are used to assess the predictability of returns. Our results show sensitivity to the evaluation criterion used and, in the context of investment decision making under an economic value criterion, we find some potential gain for the investor from assuming predictability. Copyright © 2015 John Wiley & Sons, Ltd.     

Regression‐based modeling of market option prices: with application to S&P500 options

This paper presents a simple empirical approach to modeling and forecasting market option prices using localized option regressions (LOR). LOR projects market option prices over localized regions of their state space and is robust to assumptions regarding the underlying asset dynamics (e.g. log‐normality) and volatility structure. Our empirical study using 3 years of daily S&P500 options shows that LOR yields smaller out‐of‐sample pricing errors (e.g. 32% 1‐day‐out) relative to an efficient benchmark from the literature and produces option prices free of the volatility smile. In addition to being an efficient and robust option‐modeling and valuation tool for large option books, LOR provides a simple‐to‐implement empirical benchmark for evaluating more complex risk‐neutral models. Copyright © 2007 John Wiley & Sons, Ltd.     

Dynamic relationship analysis of us gasoline and crude oil prices

This paper studies the dynamic relationships between US gasoline prices, crude oil prices, and the stock of gasoline. Using monthly data between January 1973 and December 1987, we find that the US gasoline price is mainly influenced by the price of crude oil. The stock of gasoline has little or no influence on the price of gasoline during the period before the second energy crisis, and seems to have some influence during the period after. We also find that the dynamic relationship between the prices of gasoline and crude oil changes over time, shifting from a longer lag response to a shorter lag response. Box-Jenkins ARIMA and transfer function models are employed in this study. These models are estimated using estimation procedure with and without outlier adjustment. For model estimation with outlier adjustment, an iterative procedure for the joint estimation of model parameters and outlier effects is employed. The forecasting performance of these models is carefully examined. For the purpose of illustration, we also analyze these time series using classical white-noise regression models. The results show the importance of using appropriate time-series methods in modeling and forecasting when the data are serially correlated. This paper also demonstrates the problems of time-series modeling when outliers are present.     

Forecasting time‐varying covariance with a robust Bayesian threshold model

This paper proposes a robust multivariate threshold vector autoregressive model with generalized autoregressive conditional heteroskedasticities and dynamic conditional correlations to describe conditional mean, volatility and correlation asymmetries in financial markets. In addition, the threshold variable for regime switching is formulated as a weighted average of endogenous variables to eliminate excessively subjective belief in the threshold variable decision and to serve as the proxy in deciding which market should be the price leader. The estimation is performed using Markov chain Monte Carlo methods. Furthermore, several meaningful criteria are introduced to assess the forecasting performance in the conditional covariance matrix. The proposed methodology is illustrated using daily S&P500 futures and spot prices. Copyright © 2010 John Wiley & Sons, Ltd.     

Short‐term salmon price forecasting

This study establishes a benchmark for short‐term salmon price forecasting. The weekly spot price of Norwegian farmed Atlantic salmon is predicted 1–5 weeks ahead using data from 2007 to 2014. Sixteen alternative forecasting methods are considered, ranging from classical time series models to customized machine learning techniques to salmon futures prices. The best predictions are delivered by k‐nearest neighbors method for 1 week ahead; vector error correction model estimated using elastic net regularization for 2 and 3 weeks ahead; and futures prices for 4 and 5 weeks ahead. While the nominal gains in forecast accuracy over a naïve benchmark are small, the economic value of the forecasts is considerable. Using a simple trading strategy for timing the sales based on price forecasts could increase the net profit of a salmon farmer by around 7%.     

Forecasting euro area inflation using dynamic factor measures of underlying inflation

Standard measures of prices are often contaminated by transitory shocks. This has prompted economists to suggest the use of measures of underlying inflation to formulate monetary policy and assist in forecasting observed inflation. Recent work has concentrated on modelling large data sets using factor models. In this paper we estimate factors from data sets of disaggregated price indices for European countries. We then assess the forecasting ability of these factor estimates against other measures of underlying inflation built from more traditional methods. The power to forecast headline inflation over horizons of 12 to 18 months is adopted as a valid criterion to assess forecasting. Empirical results for the five largest euro area countries, as well as for the euro area itself, are presented. Copyright © 2005 John Wiley & Sons, Ltd.     

Can panel data really improve the predictability of the monetary exchange rate model?

A common explanation for the inability of the monetary model to beat the random walk in forecasting future exchange rates is that conventional time series tests may have low power, and that panel data should generate more powerful tests. This paper provides an extensive evaluation of this power argument to the use of panel data in the forecasting context. In particular, by using simulations it is shown that although pooling of the individual prediction tests can lead to substantial power gains, pooling only the parameters of the forecasting equation, as has been suggested in the previous literature, does not seem to generate more powerful tests. The simulation results are illustrated through an empirical application. Copyright © 2007 John Wiley & Sons, Ltd.     

Time series forecasting using functional partial least square regression with stochastic volatility,GARCH, and exponential smoothing

We propose a method for improving the predictive ability of standard forecasting models used in financial economics. Our approach is based on the functional partial least squares (FPLS) model, which is capable of avoiding multicollinearity in regression by efficiently extracting information from the high‐dimensional market data. By using its well‐known ability, we can incorporate auxiliary variables that improve the predictive accuracy. We provide an empirical application of our proposed methodology in terms of its ability to predict the conditional average log return and the volatility of crude oil prices via exponential smoothing, Bayesian stochastic volatility, and GARCH (generalized autoregressive conditional heteroskedasticity) models, respectively. In particular, what we call functional data analysis (FDA) traces in this article are obtained via the FPLS regression from both the crude oil returns and auxiliary variables of the exchange rates of major currencies. For forecast performance evaluation, we compare out‐of‐sample forecasting accuracy of the standard models with FDA traces to the accuracy of the same forecasting models with the observed crude oil returns, principal component regression (PCR), and least absolute shrinkage and selection operator (LASSO) models. We find evidence that the standard models with FDA traces significantly outperform our competing models. Finally, they are also compared with the test for superior predictive ability and the reality check for data snooping. Our empirical results show that our new methodology significantly improves predictive ability of standard models in forecasting the latent average log return and the volatility of financial time series.     

Changing time scale for short-term forecasting in financial markets

A forecasting model based on high-frequency market makers quotes of financial instruments is presented. The statistical behaviour of these time series leads to discussion of the appropriate time scale for forecasting. We introduce variable time scales in a general way and define the new concept of intrinsic time. The latter reflects better the actual trading activity. Changing time scale means forecasting in two steps, first an intrinsic time forecast against physical time, then a price forecast against intrinsic time. The forecasting model consists, for both steps, of a linear combination of non-linear price-based indicators. The indicator weights are continuously re-optimized through a modified linear regression on a moving sample of past prices. The out-of-sample performance of this algorithm is reported on a set of important FX rates and interest rates over many years. It is remarkably consistent. Results for short horizons as well as techniques to measure this performance are discussed.     

Decision trees for forecasting

Recent years have seen an increasing cross-fertilization between the fields of decision analysis and forecasting. Decision-analytic models often require forecasts as inputs, and aspects of the Bayesian decision-theoretic framework underlying decision analysis have proved useful to forecasting, particularly in contexts where subjective judgemental inputs are required. This paper describes the use of decision tree analysis for forecasting and illustrates its use for corporate divisional forecasting and planning. A specialized decision-analytic technique, acts as events, is also described and illustrated to forecast a new product's earnings. Conclusions are drawn about the applicability of decision analysis for forecasting.     

Dynamic Model Averaging and CPI Inflation Forecasts: A Comparison between the Euro Area and the United States

The paper forecasts consumer price inflation in the euro area (EA) and in the USA between 1980:Q1 and 2012:Q4 based on a large set of predictors, with dynamic model averaging (DMA) and dynamic model selection (DMS). DMA/DMS allows not solely for coefficients to change over time, but also for changes in the entire forecasting model over time. DMA/DMS provides on average the best inflation forecasts with regard to alternative approaches (such as the random walk). DMS outperforms DMA. These results are robust for different sample periods and for various forecast horizons. The paper highlights common features between the USA and the EA. First, two groups of predictors forecast inflation: temporary fundamentals that have a frequent impact on inflation but only for short time periods; and persistent fundamentals whose switches are less frequent over time. Second, the importance of some variables (particularly international food commodity prices, house prices and oil prices) as predictors for consumer price index inflation increases when such variables experience large shocks. The paper also shows that significant differences prevail in the forecasting models between the USA and the EA. Such differences can be explained by the structure of these respective economies. Copyright © 2015 John Wiley & Sons, Ltd.     

Evidence for the selection of forecasting methods

Reid (1972) was among the first to argue that the relative accuracy of forecasting methods changes according to the properties of the time series. Comparative analyses of forecasting performance such as the M‐Competition tend to support this argument. The issue addressed here is the usefulness of statistics summarizing the data available in a time series in predicting the relative accuracy of different forecasting methods. Nine forecasting methods are described and the literature suggesting summary statistics for choice of forecasting method is summarized. Based on this literature and further argument a set of these statistics is proposed for the analysis. These statistics are used as explanatory variables in predicting the relative performance of the nine methods using a set of simulated time series with known properties. These results are evaluated on observed data sets, the M‐Competition data and Fildes Telecommunications data. The general conclusion is that the summary statistics can be used to select a good forecasting method (or set of methods) but not necessarily the best. Copyright © 2000 John Wiley & Sons, Ltd.     

Towards a taxonomy of forecast error measures a factor-comparative investigation of forecast error dimensions

This study explores the nature of information conveyed by 14 error measures drawn from the literature, using real-life forecasting data from 691 individual product items over six quarterly periods. Principal components analysis is used to derive factor solutions that are subsequently compared for two forecasting methods, a version of Holt's exponential smoothing, and the random walk model (Naive 1). The results reveal four underlying forecast error dimensions that are stable across the two factor solutions. The potentially confounding influence of sales volume on the derived error dimensions is also explored via correlation analysis.     

An Adaptive Multiscale Ensemble Learning Paradigm for Nonstationary and Nonlinear Energy Price Time Series Forecasting

For forecasting nonstationary and nonlinear energy prices time series, a novel adaptive multiscale ensemble learning paradigm incorporating ensemble empirical mode decomposition (EEMD), particle swarm optimization (PSO) and least square support vector machines (LSSVM) with kernel function prototype is developed. Firstly, the extrema symmetry expansion EEMD, which can effectively restrain the mode mixing and end effects, is used to decompose the energy price into simple modes. Secondly, by using the fine‐to‐coarse reconstruction algorithm, the high‐frequency, low‐frequency and trend components are identified. Furthermore, autoregressive integrated moving average is applicable to predicting the high‐frequency components. LSSVM is suitable for forecasting the low‐frequency and trend components. At the same time, a universal kernel function prototype is introduced for making up the drawbacks of single kernel function, which can adaptively select the optimal kernel function type and model parameters according to the specific data using the PSO algorithm. Finally, the prediction results of all the components are aggregated into the forecasting values of energy price time series. The empirical results show that, compared with the popular prediction methods, the proposed method can significantly improve the prediction accuracy of energy prices, with high accuracy both in the level and directional predictions. Copyright © 2016 John Wiley & Sons, Ltd.     

Optimal forecast combination based on ensemble empirical mode decomposition for agricultural commodity futures prices

Improving the prediction accuracy of agricultural product futures prices is important for investors, agricultural producers, and policymakers. This is to evade risks and enable government departments to formulate appropriate agricultural regulations and policies. This study employs the ensemble empirical mode decomposition (EEMD) technique to decompose six different categories of agricultural futures prices. Subsequently, three models—support vector machine (SVM), neural network (NN), and autoregressive integrated moving average (ARIMA)—are used to predict the decomposition components. The final hybrid model is then constructed by comparing the prediction performance of the decomposition components. The predicting performance of the combination model is then compared with the benchmark individual models: SVM, NN, and ARIMA. Our main interest in this study is on short-term forecasting, and thus we only consider 1-day and 3-day forecast horizons. The results indicate that the prediction performance of the EEMD combined model is better than that of individual models, especially for the 3-day forecasting horizon. The study also concluded that the machine learning methods outperform the statistical methods in forecasting high-frequency volatile components. However, there is no obvious difference between individual models in predicting low-frequency components.     

Forecasting the price of crude oil via convenience yield predictions

The paper develops an oil price forecasting technique which is based on the present value model of rational commodity pricing. The approach suggests shifting the forecasting problem to the marginal convenience yield, which can be derived from the cost‐of‐carry relationship. In a recursive out‐of‐sample analysis, forecast accuracy at horizons within one year is checked by the root mean squared error as well as the mean error and the frequency of a correct direction‐of‐change prediction. For all criteria employed, the proposed forecasting tool outperforms the approach of using futures prices as direct predictors of future spot prices. Vis‐à‐vis the random‐walk model, it does not significantly improve forecast accuracy but provides valuable statements on the direction of change. Copyright © 2007 John Wiley & Sons, Ltd.