The consistency,coherence and calibration of holistic,decomposed and recomposed judgemental probability forecasts

In this paper we make an empirical investigation of the relationship between the consistency, coherence and validity of probability judgements in a real-world forecasting context. Our results indicate that these measures of the adequacy of an individual's probability assessments are not closely related as we anticipated. Twenty-nine of our thirty-six subjects were better calibrated in point probabilities than in odds and our subjects were, in general more coherent using point probabilities than odds forecasts. Contrary to our expectations we found very little difference in forecasting response and performance between simple and compound holistic forecasts. This result is evidence against the ‘divide-and-conquer’ rationale underlying most applications of normative decision theory. In addition, our recompositions of marginal and conditional assessments into compound forecasts were no better calibrated or resolved than their holistic counterparts. These findings convey two implications for forecasting. First, untrained judgemental forecasters should use point probabilities in preference to odds. Second, judgemental forecasts of complex compound probabilities may be as well assessed holistically as they are using methods of decomposition and recomposition. In addition, our study provides a paradigm for further studies of the relationship between consistency, coherence and validity in judgemental probability forecasting.     

Multiple prediction intervals for time series: Comparison of simultaneous and marginal intervals

Simultaneous prediction intervals for forecasts from time series models that contain L (L ≤ 1) unknown future observations with a specified probability are derived. Our simultaneous intervals are based on two types of probability inequalities, i.e. the Bonferroni- and product-types. These differ from the marginal intervals in that they take into account the correlation structure between the forecast errors. For the forecasting methods commonly used with seasonal time series data, we show how to construct forecast error correlations and evaluate, using an example, the simultaneous and marginal prediction intervals. For all the methods, the simultaneous intervals are accurate with the accuracy increasing with the use of higher-order probability inequalities, whereas the marginal intervals are far too short in every case. Also, when L is greater than the seasonal period, the simultaneous intervals based on improved probability inequalities will be most accurate.     

Time series forecasts and extra-model information

Often a forecaster has supplementary information (e.g. field reports or forecasts from another source) that cannot be included directly in a time series model. Especially interesting are cases where this information is given at time intervals that are different from those of the time series model forecasts. Previous authors have considered a numerical and a model-based statistical method for combining extra-model information of this type with ARIMA model forecasts. This paper extends both methods to vector ARMA model forecasts and dynamic regression (transfer function) model forecasts. It is also shown that a Lagrange multiplier numerical procedure arises as a special case of the model-based procedure. An empirical example is given.     

Forecasting volatility in commodity markets

This paper uses recent advances in time-series modeling to derive long-horizon forecasts of commodity price volatility which incorporate investors' expectations of volatility. Our results are promising. We compare several different forecasts of commodity price volatility, which we divide into three categories: (1) forecasts using only expectations derived from options prices; (2) forecasts using only time-series modeling; and (3) forecasts which combine market expectations and time-series methods. The forecasts in (1) and (2) are used extensively in the literature, while those in (3) are new in this paper. On comparing these different forecasts, we find that our proposed forecasts from category (3) outperform both market expectations forecasts and time-series forecasts. This result holds both in and out of sample for virtually all commodities considered.     

Evaluating the predictive accuracy of volatility models

Standard statistical loss functions, such as mean‐squared error, are commonly used for evaluating financial volatility forecasts. In this paper, an alternative evaluation framework, based on probability scoring rules that can be more closely tailored to a forecast user's decision problem, is proposed. According to the decision at hand, the user specifies the economic events to be forecast, the scoring rule with which to evaluate these probability forecasts, and the subsets of the forecasts of particular interest. The volatility forecasts from a model are then transformed into probability forecasts of the relevant events and evaluated using the selected scoring rule and calibration tests. An empirical example using exchange rate data illustrates the framework and confirms that the choice of loss function directly affects the forecast evaluation results. Copyright © 2001 John Wiley & Sons, Ltd.     

One-sided Simultaneous Prediction Intervals for AR(1) and MA(1) Processes with Exponential Innovations

This paper is concerned with the determination of simultaneous confidence regions for some types of time series models. We derive recursive formulas which allow the determination of the probability for an AR(1) stationary process based on exponential inputs to lie under any sequence of constants during N steps. Also, probabilities of the same form are derived for an MA(1) process, based on an exponentially distributed white noise. Numerical results are obtained and comparison of prediction regions for different values of ϕ or θ is made. The results show how the use of the correlation structure of the models can reduce the confidence regions area. © 1997 by John Wiley & Sons, Ltd.     

Cryptocurrency volatility forecasting: A Markov regime-switching MIDAS approach

The primary purpose of this paper is to investigate whether a novel Markov regime-switching mixed-data sampling (MRS-MIADS) model we design can improve the prediction accuracy of the realized variance (RV) of Bitcoin. Moreover, to verify whether the importance of jumps for RV forecasting changes over time, we extend the standard MIDAS model to characterize two volatility regimes and introduce a jump-driven time-varying transition probability between the two regimes. Our results suggest that the proposed novel MRS-MIDAS model exhibits statistically significant improvement for forecasting the RV of Bitcoin. In addition, we find that jump occurrences significantly increase the persistence of the high-volatility regime and switch between high- and low-volatility regimes. A wide range of checks confirm the robustness of our results. Finally, the proposed model shows significant improvement for 2-week and 1-month horizon forecasts.     

Forecasting Interest Rates and Yield Spreads: The Informational Content of Implied Futures Yields and Best-fitting Forward Rate Models

Forecasts of interest rates for different maturities are essential for forecasts of asset prices. The growth of derivatives markets coupled with the development of complex theories of the term structure of interest rates have provided forecasters with a rich array of variables for predicting interest rates and yield spreads. This paper extends previous work on forecasting future interest rates and yield spreads using market data for T-bills, T-Notes, and Treasury Bond spot and futures contracts. The information conveyed in technical models that use market data is also assessed, using a recent innovation in interest rate modelling, the maximum smoothness approach. Forecasts from this model are compared with predicted yields and yield spreads derived from futures prices as well as with those of the random walk model. The results show some evidence of market segmentation, with more arbitrage evident for nearby maturities. Market participants appear to show a greater degree of consensus on short-term interest rates than on longer-term interest rates. There is some indication that forecasts from the futures markets are marginally better than those provided by those of the maximum-smoothness approach, consistent with the informational advantages of futures markets. Finally, futures and maximum-smoothness market forecasts are shown to outperform those of the random walk model.© 1997 John Wiley & Sons, Ltd.     

Spurious Forecasts?

P. C. B. Phillips (1998) demonstrated that deterministic trends are a valid representation of an otherwise stochastic trending mechanism; he remained skeptic, however, about the predictive power of such representations. In this paper we prove that forecasts built upon spurious regression may perform (asymptotically) as well as those issued from a correctly specified regression. We derive the order in probability of several in‐sample and out‐of‐sample predictability criteria ( test, root mean square error, Theil's U‐statistics and R²) using forecasts based upon a least squares‐estimated regression between independent variables generated by a variety of empirically relevant data‐generating processes. It is demonstrated that, when the variables are mean stationary or trend stationary, the order in probability of these criteria is the same whether the regression is spurious or not. Simulation experiments confirm our asymptotic results. Copyright © 2011 John Wiley & Sons, Ltd.     

Empirical Bayesian Density Forecasting in Iowa and Shrinkage for the Monte Carlo Era

The track record of a 20‐year history of density forecasts of state tax revenue in Iowa is studied, and potential improvements sought through a search for better‐performing ‘priors’ similar to that conducted three decades ago for point forecasts by Doan, Litterman and Sims (Econometric Reviews, 1984). Comparisons of the point and density forecasts produced under the flat prior are made to those produced by the traditional (mixed estimation) ‘Bayesian VAR’ methods of Doan, Litterman and Sims, as well as to fully Bayesian ‘Minnesota Prior’ forecasts. The actual record and, to a somewhat lesser extent, the record of the alternative procedures studied in pseudo‐real‐time forecasting experiments, share a characteristic: subsequently realized revenues are in the lower tails of the predicted distributions ‘too often’. An alternative empirically based prior is found by working directly on the probability distribution for the vector autoregression parameters—the goal being to discover a better‐performing entropically tilted prior that minimizes out‐of‐sample mean squared error subject to a Kullback–Leibler divergence constraint that the new prior not differ ‘too much’ from the original. We also study the closely related topic of robust prediction appropriate for situations of ambiguity. Robust ‘priors’ are competitive in out‐of‐sample forecasting; despite the freedom afforded the entropically tilted prior, it does not perform better than the simple alternatives. Copyright © 2014 John Wiley & Sons, Ltd.     

A comparative evaluation of objective and subjective weather forecasts in the united states

This paper is concerned primarily with the evaluation and comparison of objective and subjective weather forecasts. Operational forecasts of three weather elements are considered: (1) probability forecasts of precipitation occurrence, (2) categorical (i.e. non-probabilistic) forecasts of maximum and minimum temperatures and (3) categorical forecasts of cloud amount. The objective forecasts are prepared by numerical-statistical procedures, whereas the subjective forecasts are based on the judgements of individual forecasters. In formulating the latter, the forecasters consult information from a variety of sources, including the objective forecasts themselves. The precipitation probability forecasts are found to be both reliable and skilful, and evaluation of the temperature/cloud amount forecasts reveals that they are quite accurate/skilful. Comparison of the objective and subjective forecasts of precipitation occurrence indicates that the latter are generally more skilful than the former for shorter lead times (e.g. 12–24 hours), whereas the two types of forecasts are of approximately equal skill for longer lead times (e.g. 36–48 hours). Similar results are obtained for the maximum and minimum temperature forecasts. Objective cloud amount forecasts are more skilful than subjective cloud amount forecasts for all lead times. Examination of trends in performance over the last decade reveals that both types of forecasts for all three elements increased in skill (or accuracy) over the period, with improvements in objective forecasts equalling or exceeding improvements in subjective forecasts. The role and impact of the objective forecasts in the subjective weather forecasting process are discussed in some detail. The need to conduct controlled experiments and other studies of this process, with particular reference to the assimilation of information from different sources, is emphasized. Important characteristics of the forecasting system in meteorology are identified, and they are used to describe similarities and differences between weather forecasting and forecasting in other fields. Acquisition of some of these characteristics may be beneficial to other forecasting systems.     

The forecasting attributes of trend- and difference-stationary representations for macroeconomic time series

We analyse the forecasting attributes of trenc and diffence-stationary representations of the U.S. macroeconomic time series sudied by Nelson and Plosser (1982). Predictive densities based on models estimated for these series (which terminate in 1970) are compared with subsequent realizations compiled by Schotman and van Dijk (1991) which terminate in (1988). Predictive densities obtained using the, extended series are also derived to assess the impact of the subsequent realization on long-range forecasts. Of particular interest are comparisons of the average intervals of predictive densities corresponding to the competing specifications In general, we find that coverage intervals based on diference-stationary specifications are far wider than those based or. trend-stationary specifications for the real series, and slightly wider for the nominal series. This additional width is often a virtue in forecasting nuninal series over the 1971-1988 period, as the inflation experienced durnig this time was unprecedented in the 1900s. However, the evolution of the real series has been relatively stable in the 1900s, hence the uncertainty associated with difference-stationary specifications generally seems excessive for these data.     

Conditional distribution analyses of probabilistic forecasts

Probabilistic forecasts have good ‘external correspondence’ if events that are assigned probabilities close to 1 tend to occur frequently, whereas those assigned probabilities near 0 tend to occur rarely. This paper describes simple procedures for analysing external correspondence into meaningful components that might guide efforts to understand and improve forecasting performance. The procedures focus on differences between the judgements made by the forecaster when the target event occurs, as compared to when it does not. The illustrations involve a professional oddsmaker's predictions of baseball game outcomes, meteorologists' precipitation forecasts and physicians' diagnoses of pneumonia. The illustrations demonstrate the ability of the procedures to highlight important forecasting tendencies that are sometimes more difficult to discern by other means.     

Forecasting and trading strategies based on a price trend model

In this paper, we consider the price trend model in which it is assumed that the time series of a security's prices contain a stochastic trend component which remains constant on each of a sequence of time intervals, with each interval having random duration. A quasi‐maximum likelihood method is used to estimate the model parameters. Optimal one‐step‐ahead forecasts of returns are derived. The trading rule based on these forecasts is constructed and is found to bear similarity to a popular trading rule based on moving averages. When applying the methods to forecast the returns of the Hang Seng Index Futures in Hong Kong, we find that the performance of the newly developed trading rule is satisfactory. Copyright © 2000 John Wiley & Sons, Ltd.     

Forecast intervals of net migration: the case of the Netherlands

This study addresses problems concerning the forecasting of net migration in the preparation of population forecasts. "As the width of forecast intervals for migration in single years differs strongly from that of an interval for average migration during the forecast period, it is important that the forecaster indicates which type of interval is presented. A comparison of forecast intervals for net migration obtained from an ARIMA model to intervals in official Dutch national population forecasts shows that the uncertainty on migration has been underestimated in past official forecasts."     

Judging the probability that the next point in an observed time-series will be below,or above,a given value

We presented people with trended and untrended time series and asked them to estimate the probability that the next point would be below each of seven different reference values. The true probabilities that the point would be below these values were 0.01, 0.10, 0.25, 0.50, 0.75, 0.90 and 0.99. People overestimated probabilities of less than 0.50 and underestimated those of more than 0.50. Consequently, their subjective probability distributions were flatter than they should have been: people appeared to be under confident in their estimates of where the next point would lie. This bias was greater for the trended series. It was also greater in a second experiment in which people estimated the probability that the next item would be above the reference values. We discuss reasons for these effects and consider their implications for decision making.     

A comparison between linear and nonlinear forecasts for nonlinear AR models

In this paper the relative forecast performance of nonlinear models to linear models is assessed by the conditional probability that the absolute forecast error of the nonlinear forecast is smaller than that of the linear forecast. The comparison probability is explicitly expressed and is shown to be an increasing function of the distance between nonlinear and linear forecasts under certain conditions. This expression of the comparison probability may not only be useful in determining the predictor, which is either a more accurate or a simpler forecast, to be used but also provides a good explanation for an odd phenomenon discussed by Pemberton. The relative forecast performance of a nonlinear model to a linear model is demonstrated to be sensitive to its forecast origins. A new forecast is thus proposed to improve the relative forecast performance of nonlinear models based on forecast origins. © 1997 John Wiley & Sons, Ltd.     

Comparing ex-ante forecasts from a sem and var model: An application to the italian economy

The predictive performance of a large-scale structural econometric model (SEM) of the Italian economy the Prometeia model is compared in this paper with a vector autoregressive (VAR) model estimated for a selection of six main variables of interest. The paper concentrates on the quarterly ex-ante forecasts of GDP growth rate and the annual forecasts of GDP growth and inflation rate, over the period 1980-85. It concludes that no forecaster is systematically better than the other. In particular, the VAR model outperforms the SEM in short-run forecasts, suggesting that, for the latter, more careful attention should be addressed to questions of dynamic specification. On the other hand, for longer intervals, the SEM forecasts are more accurate than the VAR forecasts, in that they can benefit from the judgemental interventions of the model users and the model can pick up the non-linearities of the economy which cannot be captured by the VAR. Given the different kinds of information that can be extracted from the two approaches, it seems more reasonable to consider them as complementary rather than alternative tools for modelling and forecasting. Therefore, rather than attempting to establish the superiority of one type of model over the other, this kind of comparisons should be seen as a useful diagnostic tool for detecting types of model misspecification.     

Guaranteed‐content prediction intervals for non‐linear autoregressions

In this paper we present guaranteed‐content prediction intervals for time series data. These intervals are such that their content (or coverage) is guaranteed with a given high probability. They are thus more relevant for the observed time series at hand than classical prediction intervals, whose content is guaranteed merely on average over hypothetical repetitions of the prediction process. This type of prediction inference has, however, been ignored in the time series context because of a lack of results. This gap is filled by deriving asymptotic results for a general family of autoregressive models, thereby extending existing results in non‐linear regression. The actual construction of guaranteed‐content prediction intervals directly follows from this theory. Simulated and real data are used to illustrate the practical difference between classical and guaranteed‐content prediction intervals for ARCH models. Copyright © 2001 John Wiley & Sons, Ltd.     

The Effects of Disaggregation on Forecasting Nonstationary Time Series

This paper focuses on the effects of disaggregation on forecast accuracy for nonstationary time series using dynamic factor models. We compare the forecasts obtained directly from the aggregated series based on its univariate model with the aggregation of the forecasts obtained for each component of the aggregate. Within this framework (first obtain the forecasts for the component series and then aggregate the forecasts), we try two different approaches: (i) generate forecasts from the multivariate dynamic factor model and (ii) generate the forecasts from univariate models for each component of the aggregate. In this regard, we provide analytical conditions for the equality of forecasts. The results are applied to quarterly gross domestic product (GDP) data of several European countries of the euro area and to their aggregated GDP. This will be compared to the prediction obtained directly from modeling and forecasting the aggregate GDP of these European countries. In particular, we would like to check whether long‐run relationships between the levels of the components are useful for improving the forecasting accuracy of the aggregate growth rate. We will make forecasts at the country level and then pool them to obtain the forecast of the aggregate. The empirical analysis suggests that forecasts built by aggregating the country‐specific models are more accurate than forecasts constructed using the aggregated data. Copyright © 2014 John Wiley & Sons, Ltd.