Quality/value relationships for imperfect weather forecasts in a prototype multistage decision-making model

Some theoretical results concerning the nature of the relationship between the scientific quality and economic value of imperfect weather forecasts are obtained. A prototype multistage decision-making model is considered, involving only two possible actions and two possible states of weather. This particular form of model is motivated by a real-world application known as the fruit-frost problem. For an infinite-horizon, discounted version of this model it is shown that economic value remains zero below a forecast quality threshold and then rises monotonically but nonlinearly above this threshold. In particular, the relative sensitivity of economic value to changes in the quality of forecasts increases as perfect information is approached. This curve is compared with quality/value relationships that have been obtained for other versions of the model; namely, a single-stage model and a multistage, finite-horizon model.     

Short‐term salmon price forecasting

This study establishes a benchmark for short‐term salmon price forecasting. The weekly spot price of Norwegian farmed Atlantic salmon is predicted 1–5 weeks ahead using data from 2007 to 2014. Sixteen alternative forecasting methods are considered, ranging from classical time series models to customized machine learning techniques to salmon futures prices. The best predictions are delivered by k‐nearest neighbors method for 1 week ahead; vector error correction model estimated using elastic net regularization for 2 and 3 weeks ahead; and futures prices for 4 and 5 weeks ahead. While the nominal gains in forecast accuracy over a naïve benchmark are small, the economic value of the forecasts is considerable. Using a simple trading strategy for timing the sales based on price forecasts could increase the net profit of a salmon farmer by around 7%.     

An assessment of the economic value of non-linear foreign exchange rate forecasts

Recent research suggests that non-linear methods cannot improve the point forecasts of high-frequency exchange rates. These studies have been using standard forecasting criteria such as smallest mean squared error (MSE) and smallest mean absolute error (MAE). It is, however, premature to conclude from this evidence that non-linear forecasts of high-frequency financial returns are economically or statistically insignificant. We prove a proposition which implies that the standard forecasting criteria are not necessarily particularly suited for assessment of the economic value of predictions of non-linear processes where the predicted value and the prediction error may not be independently distributed. Adopting a simple non-linear forecasting procedure to 15 daily exchange rate series we find that although, when compared to simple random walk forecasts, all the non-linear forecasts give a higher MSE and MAE, when applied in a simple trading strategy these forecasts result in a higher mean return. It is also shown that the ranking of portfolio payoffs based on forecasts from a random walk, and linear and non-linear models, is closely related to a non-parametric test of market timing.     

Evaluating the predictive accuracy of volatility models

Standard statistical loss functions, such as mean‐squared error, are commonly used for evaluating financial volatility forecasts. In this paper, an alternative evaluation framework, based on probability scoring rules that can be more closely tailored to a forecast user's decision problem, is proposed. According to the decision at hand, the user specifies the economic events to be forecast, the scoring rule with which to evaluate these probability forecasts, and the subsets of the forecasts of particular interest. The volatility forecasts from a model are then transformed into probability forecasts of the relevant events and evaluated using the selected scoring rule and calibration tests. An empirical example using exchange rate data illustrates the framework and confirms that the choice of loss function directly affects the forecast evaluation results. Copyright © 2001 John Wiley & Sons, Ltd.     

Economic and statistical measures of forecast accuracy

This paper argues in favour of a closer link between the decision and the forecast evaluation problems. Although the idea of using decision theory for forecast evaluation appears early in the dynamic stochastic programming literature, and has continued to be used with meteorological forecasts, it is hardly mentioned in standard academic texts on economic forecasting. Some of the main issues involved are illustrated in the context of a two‐state, two‐action decision problem as well as in a more general setting. Relationships between statistical and economic methods of forecast evaluation are discussed and links between the Kuipers score used as a measure of forecast accuracy in the meteorology literature and the market timing tests used in finance are established. An empirical application to the problem of stock market predictability is also provided, and the conditions under which such predictability could be explained in the presence of transaction costs are discussed. Copyright © 2000 John Wiley & Sons, Ltd.     

Long‐Term Forecasting of Global Carbon Dioxide Emissions: Reducing Uncertainties Using a Per Capita Approach

Global CO₂ emission forecasts span such a wide range as to yield little guidance for climate policy and analysis. But global per capita emissions appear to be better constrained than total emissions, which we argue has an economic justification. Trend stationarity of per capita emissions may provide a means of characterizing the relative likelihood of global forecasts. On data spanning 1950 to 2009 a unit root hypothesis allowing for endogenous structural breaks is rejected, but adding in the 2010 observation pushes the p‐value slightly over 0.1. Since structural breaks cannot be detected at the end of sample this may simply indicate a power problem. Using Monte Carlo simulations we conclude that the lower half of a well‐known suite of IPCC emission scenarios is more likely to occur than the upper half, and the top quartile is particularly difficult to justify. Copyright © 2013 John Wiley & Sons, Ltd.     

Municipal budget forecasting with multivariate ARMA models

In this paper multivariate ARMA models are applied to the problem of forecasting city budget variables. Unlike univariate time-series methods, multivariate models can use relationships among budget variables as well as relationships with economic and demographic indicators. Although available budget series are shorter than what is usually believed necessary for multivariate ARMA modelling, the forecasts seem to be of higher quality than those from univariate models.     

The Effect of Nonlinearity between Credit Conditions and Economic Activity on Density Forecasts

This paper examines the effect of nonlinearities on density forecasting. It focuses on the relationship between credit markets and the rest of the economy. The possible nonlinearity of this relationship is captured by a threshold vector autoregressive model estimated on US data using Bayesian methods. Density forecasts thus account for the uncertainty in all model parameters and possible future regime changes. It is shown that considering nonlinearity can improve the probabilistic assessment of the economic outlook. Moreover, three illustrative examples are discussed to shed some light on the possible practical applicability of density forecasts derived from non‐linear models. Copyright © 2015 John Wiley & Sons, Ltd.     

Diagnostic tracking and model specification in combined forecasts of U.K. inflation

Combinations of several forecasts are now quite commonly used as inputs into business planning models. For example, capital budgeting generally involves a synthesis of several sources of economic forecasts. In such cases, where uncertainty and risk are also being explicitly considered, the statistical specification of the combined forecasts becomes particularly important. An investigation of the monthly forecasts of annual inflation from nine leading U.K. economic models was undertaken to examine the circumstances under which well-specified and efficient combined forecasts could be produced. This has helped to refine the general guidelines for the practical use of combinations in planning models.     

Collinearity and the use of latent root regression for combining GNP forecasts

In combining economic forecasts a problem often faced is that the individual forecasts display some degree of dependence. We discuss latent root regression for combining collinear GNP forecasts. Our results indicate that latent root regression produces more efficient combining weight estimates (regression parameter estimates) than ordinary least squares estimation (OLS), although out-of-sample forecasting performance is comparable to OLS.     

Unbiasedness,efficiency and the combination of economic forecasts

This paper considers the problem of determining whether forecasts are unbiased and examines the implications this has for combining different forecasts. The practical issues of how economic forecasts might be combined are discussed. There is an empirical illustration of the procedures in which the properties of UK forecasts from the London Business School, the National Institute, the Henley Centre for Forecasting, Phillips and Drew and the OECD are examined.     

The role of macroeconometric models in forecasting and policy analysis in the united states

This article stresses how little is known about the quality, particularly the relative quality, of macroeconometric models. Most economists make a strict distinction between the quality of a model per se and the accuracy of solutions based on that model. While this distinction is valid, it leaves unanswered how to compare the‘validity’of conditional models. The standard test, the accuracy of ex post simulations, is not definitive when models with differing degrees of exogeneity are compared. In addition, it is extremely difficult to estimate the relative quantitative importance of conceptual problems of models, such as parameter instability across‘policy regimes’ In light of the difficulty in comparisons of conditional macroeconometric models, many model-builders and users assume that the best models are those that have been used to make the most accurate forecasts are those made with the best models. Forecasting experience indicates that forecasters using macroeconometric models have produced more accurate macroeconomic forecasts than either naive or sophisticated unconditional statistical models. It also suggests that judgementally adjusted forecasts have been more accurate than model-based forecasts generated mechanically. The influence of econometrically-based forecasts is now so pervasive that it is difficult to find examples of‘purely judgemental’forecasts.     

Predicting Recessions with Leading Indicators: Model Averaging and Selection over the Business Cycle

Four methods of model selection—equally weighted forecasts, Bayesian model‐averaged forecasts, and two models produced by the machine‐learning algorithm boosting—are applied to the problem of predicting business cycle turning points with a set of common macroeconomic variables. The methods address a fundamental problem faced by forecasters: the most useful model is simple but makes use of all relevant indicators. The results indicate that successful models of recession condition on different economic indicators at different forecast horizons. Predictors that describe real economic activity provide the clearest signal of recession at very short horizons. In contrast, signals from housing and financial markets produce the best forecasts at longer forecast horizons. A real‐time forecast experiment explores the predictability of the 2001 and 2007 recessions. Copyright © 2015 John Wiley & Sons, Ltd.     

Quantile forecasting for credit risk management using possibly misspecified hidden Markov models

Recent models for credit risk management make use of hidden Markov models (HMMs). HMMs are used to forecast quantiles of corporate default rates. Little research has been done on the quality of such forecasts if the underlying HMM is potentially misspecified. In this paper, we focus on misspecification in the dynamics and dimension of the HMM. We consider both discrete‐ and continuous‐state HMMs. The differences are substantial. Underestimating the number of discrete states has an economically significant impact on forecast quality. Generally speaking, discrete models underestimate the high‐quantile default rate forecasts. Continuous‐state HMMs, however, vastly overestimate high quantiles if the true HMM has a discrete state space. In the reverse setting the biases are much smaller, though still substantial in economic terms. We illustrate the empirical differences using US default data. Copyright © 2008 John Wiley & Sons, Ltd.     

A test of the joint efficiency of macroeconomic forecasts using multivariate random forests

We contribute to recent research on the joint evaluation of the properties of macroeconomic forecasts in a multivariate setting. The specific property of forecasts that we are interested in is their joint efficiency. We study the joint efficiency of forecasts by means of multivariate random forests, which we use to model the links between forecast errors and predictor variables in a forecaster's information set. We then use permutation tests to study whether the Mahalanobis distance between the predicted forecast errors for the growth and inflation forecasts of four leading German economic research institutes and actual forecast errors is significantly smaller than under the null hypothesis of forecast efficiency. We reject joint efficiency in several cases, but also document heterogeneity across research institutes with regard to the joint efficiency of their forecasts.     

Expectations,use and judgmental adjustment of external financial and economic forecasts: an empirical investigation

A survey of 124 users of externally produced financial and economic forecasts in Turkey investigated their expectations and perceptions of forecast quality and their reasons for judgmentally adjusting forecasts. Expectations and quality perceptions mainly related to the timeliness of forecasts, the provision of a clear justifiable rationale and accuracy. Cost was less important. Forecasts were frequently adjusted when they lacked a justifiable explanation, when the user felt they could integrate their knowledge into the forecast, or where the user perceived a need to take responsibility for the forecast. Forecasts were less frequently adjusted when they came from a well‐known source and were based on sound explanations and assumptions. The presence of feedback on accuracy reduced the influence of these factors. The seniority and experience of users had little effect on their attitudes or propensity to make adjustments. Copyright © 2008 John Wiley & Sons, Ltd.     

Early Warning with Calibrated and Sharper Probabilistic Forecasts

Given a nonlinear model, a probabilistic forecast may be obtained by Monte Carlo simulations. At a given forecast horizon, Monte Carlo simulations yield sets of discrete forecasts, which can be converted to density forecasts. The resulting density forecasts will inevitably be downgraded by model misspecification. In order to enhance the quality of the density forecasts, one can mix them with the unconditional density. This paper examines the value of combining conditional density forecasts with the unconditional density. The findings have positive implications for issuing early warnings in different disciplines including economics and meteorology, but UK inflation forecasts are considered as an example. Copyright © 2012 John Wiley & Sons, Ltd.     

Professional economic forecasts: Are they worth their costs?

Most economic forecast evaluations dating back 20 years show that professional forecasters add little to the forecasts generated by the simplest of models. Using various types of forecast error criteria, these evaluations usually conclude that the professional forecasts are little better than the no-change or ARIM A type forecast. It is our contention that this conclusion is mistaken because the conventional error criteria may not capture why forecasts are ma& or how they are used. Using forecast directional accuracy, the criterion which has been found to be highly correlated with profits in an interest rate setting, we find that professional GNP forecasts dominate the cheaper alternatives. Moreover, there appears to be no systematic relationship between this preferred criterion and the error measures used in previous studies.     

Building Scenarios of Multiple Time Series that Take into Account the Effects of an Expected Intervention

We consider a forecasting problem that arises when an intervention is expected to occur on an economic system during the forecast horizon. The time series model employed is seen as a statistical device that serves to capture the empirical regularities of the observed data on the variables of the system without relying on a particular theoretical structure. Either the deterministic or the stochastic structure of a vector autoregressive error correction model of the system is assumed to be affected by the intervention. The information about the intervention effect is just provided by some linear restrictions imposed on the future values of the variables involved. Formulas for restricted forecasts with intervention effects and their mean squared errors are derived as a particular case of Catlin's static updating theorem. An empirical illustration uses Mexican macroeconomic data on five variables and the restricted forecasts consider targets for years 2011–2014. Copyright © 2013 John Wiley & Sons, Ltd.     

The importance of the forecast

Careful forecasts, as accurate as possible, are central to the successful implementation of policy. There are fundamental reasons why policy makers cannot ‘play by ear’, adjusting policy quickly to each unexpected deviation in economic outcomes. Specific incidents are described where economic policy went awry because of faulty forecasts. The policy process is described in detail to show precisely where the forecast enters. Forecasting as a validation tool for establishing credibility in policy formation is analysed and discussed. Some estimated measure of forecast accuracy is presented, together with commentary on the necessary degrees of precision for successful implementation of policy.