Exponential smoothing: The state of the art

This paper is a critical review of exponential smoothing since the original work by Brown and Holt in the 1950s. Exponential smoothing is based on a pragmatic approach to forecasting which is shared in this review. The aim is to develop state-of-the-art guidelines for application of the exponential smoothing methodology. The first part of the paper discusses the class of relatively simple models which rely on the Holt-Winters procedure for seasonal adjustment of the data. Next, we review general exponential smoothing (GES), which uses Fourier functions of time to model seasonality. The research is reviewed according to the following questions. What are the useful properties of these models? What parameters should be used? How should the models be initialized? After the review of model-building, we turn to problems in the maintenance of forecasting systems based on exponential smoothing. Topics in the maintenance area include the use of quality control models to detect bias in the forecast errors, adaptive parameters to improve the response to structural changes in the time series, and two-stage forecasting, whereby we use a model of the errors or some other model of the data to improve our initial forecasts. Some of the major conclusions: the parameter ranges and starting values typically used in practice are arbitrary and may detract from accuracy. The empirical evidence favours Holt's model for trends over that of Brown. A linear trend should be damped at long horizons. The empirical evidence favours the Holt-Winters approach to seasonal data over GES. It is difficult to justify GES in standard form–the equivalent ARIMA model is simpler and more efficient. The cumulative sum of the errors appears to be the most practical forecast monitoring device. There is no evidence that adaptive parameters improve forecast accuracy. In fact, the reverse may be true.     

A unified view of statistical forecasting procedures

A large number of statistical forecasting procedures for univariate time series have been proposed in the literature. These range from simple methods, such as the exponentially weighted moving average, to more complex procedures such as Box–Jenkins ARIMA modelling and Harrison–Stevens Bayesian forecasting. This paper sets out to show the relationship between these various procedures by adopting a framework in which a time series model is viewed in terms of trend, seasonal and irregular components. The framework is then extended to cover models with explanatory variables. From the technical point of view the Kalman filter plays an important role in allowing an integrated treatment of these topics.     

ARARMA models for time series analysis and forecasting

Methods of time series forecasting are proposed which can be applied automatically. However, they are not rote formulae, since they are based on a flexible philosophy which can provide several models for consideration. In addition it provides diverse diagnostics for qualitatively and quantitatively estimating how well one can forecast a series. The models considered are called ARARMA models (or ARAR models) because the model fitted to a long memory time series (t) is based on sophisticated time series analysis of AR (or ARMA) schemes (short memory models) fitted to residuals Y(t) obtained by parsimonious‘best lag’non-stationary autoregression. Both long range and short range forecasts are provided by an ARARMA model Section 1 explains the philosophy of our approach to time series model identification. Sections 2 and 3 attempt to relate our approach to some standard approaches to forecasting; exponential smoothing methods are developed from the point of view of prediction theory (section 2) and extended (section 3). ARARMA models are introduced (section 4). Methods of ARARMA model fitting are outlined (sections 5,6). Since‘the proof of the pudding is in the eating’, the methods proposed are illustrated (section 7) using the classic example of international airline passengers.     

Robust exponential smoothing

The paper is devoted to robust modifications of exponential smoothing for time series with outliers or long-tailed distributions. Classical exponential smoothing applied to such time series is sensitive to the presence of outliers or long-tailed distributions and may give inadequate smoothing and forecasting results. First, simple and double exponential smoothing in the L₁ norm (i.e. based on the least absolute deviations) are discussed in detail. Then, general exponential smoothing is made robust, replacing the least squares approach by M-estimation in such a way that the recursive character of the final formulas is preserved. The paper gives simple algorithmic procedures which preserve advantageous features of classical exponential smoothing and, in addition, which are less sensitive to outliers. Robust versions are compared numerically with classical ones.     

Prediction intervals for exponential smoothing using two new classes of state space models

Three general classes of state space models are presented, using the single source of error formulation. The first class is the standard linear model with homoscedastic errors, the second retains the linear structure but incorporates a dynamic form of heteroscedasticity, and the third allows for non‐linear structure in the observation equation as well as heteroscedasticity. These three classes provide stochastic models for a wide variety of exponential smoothing methods. We use these classes to provide exact analytic (matrix) expressions for forecast error variances that can be used to construct prediction intervals one or multiple steps ahead. These formulas are reduced to non‐matrix expressions for 15 state space models that underlie the most common exponential smoothing methods. We discuss relationships between our expressions and previous suggestions for finding forecast error variances and prediction intervals for exponential smoothing methods. Simpler approximations are developed for the more complex schemes and their validity examined. The paper concludes with a numerical example using a non‐linear model. Copyright © 2005 John Wiley & Sons, Ltd.     

Judgmental extrapolation and the salience of change

People may often forecast using cognitive procedures that resemble formal time-series extrapolation models. A model of judgmental extrapolation based on exponential smoothing is proposed in which the setting of the trend parameter is hypothesized to depend upon the relative salience of the successive changes. The salience hypothesis was first tested with exponential series by the use of a framing manipulation. As predicted, focusing the subjects' attention on the changes led to more accurate forecasts. In two investment simulation studies, the salience hypothesis was further examined by varying the statistical properties of the price changes. As predicted, subjects were more likely to sell as prices fell and to buy as prices rose (1) as the sample size of similar changes increased; (2) when the variance of the changes was low; and (3) when the absolute value of the mean change was high. Conditions that may influence judgmental forecasting processes are discussed.     

The teachers/practitioners corner comparative analysis of load forecasting techniques at a southern utility

The prime directive of any regulated electric utility is to provide adequate and reliable electricity supplies to the consuming public at reasonable cost. This requires the continual addition of new generating plants which is based on a long term forecast of energy and peak demand. This study documents the forecasting process used at a southern utility and compares the accuracy of their models to that produced using Holt's exponential smoothing and generalized adoptive filtering.     

The M-competition with a fully automatic box–jenkins procedure

CAPRI is a fully automatic and quick procedure for forecasting. It is based on the Box–Jenkins methodology and needs no a priori knowledge about the time series. The 1001 series of the Makridakis competition have been analysed with this program and its accuracy measured in comparison with other methods. CAPRI is recommended for short term forecasting horizons in cases where the user does not want to interfere with the modelling process.     

Exponential smoothing: Estimation by maximum likelihood

In this paper several forecasting methods based on exponential smoothing with an underlying seasonal autoregressive-moving average (SARIMA) model are considered. The relations between the smoothing constants and the coefficients of the autoregressive and moving average polynomials are used. On that basis, a maximum likelihood procedure for parameter estimation is described. The approach rules out the need for initial smoothed values. Prediction intervals are also obtained as a by-product of the approach and a fast algorithm for implementing the method is outlined.     

Discount weighted estimation

The parsimonious method of exponentially weighted regression (EWR) is attractive but limited in application because it depends upon just one discount factor. This paper generalizes the EWR approach to a method called discount weighted estimation (DWE) which allowed distinct model components to have different associated discount factors. The method includes EWR as a special case. The general non-limiting recurrence relationships will be useful in practice, especially when practitioners wish to specify prior information, to intervene with subjective judgement and to derive estimates and forecasts sequentially based upon limited data. Two theorems extend the important EWR limiting results of Dobbie and McKenzie to DWE. The latter permits the derivation of a large class of known processs for which DWE is optimal. The method is illustrated by two applications, one of which uses the famous international airline passenger data. This allows a comparision with the ICI MULDO system which uses a particular two discount factor forecasting method. A companion paper extends the discount methods to Bayesian forecasting, Kalman filtering and state space modelling.     

Short-term forecasting of industrial production by means of quick indicators

This paper reports results on building transfer function models with linear combinations of quick indicators as inputs for very short-term prediction of the monthly time series of the volume of industrial production in Finland. The number of input variables in the transfer function models is reduced in two alternative ways: by replacing the original indicators by their two first principal components and by omitting certain indicators. The prediction accuracy of the transfer function models is checked outside the sample and found superior to that of corresponding ARIMA models. Neither of the two ways of reducing the number of input variables leads to consistently more accurate forecasts than the other. It is also found that the prediction accuracy of the transfer function models compares rather favourably with the preliminary values of the volume of industrial production published by the Central Statistical Office during the periods of rapid growth.     

Automatic monitoring of forecast errors

This paper evaluates a variety of automatic monitoring schemes to detect biased forecast errors. Backward cumulative sum (cusum) tracking signals have been recommended in previous research to monitor exponential smoothing models. This research shows that identical performance can be had with much simpler tracking signals. The smoothed-error signal is recommended for α = 0.1, although its performance deteriorates badly as α is increased. For higher α values, the simple cusum signal is recommended. A tracking signal based on the autocorrelation in errors is recommended for forecasting models other than exponential smoothing, with one exception. If the time series has a constant variance, the backward cusum should give better results.     

Some comments on the initialization of exponential smoothing

It is shown that the traditional choice for the initial smoothed statistics in general exponential smoothing leads to the same forecasts as the equivalent ARIMA model, provided that one uses zero starting values for the initial shocks. In addition, an initialization which uses ‘backforecasts’ as initial smoothed statistics is considered, and its relationship to unconditional least squares is explored.     

An evaluation of bayesian forecasting

‘Bayesian forecasting’ is a time series method of forecasting which (in the United Kingdom) has become synonymous with the state space formulation of Harrison and Stevens (1976). The approach is distinct from other time series methods in that it envisages changes in model structure. A disjoint class of models is chosen to encompass the changes. Each data point is retrospectively evaluated (using Bayes theorem) to judge which of the models held. Forecasts are then derived conditional on an assumed model holding true. The final forecasts are weighted sums of these conditional forecasts. Few empirical evaluations have been carried out. This paper reports a large scale comparison of time series forecasting methods including the Bayesian. The approach is two fold: a simulation study to examine parameter sensitivity and an empirical study which contrasts Bayesian with other time series methods.     

The accuracy of extrapolation methods; an automatic box–jenkins package sift

Hill and Woodworth (1980) proposed an algorithm suitable for identifying Box–Jenkins models automatically without reliance on the investigator. This paper first reviews the method. It is then used on the 111 series analysed by Anderson in the Makridakis forecasting competition. The results show that the automatic method of Hill and Woodworth is comparable in terms of accuracy to the full Box–Jenkins identification procedure.     

A Comparison of Box—Jenkins and objective methods for determining the order of a non-seasonal ARMA Model

This paper evaluates different procedures for selecting the order of a non-seasonal ARMA model. Specifically, it compares the forecasting accuracy of models developed by the personalized Box-Jenkins (BJ) methodology with models chosen by numerous automatic procedures. The study uses real series modelled by experts (textbook authors) in the BJ approach. Our results show that many objective selection criteria provide structures equal or superior to the time-consuming BJ method. For the sets of data used in this study, we also examine the influence of parsimony in time-series forecasting. Defining what models are too large or too small is sensitive to the forecast horizon. Automatic techniques that select the best models for forecasting are similar in size to BJ models although they often disagree on model order.     

Computing the variance of the forecast error for the holt-winters seasonal models

This paper presents expressions for the variance of the forecast error for arbitrary lead times for both the additive and multiplicative Holt-Winters seasonal forecasting models. It is shown that even when the smoothing constants are chosen to have values between zero and one, when the period is greater than four, the variance may not be finite for some values of the smoothing constants. In addition, the regions where the variance becomes infinite are almost the same for both models. These results are of importance for practitioners, who may choose values for the smoothing constants arbitrarily, or by searching on the unit cube for values which minimize the sum of the squared errors when fitting the model to a data set. It is also shown that the variance of the forecast error for the multiplicative model is nonstationary and periodic.     

On detecting and estimating a major level or slope change in general exponential smoothing

Exponential smoothing methods do not adapt well to a major level or slope change. In this paper, Bayesian statistical theory is applied to the dynamic linear model, altered by inclusion of dummy variables, and statistics are derived to detect such changes and to estimate both the change-point and the size. The paper also gives test statistics for such problems related to exponential smoothing. The statistics are simple functions of exponentially weighted moving averages of the forecast errors, using the same discount factor used in the exponential smoothing. Gardner has derived an approximate test statistic to detect a mean change in the constant mean model. When the present results are applied to this model they give the exact statistic.     

BVAR as a category management tool: An illustration and comparison with alternative techniques

Category management—a relatively new function in marketing—involves large-scale, real-time forecasting of multiple data series in complex environments. In this paper, we illustrate how Bayesian Vector Auto regression (BVAR) fulfils the category manager's decision-support requirements by providing accurate forecasts of a category's state variables (prices, volumes and advertising levels), incorporating management interventions (merchandising events such as end-aisle displays), and revealing competitive dynamics through impulse response analyses. Using 124 weeks of point-of-sale scanner data comprising 31 variables for four brands, we compare the out-of-sample forecasts from BVAR to forecasts from exponential smoothing, univariate and multivariate Box-Jenkins transfer function analyses, and multivariate ARMA models. Theil U's indicate that BVAR forecasts are superior to those from alternate approaches. In large-scale forecasting applications, BVAR's ease of identification and parsimonious use of degrees of freedom are particularly valuable.     

ARMA Models and the Box–Jenkins Methodology

The purpose of this paper is to apply the Box–Jenkins methodology to ARIMA models and determine the reasons why in empirical tests it is found that the post-sample forecasting the accuracy of such models is generally worse than much simpler time series methods. The paper concludes that the major problem is the way of making the series stationary in its mean (i.e. the method of differencing) that has been proposed by Box and Jenkins. If alternative approaches are utilized to remove and extrapolate the trend in the data, ARMA models outperform the models selected through Box–Jenkins methodology. In addition, it is shown that using ARMA models to seasonally adjusted data slightly improves post-sample accuracies while simplifying the use of ARMA models. It is also confirmed that transformations slightly improve post-sample forecasting accuracy, particularly for long forecasting horizons. Finally, it is demonstrated that AR(1), AR(2) and ARMA(1,1) models can produce more accurate post-sample forecasts than those found through the application of Box–Jenkins methodology.© 1997 John Wiley & Sons, Ltd.