Time-Series Learning and Forecasting

Time series data consists of ordered observations that are typically autocorrelated: today's value carries information about tomorrow's, which directly violates the independent-and-identically-distributed assumption underlying most standard supervised learning theory and means observations cannot simply be shuffled and modeled independently of their neighbors. Hyndman and Athanasopoulos' textbook treatment of forecasting begins, appropriately, with graphical exploration before any model is fit: time plots reveal overall trend and obvious irregularities such as outliers or level shifts, while seasonal plots reveal repeating within-period patterns like a weekday-weekend cycle or a holiday spike. A key analytical tool is decomposition, which splits an observed series into a trend-cycle component capturing the long-run direction and slower fluctuations, a seasonal component capturing regular within-period patterns such as daily or weekly cycles, and a remainder component capturing what decomposition cannot explain; visually or numerically separating these components clarifies which patterns a subsequent forecasting model actually needs to capture, and which can be safely treated as noise.

A series is called (weakly) stationary when its statistical properties, specifically its mean, variance, and autocorrelation structure, do not depend on the particular time at which it is observed. Many classical forecasting models are built around this assumption because they estimate a single, fixed set of parameters meant to characterize the series' behavior throughout the observed period; if the underlying mean or autocorrelation structure itself changes with time, no single fixed parameterization describes the whole series well, and forecasts extrapolated from an ill-fitting model can be systematically biased. Differencing, computing the change between consecutive observations, or seasonal differencing, computing the change between observations one full seasonal period apart, is the standard technique for removing trend and seasonality and moving a raw series toward approximate stationarity before further modeling; a series sometimes requires more than one round of differencing before its remaining structure looks reasonably stationary.

Two diagnostic tools are central to identifying what temporal dependency remains after this preprocessing: the autocorrelation function (ACF), which measures how strongly a series correlates with lagged versions of itself at increasing lags, and the partial autocorrelation function (PACF), which measures the correlation with a given lag after removing the linear effect of all shorter lags. Distinctive patterns in the ACF and PACF plots, such as a sharp cutoff after a small number of lags versus a slow, gradual decay, guide the practitioner toward appropriate autoregressive or moving-average model orders, a diagnostic step covered in more depth in the next lesson. Together, decomposition, stationarity testing, and these two correlation diagnostics form the exploratory groundwork that precedes fitting any classical forecasting model, since choosing a model's structure without first understanding a series' trend, seasonality, and dependency pattern risks fitting a model poorly matched to the data it is meant to describe.

The ARIMA (AutoRegressive Integrated Moving Average) model family, developed within the Box-Jenkins methodology and presented in detail in Hyndman and Athanasopoulos' text, combines three structural components. The autoregressive (AR) part of order p regresses the current value on p of the series' own past values; the moving-average (MA) part of order q models the current value's error term as a linear combination of the past q forecast errors; and the integrated (I) part of order d applies d rounds of differencing to the raw series before the AR and MA components are fit, addressing non-stationarity directly within the model specification. The classical Box-Jenkins workflow for building an ARIMA model proceeds through an iterative identification-estimation-diagnostic-checking cycle: tentatively identify candidate orders (p, d, q) using tools like the ACF and PACF, estimate the resulting model's parameters, check residual diagnostics for remaining structure the model failed to capture, and revise the specification if the diagnostics reveal problems.

Exponential smoothing methods take a different approach, weighting more recent observations more heavily than older ones when forming a forecast, with variants adding explicit components for trend (Holt's linear method) and for seasonality (Holt-Winters seasonal method), so that a forecast can track a rising or falling level and a repeating seasonal pattern simultaneously rather than treating every past observation as equally informative. Rather than being introduced as a purely heuristic weighting scheme, Hyndman and Athanasopoulos formalize these methods as innovations state space models, commonly abbreviated ETS, which gives exponential smoothing a proper probabilistic underpinning: each model is expressed via an unobserved evolving state plus an observation equation, enabling principled estimation, automatic model selection across variants via information criteria, and coherent generation of prediction intervals rather than point forecasts alone.

In practice, exponential smoothing methods are prized for their simplicity, interpretability, and strong empirical track record on a wide range of business and operational time series, while ARIMA models offer more flexibility for series exhibiting more intricate autocorrelation structure that a simple trend-and-seasonality decomposition does not fully capture. Neither family is universally superior; a practitioner typically fits candidates from both, compares them using the evaluation techniques covered in the next lesson, and lets the data decide rather than committing to one family on principle. Both families, despite predating most modern machine learning methods, remain the standard baselines against which newer approaches, including deep-learning-based sequence models, are compared in forecasting competitions and applied practice, precisely because their assumptions about trend, seasonality, and error structure are explicit and interpretable rather than implicit in an opaque architecture.

Evaluating a forecaster's accuracy requires a numeric error measure, and different measures make different trade-offs. Mean Absolute Error (MAE) averages the absolute value of each period's forecast error, treating all error magnitudes proportionally; Root Mean Squared Error (RMSE) averages the squared errors and then takes the square root, which weights larger errors more heavily than smaller ones because squaring disproportionately inflates bigger mistakes. Scaled and percentage variants of these measures address the fact that both MAE and RMSE are scale-dependent, making them unsuitable for directly comparing forecast accuracy across series measured in different units. Because time series data is temporally dependent, ordinary random-shuffle cross-validation is generally inappropriate for evaluating forecasts: shuffling before splitting can place future observations in the training set used to predict an earlier held-out point, leaking information the model would never have access to at actual forecast time and producing an overly optimistic accuracy estimate. Rolling-origin evaluation avoids this by always restricting a test point's training data to observations that occurred strictly before it in time, closely matching how the model would genuinely be used in production.

A state-space representation models an observed series as generated from an unobserved internal state that evolves over time according to some transition process, combined with an observation equation linking that hidden state to what is actually measured. The ETS formulation of exponential smoothing from the previous lesson is one instance of this general framework, and state-space models more broadly support recursive, Kalman-filter-style estimation and prediction, updating beliefs about the hidden state efficiently as each new observation arrives rather than re-fitting the whole model from scratch. A particularly valuable byproduct of the state-space formulation is that it naturally produces prediction intervals whose width grows with the forecast horizon, reflecting the intuitive and empirically correct fact that uncertainty about the future compounds the further ahead a forecast reaches.

Several aspects of time-series forecasting remain genuinely open areas of research rather than settled textbook material: reliably quantifying uncertainty over long forecast horizons, where errors compound and simple state-space assumptions can understate true risk; coherently incorporating exogenous regressors and reconciling forecasts across hierarchical or grouped series, for example ensuring that store-level and regional-level demand forecasts sum consistently rather than being produced independently and left to disagree; and the ongoing empirical comparison between classical statistical models like ARIMA and ETS and modern deep-learning sequence models, which do not uniformly outperform classical baselines across the standardized forecasting competitions used to benchmark them, despite their far greater computational cost and data requirements. A practitioner encountering a new forecasting problem is generally well served by starting from these classical, interpretable baselines and only reaching for more complex models once they demonstrably improve on the metrics covered earlier in this lesson.