Order of integration
In time series analysis, the order of integration of a time series process is the minimum number of times the series must be differenced to become stationary. A process that requires d differences to achieve stationarity is said to be integrated of order d, denoted as I(d). For example, a stationary series such as white noise is I(0), while a random walk with a unit root is I(1).[1] This concept is fundamental in econometrics for modeling non-stationary data, such as economic and financial time series, where understanding the integration order helps avoid spurious regressions and informs techniques like cointegration analysis.[2]Basic Concepts
Definition of Order of Integration
In time series analysis, a stochastic process \{X_t\} is defined as integrated of order d, denoted I(d), if the d-th difference of the series is covariance stationary while lower-order differences are not, with d being a non-negative integer. This measure quantifies the degree of non-stationarity in the series, indicating the minimum number of differencing operations required to transform it into a stationary process with constant mean, variance, and autocovariances that depend only on the lag. Processes classified as I(0) are already covariance stationary, requiring no differencing, which distinguishes them from higher-order integrated series that exhibit persistent trends or random walks. The formal mathematical representation employs the lag operator L, defined such that L X_t = X_{t-1}, allowing higher lags via powers like L^k X_t = X_{t-k}. The first-order difference operator is \Delta X_t = X_t - X_{t-1} = (1 - L) X_t, and the d-th order difference generalizes to \Delta^d X_t = (1 - L)^d X_t. For a series X_t \sim I(d), the expression (1 - L)^d X_t yields a stationary process, capturing the cumulative effect of shocks that propagate indefinitely in lower-differenced forms. The concept of order of integration emerged in econometrics during the 1970s and 1980s, extending the autoregressive integrated moving average (ARIMA) framework introduced by Box and Jenkins in their seminal 1970 work on time series forecasting. Granger formalized the "degree of integration" in 1981, providing a precise summary statistic for non-stationarity that facilitated advancements in modeling economic variables often characterized by unit roots and persistent dynamics. This development built directly on ARIMA's differencing parameter d, shifting focus from ad hoc transformations to a structured characterization of integration orders.Stationarity and Differencing
A time series process is defined as weakly stationary, also known as covariance stationary or second-order stationary, if its expected value (mean) is constant over time, its variance is finite and constant, and the covariance between any two points depends solely on the time lag between them rather than their absolute positions in time. This definition ensures that the statistical properties of the series do not systematically change, allowing for reliable modeling of dependencies. Non-stationarity in time series manifests in distinct ways, primarily as trend-stationarity or difference-stationarity. In a trend-stationary process, deviations from a deterministic trend (such as a linear or polynomial function of time) are stationary, meaning the series can be rendered stationary by subtracting the trend. In contrast, a difference-stationary process, often termed an integrated process, exhibits persistent shocks that do not revert to a fixed mean and requires differencing to achieve stationarity; this form is prevalent in macroeconomic data where unit roots induce random wanderings. Differencing plays a crucial role in transforming non-stationary series into stationary ones by eliminating trends or stochastic drifts. The first difference, defined as \Delta X_t = X_t - X_{t-1}, removes a linear trend or a unit root, stabilizing the mean of an integrated process. For series with higher-order trends, such as quadratic, second- or higher-order differencing (\Delta^2 X_t = \Delta (\Delta X_t)) is applied iteratively until stationarity is attained. The unit root concept describes a non-stationary autoregressive process of order 1 (AR(1)) where the autoregressive coefficient equals 1, as in X_t = X_{t-1} + \epsilon_t + \mu, causing innovations to have permanent effects and variance to grow over time.[3] This structure, tested via procedures like the Dickey-Fuller test, equates to integration of order 1 and distinguishes difference-stationary behavior from trend-stationarity.[3] A prototypical example of a unit root process is the simple random walk, given by X_t = X_{t-1} + \epsilon_t, where \{\epsilon_t\} is white noise with mean zero and constant variance; this process is non-stationary but becomes stationary after first differencing, exemplifying an I(1) series.Integer-Order Processes
First-Order Integration I(1)
A first-order integrated process, denoted I(1), is a non-stationary time series that requires differencing once to become stationary, or I(0).[4] Key properties include a non-constant mean, which may incorporate a drift term leading to a stochastic trend, unbounded variance that increases over time, and autocorrelations that decay very slowly due to persistent dependence between observations.[4] These characteristics imply that I(1) processes exhibit random wandering behavior without reverting to a fixed level, making them common in economic data such as GDP or stock prices where shocks have permanent effects.[5] The canonical example of an I(1) process is the random walk, which can include a drift component to model systematic trends. Without drift, it follows X_t = X_{t-1} + \epsilon_t, where \epsilon_t is white noise with mean zero and variance \sigma^2.[5] With drift, the model becomes X_t = \mu + X_{t-1} + \epsilon_t, where \mu is the constant drift parameter. In this case, the expected value is E[X_t] = \mu t + E[X_0], producing a linear trend in expectation while retaining the stochastic fluctuations of the random walk.[5] The variance of the partial sums for an I(1) process grows linearly with time, specifically \text{Var}(X_t) \approx t \sigma^2 for the driftless case, highlighting the accumulating uncertainty over longer horizons.[6] A critical implication of I(1) processes is the risk of spurious regression, where regressing two independent I(1) series on each other produces misleadingly high coefficients of determination (R^2) and statistically significant t-statistics purely by chance, undermining standard inference.[7] This occurs because the non-stationarity amplifies apparent correlations that do not reflect causal relationships, necessitating pre-testing for unit roots before applying ordinary least squares.[7] In the ARIMA framework, an I(1) process is modeled as ARIMA(p,1,q), where the integration order d=1 indicates that first differencing transforms the series into a stationary ARMA(p,q) process suitable for further autoregressive and moving average modeling.[8] This differencing step addresses the unit root, allowing forecasts of the original series by integrating the stationary differences.[8]Higher-Order Integration I(d) for d > 1
Higher-order integrated processes of order d > 1, denoted I(d), extend the concept of integration beyond the standard first-order case by requiring multiple differencings to achieve stationarity. Specifically, a process X_t is I(d) if the (d-1)-th difference (1 - L)^{d-1} X_t is I(1), and the full d-th difference (1 - L)^d X_t is stationary.[4] This structure builds on I(1) processes as foundational building blocks, where successive integrations accumulate non-stationarity.[9] A general mathematical form for such processes is X_t = X_{t-1} + Y_{t-1}, where Y_t is an I(1) process, implying that X_t integrates the non-stationary increments of Y_t.[9] Properties of I(d) for d > 1 include even slower mean reversion compared to I(1), as shocks propagate with greater persistence across multiple lags. The variance of the process grows as t^{2d-1}, leading to explosive uncertainty over time; for instance, in an I(2) process, this growth is cubic (O(t^3)). Over-differencing to achieve stationarity can introduce non-invertible moving average components, complicating model identification and inference.[4][10] An illustrative example is the double-integrated I(2) process, which arises as the cumulative sum of a random walk, often modeled in physical or economic acceleration contexts—such as position as I(2) when velocity is I(1).[9] In economics, I(d) processes with d > 1 are rare, as empirical data seldom support orders beyond 2, and higher differencing risks over-differencing that obscures long-run relationships and information. Such processes may also be misinterpreted as evidence of structural breaks due to their pronounced trend accelerations.[10][11][12]Construction Methods
Building from Stationary Series
One common method to construct an integrated time series of order d, denoted I(d), begins with a stationary I(0) process and applies successive differencing in reverse through cumulative summation. Specifically, to obtain an I(d) series from an I(d-1) series X_t, define the integrated process as Z_t = Z_{t-1} + X_t with Z_0 = 0, iterating this summation d times starting from the stationary base. This approach ensures the resulting series requires d differences to return to stationarity.[4] A practical simulation example illustrates this iterative integration. Start with white noise \epsilon_t \sim N(0, \sigma^2), which is I(0). Integrating once yields a random walk Z_t = Z_{t-1} + \epsilon_t, an I(1) process characterized by a unit root. Integrating again produces an I(2) series W_t = W_{t-1} + Z_t, where the variance grows cubically over time (proportional to t^3). Such simulations are essential for understanding the behavior of higher-order integrated processes in econometric modeling.[4] Deterministic trends can be incorporated during integration to reflect real-world non-stationarities like economic growth. For an I(1) process with constant drift, modify the summation to Z_t = Z_{t-1} + \mu + \epsilon_t, where \mu is the drift parameter; the expected value then follows a linear trend E[Z_t] = \mu t. Higher-order trends, such as quadratic, arise from integrating a linear trend in the differences, adding realism to simulated series for trend-stationary versus difference-stationary distinctions.[13] Software tools streamline these constructions for empirical analysis. In R, thecumsum() function applied to a vector of stationary innovations, such as cumsum(rnorm(n)), directly generates an I(1) random walk. Python's NumPy library offers analogous functionality via numpy.cumsum(), enabling quick simulation of integrated series from ARMA or white noise inputs.[14]
This building approach traces back to foundational work in time series, including early simulations in the Box-Jenkins methodology, where integrated ARIMA processes were generated to aid model identification through autocorrelation function patterns.