FTS-4 ARCH, GARCH, Multivariate Time Series

Background

In ARMA models, we only considered time series that are stationary and driven by white noise processes. We only modeled the mean of given the past values of the time series and did not consider the conditional variability. For better modeling of financial time series, we need to also consider the modeling of conditional variability.

Some of the characteristics of volatility that have been noted in stock price behavior:

Volatility clustering: most real-life financial time series exhibit non-constant conditional variance. Such non-constant conditional variance also clusters into periods of low and high volatility.

Volatility is continuous, not a jump process.

Volatility does not diverge to infinity; it is approximately stationary

Volatility increases more following big price decreases than after big price increases (the leverage effect)

ARCH model

Definition

In a non-parametric regression model, the conditional variability is introduced as

with and .

Note that ARCH is a more structured version of a white noise process.

Let be a martingale difference sequence (MDS), i.e. . Assume . Define process

where

This is called the autoregressive conditional heteroscedastic (ARCH) model of order .

The is a martingale difference sequence, because is only a function of , and we get

This implies that is a martingale difference sequence, which implies that

This does not mean is white noise because the variance may not be constant. Only under some conditions, it is a constant.

The process can be equivalently described by

ARCH process is used to model a residual process and should be used in conjunction with ARMA model. For example, the model with errors can be written as

Similarly, an process with noise process is given by

Properties

Conditions on Coefficients

Under , we have

To ensure and to not blow up as increases, one sufficient condition is

ARCH(1): Stationarity and Causality

Let and . Since

This implies that if ,

Repeating the recursion:

In this case, where .

ARCH(1) squared as AR(1)

If , then we got that (and hence ) are causal in the sense that they are written as functions of past noise. Bust it is not a linear process (in terms of ).

Hence is a martingale difference sequence with constant variance, and is linearly unpredictable as well as weakly stationary.

We can represent the ARCH(1) process differently as

Hence ARCH(1) is like an AR(1) process with a new noise process satisfying (MDS, but not constant variance).

From the AR(1) representation of and assuming (independent of ), it follows that

This reveals the non-negative and geometric decaying nature of the ACF of . This also explains volatility clustering. Larger values of are followed by large values of .

Recall that ARCH(1) equation . If log returns are modeled as ARCH(1), then this equation partially captures the phenomenon that large changes in return would be followed by larger volatility.

ARCH(p) squared in an AR(p) process

THe process is defined as

This implies that

By definition, , which implies is a martingale difference sequence. This implies that the square of an process behaves like an process. This can be used in identifying using the PACF plot for . We can also use Ljung-Box and Box-Pierce tests for the existence of ARCH model on squared residuals.

Weakness of ARCH(1) model

The model assumes that positive and negative shocks () have the same effects on volatility because the model assumes that the next shock depends only on the square of the positive shock(s). The empirical evidence suggests that returns respond differently to positive (price increase) and negative (price drop) shocks

The model is restrictive. If are i.i.d. , then exists if and only if

The ARCH model does not provide any insight into understanding the source of volatility. It only provides a mechanical way to describe the behavior of conditional variance. It gives no indication of what causes such behavior to occur

ARCH models are likely to over predict the volatility because they respond slowly to large isolated shocks to the return series

Generalized ARCH (GARCH) model

Definition

is defined as

If is MDS with unit conditional variance, i.e., and , then

Properties

GARCH and ARMA

Consider GARCH(1, 1): , where . It follows that

Hence, square of a GARCH(1, 1) is an ARMA(1, 1) driven by a new noise process:

Note that is an MDS: .

As for GARCH(p, q), if with

then setting for and , we get

If has all its roots outside of the unit circle, then can be represented as a linear combination of and is constant in

When (GARCH(1, 1)), we have

When GARCH(1, 1) is fit in practice, it is often found that . If this happens, the process is no longer stationary and this lead to integrated GARCH(1, 1) model or simply IGARCH(1, 1) model. Generally, we have

Hence, if , the recursive equation becomes

The volatility persists and blows up asymptotically as .

E-GARCH

ARCH, GARCH, and IGARCH share the same drawback that they are defined using square function and react same to price up and to price down which is unrealistic. The asymmetry here is referred to as leverage effect.

EGARCH (exponential GARCH) uses the exponential function in place of square to react differently to negative and positive values of . The idea is to incorporate the leverage effect by making sure is an increasing function of if is negative and a decreasing function of is is positive. For an MDS or a white noise , define

Assuming has the same mean for all , this is same as . Fix

If , then is a function of , i.e. does not capture leverage effect

If , then is large for . The coefficient of is which is positive, i.e., larger volatility when stock price decreases

If , then is large for . The coefficient of is which is positive, i.e., larger volatility when stock price increases

The E-GARCH(p, q) model is given by , where

Here . Since we are defining , will always be non-negative without any restriction on the coefficients .

If for all , then , and

Further, if are iid, then can be thought of as an ARMA(p, q) process with an independent noise process, but this analogy does not work as well as it did in the GARCH case.

Test for ARCH effects

Testing : is ARCH(p) model.

Fit on and form a regression equation

Let denote the coefficient of determination . Under (equivalently ), converges asymptotically to . Reject if . This is the Engle’s test for ARCH effects and can be shown to be equivalent to the Lagrange multiplier test statistic.

The test is available via statsmodels.stats.diagnostic.het_arch . We can also use Ljung-Box and Box-Pierce tests on .

Estimation in ARMA + (E-)(G)ARCH models

For any given time series, estimation of ARCH/GARCH/E-GARCH can be done in Python using the arch_model function. The approach is basically MLE or quasi-MLE.

ARCH(p)

Assume , then . Note that only depends on

With observed data , the likelihood is given by

Substitute and maximize the likelihood function over all . This method is reminiscent of weighted least squares regression in linear regression with heteroscedastic noise.

GARCH, E-GARCH

The likelihood method can be applied with GARCH and E-GARCH models but the inclusion of an MA-type component in the model makes the estimation complicated. Consider GARCH(1, 1):

We observe but cannot write and in terms of observed data. If we start with assuming for all . The best guess for unobserved white noise is zero. Then conditional on this assumption:

This sequence of equations represents in terms of . We can then perform MLE as before.

ARMA + ARCH/GARCH/E-GARCH

ARCH is a noise process. We usually do not observe this process, but model the data via ARMA/ARIMA + ARCH/GARCH/E-GARCH.

If , then

We should estimate the parameters of ARMA and GARCH simultaneously for consistency and efficiency. The basic idea is to represent in terms of the observed data and find parameters that minimize .

A simple strategy to estimate the parameters in ARMA + ARCH/GARCH models is to first fit the ARMA model to data, get the residuals and then fit ARCH/GARCH model to the residuals. However, just doing this we would obtain inconsistent estimators because the second-stage estimators depend on the first stage. We should continue this step by using weighted least squares:

Fit an ARMA model to the data by minimizing the sum of squares . Obtain the fitted residuals

Fit a ARCH / GARCH / E-GARCH on the fitted residuals . Obtain fitted volatility

Refit the ARMA model to the data by minimizing

Repeat this process until the convergence of the estimators

Multivariate Time Series

Time series arising in practice are best considered as components of some vector-valued time series having dependencies across coordinates and time.

Mean and Covariance

For a dimensional multivariate time series , define

Define the covariance matrices

where .

In matrix notation,

The autocorrelation matrix is defined with -th entry given by

Multivariate Stationary Time Series

The process is said to be weakly stationary if the moments and do not depend on , in which case we can use the notation

The simplest multivariate time series is white noise. is said to be white noise iff is stationary with and if and zero otherwise.

A natural estimator of the mean vector in terms of the observations is the vector of sample means:

For , we use

AR, MA and ARMA

Given the multivariate white noise process , AR, MA, and ARMA process can be defined analogously. A vector process denoted by is defined via

for a multivariate white noise process .

Generally, a vector process denoted by is defined via

where is a white noise . These can be written in terms of the backshift operator. Define

These are polynomials in a real value with matrix coefficients.

VAR as MA

Similar to for , there is a matrix expansion

if all the eigenvalues of are less than 1 in the absolute value, i.e.,

for all such that .

Using this we can write as

if has all its eigenvalues less than 1.

(Non-) Unique Representation

In the univariate case, the representation of as MA is unique. But in the multivariate case, such a representation need not be unique. Take

The MA representation holds true with because has its eigenvalues bounded by 1.

Note that

Hence, is and , the representation is not unique.