TOC
1. Basic Regression Analysis1.1 Time Series Data1.2 Distributed Lag Model1.3 Classical Assumptions1.4 Trends and Seasonality2. Further Issues for OLS on TS data2.1 Stationary and Weakly Dependent2.2 Asymptotic Properties of OLS2.3 Highly Persistent Time Series3. Serial Correlation and Heteroskedasticity3.1 Testing for Serial Correlation
1. Basic Regression Analysis
1.1 Time Series Data
Definition
Time series data are data collected on the same observational unit at multiple time periods.
Unlike cross-section data, time series data has a temporal ordering: the past can affect the future, but not vice versa. We need to alter some of our assumptions since we no longer have a random sample of individuals. Instead, we have a stochastic (i.e. random) process, i.e. a sequence of random variables indexed by time. One realization of a stochastic (i.e. random) process is a time series data.
Notations and Terms
Dataset: are observations on the time series variable . We consider only consecutive, evenly-spaced observations.
Lags: The first lag of a time series is ; the j-th lag is .
First difference of a series :
Growth Rates: The percentage change of a time series between periods and is approximately , where the approximation is most accurate when the percentage chang is small.
Autocorrelation (Serial Correlation)
The correlation (covariance) of a series with its own lagged values is called autocorrelation (autocovariance) or serial correlation.
The j-th sample autocorrelation is an estimate of the j-th population autocorrelation
where . is the sample average of computed over observations
1.2 Distributed Lag Model
Dynamic effects necessarily occur over time. The econometric model used to estimate dynamic causal effects needs to incorporate lags. The distributed lag model of order is:
where
- = impact effect (impact propensity) of change in = effect of change in on , holding past constant
- = 1-period dynamic multiplier = effect of change in on , holding constant
- = 2-period dynamic multiplier = effect of change in on , holding constant
- โฆ
For any horizon , we can define the cumulative effect as , which is interpreted as the change in the expected outcome periods after a permanent, one-unit increase in .
LPR is the cumulative effect after all changes have taken place, it is simply the sum of all of the coefficients.
1.3 Classical Assumptions
TS.1: Linear in parameters
The stochastic process follows the linear model
where is the sequence of errors or disturbances. Here, is the number of observations (time periods).
TS.2: No perfect collinearity
In the sample (and therefore in the underlying time series process), no independent variable is constant nor a perfect linear combination of the others.
TS.3: Zero conditional mean
For each , the expected value of the error , given the explanatory variables for all time periods, is zero, that is
- : contemporaneouly exogenous
- : strictly exogenous
TS.4 Homoskedasticity
Conditional on , the variance of is the same for all :
A sufficient condition is that volatility of the error is independent of the explanatory variables and that it is constant over time. In the time series context, homoskedasticity may also be easily violated, e.g. if the volatitliy of the dependent variable depends on regime changes
TS.5 No serial correlation
We do not assume the errors for different cross-sectional sampling because we already have the random sampling assumption, under which and are independent for any two observations and .
TS.1~TS.5 are the appropriate Gauss-Markov assumptions for time series applications.
OLS Sampling Variances
Under the time series Gauss-Markov Assumptions TS.1~TS.5, the variance of , conditional on is
where is the total sum of squares of and is the R square from the regression of on the other independent variables.
Unbiased Estimation of the error variance
Under TS.1~TS.5,
Gauss-Markov Theorem
Under TS.1~TS.5, the OLS estimators have the minimal variance of all linear unbiased estimators of the regression coefficients. This holds conditional as well as unconditional on the regressors.
TS.6 Normality
independently of
Normal sampling distributions
Under TS.1~TS.6, the OLS estimators have the usual normal distribution (conditional on ). The usual F and t tests are valid.
1.4 Trends and Seasonality
Trends and Detrends
We must recognize that some series contain a time trend in order to draw causal inference using time series data. Ignoring the fact that two sequences are trending in the same or opposite directions can lead us to falsely conclude that changes in on variable are actually caused by changes in another variable.
The phenomenon of finding a relationship between two or more trending variables simply because each is growing over time is an example of a spurious regression problem. We can add a linear trend term to the regression to โdetrendโ series, that is
can also be obtained by:
- regress each on and , denote the residuals as
- regress on without intercept
The from a regression on detrended data, which better reflects how well explain , is usually samller than that on trended data.
Seasonality and Deseasonalize
Often time-series data exhibits some periodicity, referred to seasonality. Seasonality can be dealt with by adding a set of seasonal dummies, e.g.
where are dummy variables indicating whether time period corresponds to the appropriate month. Here, January is the base month.
2. Further Issues for OLS on TS data
2.1 Stationary and Weakly Dependent
Stationary Deifinition
A stochastic process is strict stationary, if for every collection of indices , the joint distribution of is the same as that of for all integers .
Stationary implies:
- are identically distributed
- the adjacent terms may be highly correlated, but the nature of any correlation between adjacent terms is the same across all periods
Nonstationary: e.g. A process with a time trend.
Covariance Stationary Process
A stochastic proce is covariance stationary if its expected value, its variance, and its covariances are constant over time:
If a (strict) stationary process has a finite second moment, then it must be covariance stationary.
If and have stationary distributions,
- the coefficients donโt change within the sample
- the results can be extrapolated outside the sample
Autocorrelation Function (ACF)
The graph of is called correlogram.
Partial Autocorrelation Function (PACF)
the h-th PACF is the effect of on after controlling for all intermediate lages . Use OLS to estimate it.
Weakly Dependent Definition
A stationary time series is weakly dependent if and are almost independent as increases.
If for a covariance stationary process
weโll say this covariance stationary process is weakly dependent or asymptotically uncorrelated.
- For LLN and CLT to hold, the individual observations must not be too strongly related to each other; in particular their relation must become weaker the farther they are apart. It ensures in large sample there is sufficient randomness in the data.
- Time series counterpart of the โindependently distributedโ part of i.i.d.
For many applications, a stationary process is weakly dependent, but sometimes not.
Stationary and Weakly Dependent Time Series
White noise:
A sequence of random variables is said to be white noise if for all :
โ white noise is (srtict) stationary and weakly dependent
Indepnedent W.N.
- (extra condition): are independent.
Gaussian W.N.
- if
Moving average process of order one, MA(1)
- the process is a short moving average of an i.i.d. series
โ MA(1) is covariance stationary
- , ,
โ MA(1) is weakly dependent
thus the LLN and CLT can be applied to .
Autoregressive process of order one, AR(1)
- the process carries over a certain extent the value the previous period, plus random shocks from an i.i.d. series
โ AR(1) is covariance stationary (proof needs some technical). Then we know that with , this can happen only if .
โ AR(1) is weakly dependent
- โ thus
- Covariance between and for
since , multiply this last equation by and take expectation to obtain
Thus
2.2 Asymptotic Properties of OLS
Assumptions
TS.1* Linear parameters
Same as assumption TS.1 but now the dependent and independent variables are assumed to be stationary and weakly dependent. In particular, the LLN and CLT can be applied to sample averages.
TS.2* No perfect collinearity
Same as assumption TS.2
TS.3* Zero conditional mean
Now the explanatory variabels are assumed to be only contemporaneously exogenous rather than strictly exogenous, i.e.
Under TS.1*~TS.3*, the OLS estimators are consistent
Note: For consistency, it would suffice to assume that the explanatory variables are merely contemporaneously uncorrelated with the error term.
TS.4* Homoskedasticity
The errors are contemporaneously homoskedastic
TS.5* No serial correlation
Conditional on the explanatory variables in periods and , the errors are uncorrelated.
Under TS.1*~TS.5*, the OLS estimators are asymptotically normally distributed. Further, the usual OLS standard errors, t-stats, F-stats, are asympottically valid.
Trends
- Time series with deterministic time trends are non-stationary
- If they are stationary around the trend and in addition weakly dependent, they are called trend-stationary processes
- As long as a trend is included, all is well
2.3 Highly Persistent Time Series
Above shows that, provided the time series we use are weakly dependent, usual OLS inference procedures are valid under assumptions weaker than the classical linear model assumptions. Unfortunately, many economic time series cannot be characterized by weak dependence. Time series in these situations violate weak dependence because they are highly persistent (=strongly dependent). In this case, OLS methods are generally invalid. In some cases, transformations to weak dependence are possible.
Random Walk
The value today is the accumulation of all past shocks plus an initial value.
- The random walk is not covariance stationary because its variances and its covariance depend on time.
- It is also not weakly dependent because the correlation between observations vanished very slowly and this depends on how large is.
Random walk is a special case of a unit root process.
Random walks with drift
In addition to the usual random walk mechanism, there is a deterministic increase/decrease (=drift) in each period.
This leads to a linear time trend around whhich the series follows its random walk behaviour. As there is no clear direction in which the random walk develops, it may also wander away from the trend.
Transformation of highly persistent TS
Order of intergration:
- Weakly dependent time series are integreated of order zero .
- This means that nothing needs to be done to such series before using them in regression analysis.
- averges of such sequences already satisfy the standard limit theorems
- Unit root process, such as random walk (or with a drift), are said to be integrated of order one,
- This means that the first difference of the process is weakly dependent (and often stationary)
- is often said to be a difference-stationary process
- differencing is often a way to achieve weak dependence
Deciding whether a time series is
There are statistical tests for testing whether a time series is : unit root tests.
Alternatively, look at the sample first order autocorrelation:
If the sample first order autocorrelation is close to one, this suggests that the time series may be highly persistent (contains a unit root). Alternatively, the series may have a deterministic trend. Both unit root and tend may be eliminated by differencing.
Dynamically Complete Models
A model is said to be dynamically complete if enough lagged variables have been included as explanatory variables so that further lags do not help to explain the dependent variable
Dynamic completeness implies absence of serial correlation. If further lags actually belong in the regression, their omission will cause serial correlation (if the variables are serially correlated)
A set of explanatory variables is said to be sequentially exogenous if โenoughโ lagged explanatory variables have been included:
Sequential exogeneity is weaker than strict exogeneity. Sequential exogeneity is equivalent to dynamic completeness if the explanatory variables contain a lagged dependent variable.
3. Serial Correlation and Heteroskedasticity
3.1 Testing for Serial Correlation
Properties of OLS with Serially Correlated Errors
- OLS is no long BLUE in the presence of serial correlation
- the usual OLS standard error in the presence of serial correlation is invalid
- t-stats are no longer valid for testing single hypotheses. Since a smaller standard error means a larger t-stats, the usual t-stats will often be too large when
- The usual F and LM stat for testing multiple hypotheses are also invalid
- Goodness-of-Fit: our usual goodness-of-fit measures, , adjusted still work provided the data are stationary and weakly dependent.
Testing for AR(1) Serial Correlation with Strictly Exogenous Regressors
For model (multiple regressors model)
- Run the OLS regression of on and obtain the OLS residuals, , for all
- Run the regression of on , for all . obtaining the coefficient on and its t-stat
- Use (or p-value) to test against
Note that any serial correlation taht causes can be picked up by this test. While others , e.g. cannot.
The Durbin-Watson Test Under Classical Assumptions
Under assumptions TS.1~TS.5, the DW test is an exact test (whereas previous t test is only valid asymptotically) for AR(1) serial correlation
v.s.
Even with moderate sample sizes, the approximation is often pretty close. Therefore, tests based on DW and the t test based on are conceptually the same.
Reject if DW < , fail to reject if DW > . The DW test works with a lower and an upper bound for the critical value. In the area between the bounds the test result is inconclusive.
Note that if the regressors are not strictly exogenous, then neither the nor DW test will work.
The test can be easily generalized
- Run the OLS regression of on and obtain the OLS residuals, for all
- Run the regression of on for all to obtain the coefficient on and its t statistic
- Use to test v.s.
Testing for Higher Order Serial Correlation
Just include q lags of the residuals in the regression and test for joint significance
Test:
Using F, Wald, or LM test
An alternative to computing the F test is to use the Lagrange multiplier (LM) form of the statistic.
where is the R square of above step 2.
Q Tests for Serial Correlation
ย
Loading Comments...