Donsker's Theorem, also known as the Donsker-Varadhan theorem or the invariance principle, is a fundamental result in probability theory and statistics. It provides a foundation for the asymptotic theory of empirical processes and has far-reaching implications in various fields, including statistical inference, stochastic processes, and mathematical finance. The theorem states that the empirical process, which is a random process derived from a sequence of independent and identically distributed (i.i.d.) random variables, converges weakly to a Gaussian process, known as the Brownian bridge, as the sample size increases.
Introduction to Donsker’s Theorem
Donsker’s Theorem was first proved by Monroe Donsker in 1951, and later generalized by S.R.S. Varadhan. The theorem has since become a cornerstone of modern probability theory and statistics. It establishes a deep connection between the empirical process and the Brownian bridge, which is a Gaussian process with zero mean and covariance function given by cov(t, s) = min(t, s) - ts. The Brownian bridge is a continuous stochastic process that arises as the limit of the empirical process, and it plays a crucial role in the asymptotic theory of statistical inference.
Statement of Donsker’s Theorem
Let X1, X2, …, Xn be a sequence of i.i.d. random variables with mean 0 and variance 1. The empirical process is defined as:
Empirical Process: Gn(t) = √n(Fn(t) - t), where Fn(t) is the empirical distribution function.
Donsker's Theorem states that the empirical process Gn(t) converges weakly to the Brownian bridge B(t) as n → ∞. The convergence is in the sense of the uniform topology on the space of continuous functions on [0, 1].
| Empirical Process | Brownian Bridge |
|---|---|
| Gn(t) = √n(Fn(t) - t) | B(t) = W(t) - tW(1) |
| Mean: 0 | Mean: 0 |
| Variance: 1 | Variance: t(1 - t) |
Proof of Donsker’s Theorem
The proof of Donsker’s Theorem involves several technical steps, including the use of the Central Limit Theorem (CLT) and the Continuous Mapping Theorem (CMT). The basic idea is to show that the empirical process Gn(t) converges weakly to the Brownian bridge B(t) by establishing the convergence of the finite-dimensional distributions and the tightness of the empirical process.
Finite-Dimensional Convergence
Let t1, t2, …, tk be a set of distinct points in [0, 1]. The finite-dimensional distribution of the empirical process is given by:
Gn(t1), Gn(t2), ..., Gn(tk) → B(t1), B(t2), ..., B(tk)
as n → ∞, where the convergence is in distribution. This result follows from the CLT and the fact that the empirical process is a linear combination of i.i.d. random variables.
Tightness of the Empirical Process
To establish the tightness of the empirical process, we need to show that the empirical process is uniformly bounded and that the oscillations of the empirical process are controlled. This can be achieved by using the boundedness of the empirical process and the equicontinuity of the empirical process.
What is the significance of Donsker's Theorem in statistical inference?
+Donsker's Theorem provides a foundation for the asymptotic theory of statistical inference. It enables the use of Gaussian process theory to analyze the asymptotic behavior of statistical estimators and tests, and it has far-reaching implications in various fields, including hypothesis testing, confidence sets, and non-parametric statistics.
Applications of Donsker’s Theorem
Donsker’s Theorem has numerous applications in statistical inference, including:
- Hypothesis Testing: Donsker's Theorem provides a framework for testing hypotheses about the distribution of a population. It enables the use of Gaussian process theory to analyze the asymptotic behavior of test statistics and to construct confidence sets.
- Confidence Sets: Donsker's Theorem can be used to construct confidence sets for population parameters. The theorem provides a framework for analyzing the asymptotic behavior of confidence sets and for constructing confidence sets with guaranteed coverage probabilities.
- Non-Parametric Statistics: Donsker's Theorem has far-reaching implications in non-parametric statistics. It enables the use of Gaussian process theory to analyze the asymptotic behavior of non-parametric estimators and tests, and it provides a framework for constructing confidence sets and testing hypotheses about the distribution of a population.
In conclusion, Donsker’s Theorem is a fundamental result in probability theory and statistics. It provides a foundation for the asymptotic theory of empirical processes and has far-reaching implications in various fields, including statistical inference, stochastic processes, and mathematical finance. The theorem has numerous applications in hypothesis testing, confidence sets, and non-parametric statistics, and it continues to be an active area of research in modern probability theory and statistics.
