Cramér's theorem (large deviations)

☆ Save On Wikipedia ↗

Cramér's theorem is a fundamental result in the theory of large deviations, a subdiscipline of probability theory. It determines the rate function of a series of iid random variables. A weak version of this result was first shown by Harald Cramér in 1938.

Statement

The logarithmic moment generating function (which is the cumulant-generating function) of a random variable is defined as:

Λ ( t ) = log ⁡ E ⁡ [ exp ⁡ ( t X 1 ) ] . {\displaystyle \Lambda (t)=\log \operatorname {E} [\exp(tX_{1})].} {\displaystyle \Lambda (t)=\log \operatorname {E} [\exp(tX_{1})].}

Let X 1 , X 2 , … {\displaystyle X_{1},X_{2},\dots } {\displaystyle X_{1},X_{2},\dots } be a sequence of iid real random variables with finite logarithmic moment generating function, i.e. Λ ( t ) < ∞ {\displaystyle \Lambda (t)<\infty } {\displaystyle \Lambda (t)<\infty } for all t ∈ R {\displaystyle t\in \mathbb {R} } {\displaystyle t\in \mathbb {R} }.

Then the Legendre transform of Λ {\displaystyle \Lambda } {\displaystyle \Lambda }:

Λ ∗ ( x ) := sup t ∈ R ( t x − Λ ( t ) ) {\displaystyle \Lambda ^{*}(x):=\sup _{t\in \mathbb {R} }\left(tx-\Lambda (t)\right)} {\displaystyle \Lambda ^{*}(x):=\sup _{t\in \mathbb {R} }\left(tx-\Lambda (t)\right)}

satisfies,

lim n → ∞ 1 n log ⁡ ( P ( ∑ i = 1 n X i ≥ n x ) ) = − Λ ∗ ( x ) {\displaystyle \lim _{n\to \infty }{\frac {1}{n}}\log \left(P\left(\sum _{i=1}^{n}X_{i}\geq nx\right)\right)=-\Lambda ^{*}(x)} {\displaystyle \lim _{n\to \infty }{\frac {1}{n}}\log \left(P\left(\sum _{i=1}^{n}X_{i}\geq nx\right)\right)=-\Lambda ^{*}(x)}

for all x > E ⁡ [ X 1 ] . {\displaystyle x>\operatorname {E} [X_{1}].} {\displaystyle x>\operatorname {E} [X_{1}].}[1]: 508 

In the terminology of the theory of large deviations the result can be reformulated as follows:

If X 1 , X 2 , … {\displaystyle X_{1},X_{2},\dots } {\displaystyle X_{1},X_{2},\dots } is a series of iid random variables, then the distributions ( L ( 1 n ∑ i = 1 n X i ) ) n ∈ N {\displaystyle \left({\mathcal {L}}({\tfrac {1}{n}}\sum _{i=1}^{n}X_{i})\right)_{n\in \mathbb {N} }} {\displaystyle \left({\mathcal {L}}({\tfrac {1}{n}}\sum _{i=1}^{n}X_{i})\right)_{n\in \mathbb {N} }} satisfy a large deviation principle with rate function Λ ∗ {\displaystyle \Lambda ^{*}} {\displaystyle \Lambda ^{*}}, where L ( X ) {\displaystyle {\mathcal {L}}(X)} {\displaystyle {\mathcal {L}}(X)} denotes the distribution of the random variable X {\displaystyle X} {\displaystyle X}.[1]: 514 

References

  1. Klenke, Achim (2008). Probability Theory. Berlin: Springer. doi:10.1007/978-1-84800-048-3. ISBN 978-1-84800-047-6.