Chi-squared distribution


































chi-squared

Probability density function

Chi-square pdf.svg

Cumulative distribution function

Chi-square cdf.svg
Notation
χ2(k)displaystyle chi ^2(k);displaystyle chi ^2(k); or χk2displaystyle chi _k^2!chi _k^2!
Parameters
k∈N>0  displaystyle kin mathbb N _>0~~kin mathbb N _>0~~ (known as "degrees of freedom")
Support
x∈(0,+∞)displaystyle xin (0,+infty );displaystyle xin (0,+infty ); if k=1displaystyle k=1k=1, otherwise x∈[0,+∞)displaystyle xin [0,+infty );displaystyle xin [0,+infty );
PDF12k/2Γ(k/2)xk/2−1e−x/2displaystyle frac 12^k/2Gamma (k/2);x^k/2-1e^-x/2;displaystyle frac 12^k/2Gamma (k/2);x^k/2-1e^-x/2;
CDF1Γ(k/2)γ(k2,x2)displaystyle frac 1Gamma (k/2);gamma left(frac k2,,frac x2right);displaystyle frac 1Gamma (k/2);gamma left(frac k2,,frac x2right);
Meankdisplaystyle kk
Median≈k(1−29k)3displaystyle approx kbigg (1-frac 29kbigg )^3;displaystyle approx kbigg (1-frac 29kbigg )^3;
Modemax(k−2,0)displaystyle max(k-2,0);displaystyle max(k-2,0);
Variance2kdisplaystyle 2k;displaystyle 2k;
Skewness8/kdisplaystyle scriptstyle sqrt 8/k,scriptstyle sqrt 8/k,
Ex. kurtosis12kdisplaystyle frac 12kdisplaystyle frac 12k
Entropyk2+log⁡(2Γ(k/2))+(1−k/2)ψ(k/2)(nats)displaystyle beginalignedtfrac k2&+log(2Gamma (k/2))\&!+(1-k/2)psi (k/2),scriptstyle text(nats)endaligneddisplaystyle beginalignedtfrac k2&+log(2Gamma (k/2))\&!+(1-k/2)psi (k/2),scriptstyle text(nats)endaligned
MGF(1−2t)−k/2 for t<12displaystyle (1-2t)^-k/2text for t<frac 12;displaystyle (1-2t)^-k/2text for t<frac 12;
CF
(1−2it)−k/2displaystyle (1-2it)^-k/2displaystyle (1-2it)^-k/2      [1]
PGF(1−2ln⁡t)−k/2 for 0<t<edisplaystyle (1-2ln t)^-k/2text for 0<t<sqrt e;displaystyle (1-2ln t)^-k/2text for 0<t<sqrt e;

In probability theory and statistics, the chi-squared distribution (also chi-square or χ2-distribution) with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables. The chi-squared distribution is a special case of the gamma distribution and is one of the most widely used probability distributions in inferential statistics, notably in hypothesis testing or in construction of confidence intervals.[2][3][4][5] When it is being distinguished from the more general noncentral chi-squared distribution, this distribution is sometimes called the central chi-squared distribution.


The chi-squared distribution is used in the common chi-squared tests for goodness of fit of an observed distribution to a theoretical one, the independence of two criteria of classification of qualitative data, and in confidence interval estimation for a population standard deviation of a normal distribution from a sample standard deviation. Many other statistical tests also use this distribution, such as Friedman's analysis of variance by ranks.




Contents





  • 1 Definition


  • 2 Introduction


  • 3 Characteristics

    • 3.1 Probability density function


    • 3.2 Cumulative distribution function


    • 3.3 Additivity


    • 3.4 Sample mean


    • 3.5 Entropy


    • 3.6 Noncentral moments


    • 3.7 Cumulants


    • 3.8 Asymptotic properties



  • 4 Relation to other distributions


  • 5 Generalizations

    • 5.1 Linear combination


    • 5.2 Chi-squared distributions

      • 5.2.1 Noncentral chi-squared distribution


      • 5.2.2 Generalized chi-squared distribution



    • 5.3 Gamma, exponential, and related distributions



  • 6 Occurrence and applications


  • 7 Table of χ2 values vs p-values


  • 8 History and name


  • 9 See also


  • 10 References


  • 11 Further reading


  • 12 External links




Definition


If Z1, ..., Zk are independent, standard normal random variables, then the sum of their squares,


Q =∑i=1kZi2,displaystyle Q =sum _i=1^kZ_i^2,Q =sum _i=1^kZ_i^2,

is distributed according to the chi-squared distribution with k degrees of freedom. This is usually denoted as


Q ∼ χ2(k)  or  Q ∼ χk2.displaystyle Q sim chi ^2(k) textor Q sim chi _k^2.Q sim chi ^2(k) textor Q sim chi _k^2.

The chi-squared distribution has one parameter: k, a positive integer that specifies the number of degrees of freedom (the number of Zi’s).



Introduction


The chi-squared distribution is used primarily in hypothesis testing, and to a lesser extent for confidence intervals for population variance when the underlying distribution is normal. Unlike more widely known distributions such as the normal distribution and the exponential distribution, the chi-squared distribution is not as often applied in the direct modeling of natural phenomena. It arises in the following hypothesis tests, among others.



  • Chi-squared test of independence in contingency tables


  • Chi-squared test of goodness of fit of observed data to hypothetical distributions


  • Likelihood-ratio test for nested models


  • Log-rank test in survival analysis


  • Cochran–Mantel–Haenszel test for stratified contingency tables

It is also a component of the definition of the t-distribution and the F-distribution used in t-tests, analysis of variance, and regression analysis.


The primary reason that the chi-squared distribution is used extensively in hypothesis testing is its relationship to the normal distribution. Many hypothesis tests use a test statistic, such as the t-statistic in a t-test. For these hypothesis tests, as the sample size, n, increases, the sampling distribution of the test statistic approaches the normal distribution (central limit theorem). Because the test statistic (such as t) is asymptotically normally distributed, provided the sample size is sufficiently large, the distribution used for hypothesis testing may be approximated by a normal distribution. Testing hypotheses using a normal distribution is well understood and relatively easy. The simplest chi-squared distribution is the square of a standard normal distribution. So wherever a normal distribution could be used for a hypothesis test, a chi-squared distribution could be used.


Specifically, suppose that Z is a standard normal random variable, with mean = 0 and variance = 1. Z ~ N(0,1). A sample drawn at random from Z is a sample from the distribution shown in the graph of the standard normal distribution.
Define a new random variable Q. To generate a random sample from Q, take a sample from Z and square the value. The distribution of the squared values is given by the random variable Q = Z2. The distribution of the random variable Q is an example of a chi-squared distribution:  Q ∼ χ12.displaystyle Q sim chi _1^2. Q sim chi _1^2. The subscript 1 indicates that this particular chi-squared distribution is constructed from only 1 standard normal distribution. A chi-squared distribution constructed by squaring a single standard normal distribution is said to have 1 degree of freedom. Thus, as the sample size for a hypothesis test increases, the distribution of the test statistic approaches a normal distribution, and the distribution of the square of the test statistic approaches a chi-squared distribution. Just as extreme values of the normal distribution have low probability (and give small p-values), extreme values of the chi-squared distribution have low probability.


An additional reason that the chi-squared distribution is widely used is that it is a member of the class of likelihood ratio tests (LRT).[6] LRT's have several desirable properties; in particular, LRT's commonly provide the highest power to reject the null hypothesis (Neyman–Pearson lemma). However, the normal and chi-squared approximations are only valid asymptotically. For this reason, it is preferable to use the t distribution rather than the normal approximation or the chi-squared approximation for small sample size. Similarly, in analyses of contingency tables, the chi-squared approximation will be poor for small sample size, and it is preferable to use Fisher's exact test. Ramsey shows that the exact binomial test is always more powerful than the normal approximation.[7]


Lancaster shows the connections among the binomial, normal, and chi-squared distributions, as follows.[8] De Moivre and Laplace established that a binomial distribution could be approximated by a normal distribution. Specifically they showed the asymptotic normality of the random variable


χ=m−NpNpqdisplaystyle chi =m-Np over sqrt Npqdisplaystyle chi =m-Np over sqrt Npq

where m is the observed number of successes in N trials, where the probability of success is p, and q = 1 − p.


Squaring both sides of the equation gives


χ2=(m−Np)2Npqdisplaystyle chi ^2=(m-Np)^2 over Npqdisplaystyle chi ^2=(m-Np)^2 over Npq

Using N = Np + N(1 − p), N = m + (Nm), and q = 1 − p, this equation simplifies to


χ2=(m−Np)2Np+(N−m−Nq)2Nqdisplaystyle chi ^2=(m-Np)^2 over Np+(N-m-Nq)^2 over Nqdisplaystyle chi ^2=(m-Np)^2 over Np+(N-m-Nq)^2 over Nq

The expression on the right is of the form that Pearson would generalize to the form:


χ2=∑i=1n(Oi−Ei)2Eidisplaystyle chi ^2=sum _i=1^nfrac (O_i-E_i)^2E_idisplaystyle chi ^2=sum _i=1^nfrac (O_i-E_i)^2E_i

where



χ2displaystyle chi ^2chi ^2 = Pearson's cumulative test statistic, which asymptotically approaches a χ2displaystyle chi ^2chi ^2 distribution.


Oidisplaystyle O_iO_i = the number of observations of type i.


Ei=Npidisplaystyle E_i=Np_iE_i=Np_i = the expected (theoretical) frequency of type i, asserted by the null hypothesis that the fraction of type i in the population is pidisplaystyle p_ip_i


ndisplaystyle nn = the number of cells in the table.

In the case of a binomial outcome (flipping a coin), the binomial distribution may be approximated by a normal distribution (for sufficiently large n). Because the square of a standard normal distribution is the chi-squared distribution with one degree of freedom, the probability of a result such as 1 heads in 10 trials can be approximated either by the normal or the chi-squared distribution. However, many problems involve more than the two possible outcomes of a binomial, and instead require 3 or more categories, which leads to the multinomial distribution. Just as de Moivre and Laplace sought for and found the normal approximation to the binomial, Pearson sought for and found a multivariate normal approximation to the multinomial distribution. Pearson showed that the chi-squared distribution, the sum of multiple normal distributions, was such an approximation to the multinomial distribution [8]



Characteristics


Further properties of the chi-squared distribution can be found in the box at the upper right corner of this article.



Probability density function


The probability density function (pdf) of the chi-square distribution is


f(x;k)={xk2−1e−x22k2Γ(k2),x>0;0,otherwise.displaystyle f(x;,k)=begincasesdfrac x^frac k2-1e^-frac x22^frac k2Gamma left(frac k2right),&x>0;\0,&textotherwise.endcasesdisplaystyle f(x;,k)=begincasesdfrac x^frac k2-1e^-frac x22^frac k2Gamma left(frac k2right),&x>0;\0,&textotherwise.endcases

where Γ(k/2)textstyle Gamma (k/2)textstyle Gamma (k/2) denotes the gamma function, which has closed-form values for integer k.


For derivations of the pdf in the cases of one, two and k degrees of freedom, see Proofs related to chi-squared distribution.



Cumulative distribution function




Chernoff bound for the CDF and tail (1-CDF) of a chi-squared random variable with ten degrees of freedom (k = 10)


Its cumulative distribution function is:


F(x;k)=γ(k2,x2)Γ(k2)=P(k2,x2),displaystyle F(x;,k)=frac gamma (frac k2,,frac x2)Gamma (frac k2)=Pleft(frac k2,,frac x2right),F(x;,k)=frac gamma (frac k2,,frac x2)Gamma (frac k2)=Pleft(frac k2,,frac x2right),

where γ(s,t)displaystyle gamma (s,t)displaystyle gamma (s,t) is the lower incomplete gamma function and P(s,t)textstyle P(s,t)textstyle P(s,t) is the regularized gamma function.


In a special case of k = 2 this function has a simple form:[citation needed]


F(x;2)=1−e−x/2displaystyle F(x;,2)=1-e^-x/2displaystyle F(x;,2)=1-e^-x/2

and the integer recurrence of the gamma function makes it easy to compute for other small even k.


Tables of the chi-squared cumulative distribution function are widely available and the function is included in many spreadsheets and all statistical packages.


Letting z≡x/kdisplaystyle zequiv x/kzequiv x/k, Chernoff bounds on the lower and upper tails of the CDF may be obtained.[9] For the cases when 0<z<1displaystyle 0<z<10<z<1 (which include all of the cases when this CDF is less than half):


F(zk;k)≤(ze1−z)k/2.displaystyle F(zk;,k)leq (ze^1-z)^k/2.F(zk;,k)leq (ze^1-z)^k/2.

The tail bound for the cases when z>1displaystyle z>1z>1, similarly, is


1−F(zk;k)≤(ze1−z)k/2.displaystyle 1-F(zk;,k)leq (ze^1-z)^k/2.1-F(zk;,k)leq (ze^1-z)^k/2.

For another approximation for the CDF modeled after the cube of a Gaussian, see under Noncentral chi-squared distribution.



Additivity


It follows from the definition of the chi-squared distribution that the sum of independent chi-squared variables is also chi-squared distributed. Specifically, if Xii=1n are independent chi-squared variables with kii=1n degrees of freedom, respectively, then Y = X1 + ⋯ + Xn is chi-squared distributed with k1 + ⋯ + kn degrees of freedom.



Sample mean


The sample mean of ndisplaystyle nn i.i.d. chi-squared variables of degree kdisplaystyle kk is distributed according to a gamma distribution with shape αdisplaystyle alpha alpha and scale θdisplaystyle theta theta parameters:


X¯=1n∑i=1nXi∼Gamma⁡(α=nk/2,θ=2/n)where Xi∼χ2(k)displaystyle bar X=frac 1nsum _i=1^nX_isim operatorname Gamma left(alpha =n,k/2,theta =2/nright)qquad textwhere X_isim chi ^2(k)displaystyle bar X=frac 1nsum _i=1^nX_isim operatorname Gamma left(alpha =n,k/2,theta =2/nright)qquad textwhere X_isim chi ^2(k)

Asymptotically, given that for a scale parameter αdisplaystyle alpha alpha going to infinity, a Gamma distribution converges towards a normal distribution with expectation μ=α⋅θdisplaystyle mu =alpha cdot theta mu =alpha cdot theta and variance σ2=αθ2displaystyle sigma ^2=alpha ,theta ^2sigma ^2=alpha ,theta ^2, the sample mean converges towards:


X¯→n→∞N(μ=k,σ2=2k/n)displaystyle bar Xxrightarrow nto infty N(mu =k,sigma ^2=2,k/n)bar Xxrightarrow nto infty N(mu =k,sigma ^2=2,k/n)

Note that we would have obtained the same result invoking instead the central limit theorem, noting that for each chi-squared variable of degree kdisplaystyle kk the expectation is kdisplaystyle kk , and its variance 2kdisplaystyle 2,k2,k (and hence the variance of the sample mean X¯displaystyle bar Xbar X being σ2=2k/ndisplaystyle sigma ^2=2,k/nsigma ^2=2,k/n).



Entropy


The differential entropy is given by


h=∫0∞f(x;k)ln⁡f(x;k)dx=k2+ln⁡[2Γ(k2)]+(1−k2)ψ[k2],displaystyle h=int _0^infty f(x;,k)ln f(x;,k),dx=frac k2+ln left[2,Gamma left(frac k2right)right]+left(1-frac k2right),psi !left[frac k2right],displaystyle h=int _0^infty f(x;,k)ln f(x;,k),dx=frac k2+ln left[2,Gamma left(frac k2right)right]+left(1-frac k2right),psi !left[frac k2right],

where ψ(x) is the Digamma function.


The chi-squared distribution is the maximum entropy probability distribution for a random variate X for which E⁡(X)=kdisplaystyle operatorname E (X)=kdisplaystyle operatorname E (X)=k and E⁡(ln⁡(X))=ψ(k/2)+ln⁡(2)displaystyle operatorname E (ln(X))=psi (k/2)+ln(2)displaystyle operatorname E (ln(X))=psi (k/2)+ln(2) are fixed. Since the chi-squared is in the family of gamma distributions, this can be derived by substituting appropriate values in the Expectation of the log moment of gamma. For derivation from more basic principles, see the derivation in moment-generating function of the sufficient statistic.



Noncentral moments


The moments about zero of a chi-squared distribution with k degrees of freedom are given by[10][11]


E⁡(Xm)=k(k+2)(k+4)⋯(k+2m−2)=2mΓ(m+k2)Γ(k2).displaystyle operatorname E (X^m)=k(k+2)(k+4)cdots (k+2m-2)=2^mfrac Gamma left(m+frac k2right)Gamma left(frac k2right).displaystyle operatorname E (X^m)=k(k+2)(k+4)cdots (k+2m-2)=2^mfrac Gamma left(m+frac k2right)Gamma left(frac k2right).


Cumulants


The cumulants are readily obtained by a (formal) power series expansion of the logarithm of the characteristic function:


κn=2n−1(n−1)!kdisplaystyle kappa _n=2^n-1(n-1)!,kkappa _n=2^n-1(n-1)!,k


Asymptotic properties


By the central limit theorem, because the chi-squared distribution is the sum of k independent random variables with finite mean and variance, it converges to a normal distribution for large k. For many practical purposes, for k > 50 the distribution is sufficiently close to a normal distribution for the difference to be ignored.[12] Specifically, if X ~ χ2(k), then as k tends to infinity, the distribution of (X−k)/2kdisplaystyle (X-k)/sqrt 2k(X-k)/sqrt 2k tends to a standard normal distribution. However, convergence is slow as the skewness is 8/kdisplaystyle sqrt 8/ksqrt 8/k and the excess kurtosis is 12/k.


The sampling distribution of ln(χ2) converges to normality much faster than the sampling distribution of χ2,[13] as the logarithm removes much of the asymmetry.[14] Other functions of the chi-squared distribution converge more rapidly to a normal distribution. Some examples are:


  • If X ~ χ2(k) then 2Xdisplaystyle scriptstyle sqrt 2Xscriptstyle sqrt 2X is approximately normally distributed with mean 2k−1displaystyle scriptstyle sqrt 2k-1scriptstyle sqrt 2k-1 and unit variance (1922, by R. A. Fisher, see (18.23), p. 426 of.[4]

  • If X ~ χ2(k) then X/k3displaystyle scriptstyle sqrt[3]X/kscriptstyle sqrt[3]X/k is approximately normally distributed with mean 1−2/(9k)displaystyle scriptstyle 1-2/(9k)scriptstyle 1-2/(9k) and variance 2/(9k).displaystyle scriptstyle 2/(9k).scriptstyle 2/(9k).[15] This is known as the Wilson–Hilferty transformation, see (18.24), p. 426 of.[4]


Relation to other distributions




Approximate formula for median compared with numerical quantile (top). Difference between numerical quantile and approximate formula (bottom).


  • As k→∞displaystyle kto infty kto infty , (χk2−k)/2k →d N(0,1)displaystyle (chi _k^2-k)/sqrt 2k~xrightarrow d N(0,1),(chi _k^2-k)/sqrt 2k~xrightarrow d N(0,1), (normal distribution)


  • χk2∼χ′k2(0)displaystyle chi _k^2sim chi '_k^2(0)chi _k^2sim chi '_k^2(0) (noncentral chi-squared distribution with non-centrality parameter λ=0displaystyle lambda =0lambda =0)

  • If Y∼F(ν1,ν2)displaystyle Ysim mathrm F (nu _1,nu _2)displaystyle Ysim mathrm F (nu _1,nu _2) then X=limν2→∞ν1Ydisplaystyle X=lim _nu _2to infty nu _1Ydisplaystyle X=lim _nu _2to infty nu _1Y has the chi-squared distribution χν12displaystyle chi _nu _1^2chi _nu _1^2

  • As a special case, if Y∼F(1,ν2)displaystyle Ysim mathrm F (1,nu _2),displaystyle Ysim mathrm F (1,nu _2), then X=limν2→∞Ydisplaystyle X=lim _nu _2to infty Y,displaystyle X=lim _nu _2to infty Y, has the chi-squared distribution χ12displaystyle chi _1^2chi _1^2

  • ‖Ni=1,…,k(0,1)‖2∼χk2boldsymbol N_i=1,ldots ,k(0,1)boldsymbol N_i=1,ldots ,k(0,1) (The squared norm of k standard normally distributed variables is a chi-squared distribution with k degrees of freedom)

  • If X∼χ2(ν)displaystyle Xsim chi ^2(nu ),displaystyle Xsim chi ^2(nu ), and c>0displaystyle c>0,c>0,, then cX∼Γ(k=ν/2,θ=2c)displaystyle cXsim Gamma (k=nu /2,theta =2c),displaystyle cXsim Gamma (k=nu /2,theta =2c),. (gamma distribution)

  • If X∼χk2displaystyle Xsim chi _k^2Xsim chi _k^2 then X∼χkdisplaystyle sqrt Xsim chi _ksqrt Xsim chi _k (chi distribution)

  • If X∼χ2(2)displaystyle Xsim chi ^2(2)displaystyle Xsim chi ^2(2), then X∼Exp⁡(1/2)displaystyle Xsim operatorname Exp (1/2)displaystyle Xsim operatorname Exp (1/2) is an exponential distribution. (See gamma distribution for more.)

  • If X∼χ2(2k)displaystyle Xsim chi ^2(2k)displaystyle Xsim chi ^2(2k), then X∼Erlang⁡(k,1/2)displaystyle Xsim operatorname Erlang (k,1/2)displaystyle Xsim operatorname Erlang (k,1/2) is an Erlang distribution.

  • If X∼Erlang⁡(k,λ)displaystyle Xsim operatorname Erlang (k,lambda )displaystyle Xsim operatorname Erlang (k,lambda ), then 2λX∼χ2k2displaystyle 2lambda Xsim chi _2k^2displaystyle 2lambda Xsim chi _2k^2

  • If X∼Rayleigh⁡(1)displaystyle Xsim operatorname Rayleigh (1),displaystyle Xsim operatorname Rayleigh (1), (Rayleigh distribution) then X2∼χ2(2)displaystyle X^2sim chi ^2(2),X^2sim chi ^2(2),

  • If X∼Maxwell⁡(1)displaystyle Xsim operatorname Maxwell (1),displaystyle Xsim operatorname Maxwell (1), (Maxwell distribution) then X2∼χ2(3)displaystyle X^2sim chi ^2(3),X^2sim chi ^2(3),

  • If X∼χ2(ν)displaystyle Xsim chi ^2(nu )Xsim chi ^2(nu ) then 1X∼Inv-⁡χ2(ν)displaystyle tfrac 1Xsim operatorname Inv- chi ^2(nu ),displaystyle tfrac 1Xsim operatorname Inv- chi ^2(nu ), (Inverse-chi-squared distribution)

  • The chi-squared distribution is a special case of type 3 Pearson distribution

  • If X∼χ2(ν1)displaystyle Xsim chi ^2(nu _1),Xsim chi ^2(nu _1), and Y∼χ2(ν2)displaystyle Ysim chi ^2(nu _2),Ysim chi ^2(nu _2), are independent then XX+Y∼Beta⁡(ν12,ν22)displaystyle tfrac XX+Ysim operatorname Beta (tfrac nu _12,tfrac nu _22),displaystyle tfrac XX+Ysim operatorname Beta (tfrac nu _12,tfrac nu _22), (beta distribution)

  • If X∼U⁡(0,1)displaystyle Xsim operatorname U (0,1),displaystyle Xsim operatorname U (0,1), (uniform distribution) then −2log⁡(X)∼χ2(2)displaystyle -2log(X)sim chi ^2(2),displaystyle -2log(X)sim chi ^2(2),


  • χ2(6)displaystyle chi ^2(6),chi ^2(6), is a transformation of Laplace distribution

  • If Xi∼Laplace⁡(μ,β)displaystyle X_isim operatorname Laplace (mu ,beta ),displaystyle X_isim operatorname Laplace (mu ,beta ), then ∑i=1n2|Xi−μ|β∼χ2(2n)displaystyle sum _i=1^nfrac 2beta sim chi ^2(2n),displaystyle sum _i=1^nfrac 2beta sim chi ^2(2n),

  • If Xidisplaystyle X_iX_i follows the generalized normal distribution (version 1) with parameters μ,α,βdisplaystyle mu ,alpha ,beta displaystyle mu ,alpha ,beta then ∑i=1n2|Xi−μ|βα∼χ2(2nβ)displaystyle sum _i=1^nfrac X_i-mu alpha sim chi ^2left(frac 2nbeta right),displaystyle sum _i=1^nfrac X_i-mu alpha sim chi ^2left(frac 2nbeta right), [16]

  • chi-squared distribution is a transformation of Pareto distribution


  • Student's t-distribution is a transformation of chi-squared distribution


  • Student's t-distribution can be obtained from chi-squared distribution and normal distribution


  • Noncentral beta distribution can be obtained as a transformation of chi-squared distribution and Noncentral chi-squared distribution


  • Noncentral t-distribution can be obtained from normal distribution and chi-squared distribution

A chi-squared variable with k degrees of freedom is defined as the sum of the squares of k independent standard normal random variables.


If Y is a k-dimensional Gaussian random vector with mean vector μ and rank k covariance matrix C, then X = (Yμ)TC−1(Y − μ) is chi-squared distributed with k degrees of freedom.


The sum of squares of statistically independent unit-variance Gaussian variables which do not have mean zero yields a generalization of the chi-squared distribution called the noncentral chi-squared distribution.


If Y is a vector of k i.i.d. standard normal random variables and A is a k×k symmetric, idempotent matrix with rank k−n then the quadratic form YTAY is chi-squared distributed with k−n degrees of freedom.


If Σdisplaystyle Sigma Sigma is a p×pdisplaystyle ptimes pptimes p positive-semidefinite covariance matrix with strictly positive diagonal entries, then for X∼N(0,Σ)displaystyle Xsim N(0,Sigma )displaystyle Xsim N(0,Sigma ) and wdisplaystyle ww a random pdisplaystyle pp-vector independent of Xdisplaystyle XX such that w1+⋯+wp=1displaystyle w_1+cdots +w_p=1displaystyle w_1+cdots +w_p=1 and wi≥0,i=1,⋯,p,displaystyle w_igeq 0,i=1,cdots ,p,displaystyle w_igeq 0,i=1,cdots ,p, it holds that


1(w1X1,⋯,wpXp)Σ(w1X1,⋯,wpXp)⊤∼χ12.displaystyle frac 1left(frac w_1X_1,cdots ,frac w_pX_pright)Sigma left(frac w_1X_1,cdots ,frac w_pX_pright)^top sim chi _1^2.displaystyle frac 1left(frac w_1X_1,cdots ,frac w_pX_pright)Sigma left(frac w_1X_1,cdots ,frac w_pX_pright)^top sim chi _1^2.[14]


The chi-squared distribution is also naturally related to other distributions arising from the Gaussian. In particular,



  • Y is F-distributed, Y ~ F(k1,k2) if Y=X1/k1X2/k2displaystyle scriptstyle Y=frac X_1/k_1X_2/k_2scriptstyle Y=frac X_1/k_1X_2/k_2 where X1 ~ χ²(k1) and X2  ~ χ²(k2) are statistically independent.

  • If X1  ~  χ2k1 and X2  ~  χ2k2 are statistically independent, then X1 + X2  ~ χ2k1+k2. If X1 and X2 are not independent, then X1 + X2 is not chi-squared distributed.


Generalizations


The chi-squared distribution is obtained as the sum of the squares of k independent, zero-mean, unit-variance Gaussian random variables. Generalizations of this distribution can be obtained by summing the squares of other types of Gaussian random variables. Several such distributions are described below.



Linear combination


If X1,…,Xndisplaystyle X_1,ldots ,X_nX_1,ldots ,X_n are chi square random variables and a1,…,an∈R>0displaystyle a_1,ldots ,a_nin mathbb R _>0displaystyle a_1,ldots ,a_nin mathbb R _>0, then a closed expression for the distribution of X=∑i=1naiXidisplaystyle X=sum _i=1^na_iX_idisplaystyle X=sum _i=1^na_iX_i is not known. It may be, however, approximated efficiently using the property of characteristic functions of chi-squared random variables.[17]



Chi-squared distributions



Noncentral chi-squared distribution



The noncentral chi-squared distribution is obtained from the sum of the squares of independent Gaussian random variables having unit variance and nonzero means.



Generalized chi-squared distribution



The generalized chi-squared distribution is obtained from the quadratic form z′Az where z is a zero-mean Gaussian vector having an arbitrary covariance matrix, and A is an arbitrary matrix.



Gamma, exponential, and related distributions


The chi-squared distribution X∼χk2displaystyle Xsim chi _k^2displaystyle Xsim chi _k^2 is a special case of the gamma distribution, in that X∼Γ(k2,12)displaystyle Xsim Gamma left(frac k2,frac 12right)displaystyle Xsim Gamma left(frac k2,frac 12right) using the rate parameterization of the gamma distribution (or
X∼Γ(k2,2)displaystyle Xsim Gamma left(frac k2,2right)displaystyle Xsim Gamma left(frac k2,2right) using the scale parameterization of the gamma distribution)
where k is an integer.


Because the exponential distribution is also a special case of the gamma distribution, we also have that if X∼χ22displaystyle Xsim chi _2^2displaystyle Xsim chi _2^2, then X∼Exp⁡(12)displaystyle Xsim operatorname Exp left(frac 12right)displaystyle Xsim operatorname Exp left(frac 12right) is an exponential distribution.


The Erlang distribution is also a special case of the gamma distribution and thus we also have that if X∼χk2displaystyle Xsim chi _k^2displaystyle Xsim chi _k^2 with even k, then X is Erlang distributed with shape parameter k/2 and scale parameter 1/2.



Occurrence and applications


The chi-squared distribution has numerous applications in inferential statistics, for instance in chi-squared tests and in estimating variances. It enters the problem of estimating the mean of a normally distributed population and the problem of estimating the slope of a regression line via its role in Student's t-distribution. It enters all analysis of variance problems via its role in the F-distribution, which is the distribution of the ratio of two independent chi-squared random variables, each divided by their respective degrees of freedom.


Following are some of the most common situations in which the chi-squared distribution arises from a Gaussian-distributed sample.


  • if X1, ..., Xn are i.i.d. N(μ, σ2) random variables, then ∑i=1n(Xi−X¯)2∼σ2χn−12displaystyle sum _i=1^n(X_i-bar X)^2sim sigma ^2chi _n-1^2sum _i=1^n(X_i-bar X)^2sim sigma ^2chi _n-1^2 where X¯=1n∑i=1nXidisplaystyle bar X=frac 1nsum _i=1^nX_ibar X=frac 1nsum _i=1^nX_i.

  • The box below shows some statistics based on Xi ∼ Normal(μi, σ2i), i = 1, ⋯, k, independent random variables that have probability distributions related to the chi-squared distribution:











NameStatistic
chi-squared distribution
∑i=1k(Xi−μiσi)2displaystyle sum _i=1^kleft(frac X_i-mu _isigma _iright)^2sum _i=1^kleft(frac X_i-mu _isigma _iright)^2
noncentral chi-squared distribution
∑i=1k(Xiσi)2displaystyle sum _i=1^kleft(frac X_isigma _iright)^2sum _i=1^kleft(frac X_isigma _iright)^2
chi distribution
∑i=1k(Xi−μiσi)2displaystyle sqrt sum _i=1^kleft(frac X_i-mu _isigma _iright)^2sqrt sum _i=1^kleft(frac X_i-mu _isigma _iright)^2
noncentral chi distribution
∑i=1k(Xiσi)2displaystyle sqrt sum _i=1^kleft(frac X_isigma _iright)^2sqrt sum _i=1^kleft(frac X_isigma _iright)^2

The chi-squared distribution is also often encountered in magnetic resonance imaging.[18]



Table of χ2 values vs p-values


The p-value is the probability of observing a test statistic at least as extreme in a chi-squared distribution. Accordingly, since the cumulative distribution function (CDF) for the appropriate degrees of freedom (df) gives the probability of having obtained a value less extreme than this point, subtracting the CDF value from 1 gives the p-value. A low p-value, below the chosen significance level, indicates statistical significance, i.e., sufficient evidence to reject the null hypothesis. A significance level of 0.05 is often used as the cutoff between significant and not-significant results.


The table below gives a number of p-values matching to χ2 for the first 10 degrees of freedom.








































































































































Degrees of freedom (df)

χ2 value[19]
1
0.004
0.02
0.06
0.15
0.46
1.07
1.64
2.71
3.84
6.63
10.83
2
0.10
0.21
0.45
0.71
1.39
2.41
3.22
4.61
5.99
9.21
13.82
3
0.35
0.58
1.01
1.42
2.37
3.66
4.64
6.25
7.81
11.34
16.27
4
0.71
1.06
1.65
2.20
3.36
4.88
5.99
7.78
9.49
13.28
18.47
5
1.14
1.61
2.34
3.00
4.35
6.06
7.29
9.24
11.07
15.09
20.52
6
1.63
2.20
3.07
3.83
5.35
7.23
8.56
10.64
12.59
16.81
22.46
7
2.17
2.83
3.82
4.67
6.35
8.38
9.80
12.02
14.07
18.48
24.32
8
2.73
3.49
4.59
5.53
7.34
9.52
11.03
13.36
15.51
20.09
26.12
9
3.32
4.17
5.38
6.39
8.34
10.66
12.24
14.68
16.92
21.67
27.88
10
3.94
4.87
6.18
7.27
9.34
11.78
13.44
15.99
18.31
23.21
29.59
P value (Probability)
0.95
0.90
0.80
0.70
0.50
0.30
0.20
0.10
0.05
0.01
0.001

These values can be calculated evaluating the quantile function (also known as “inverse CDF” or “ICDF”) of the chi-squared distribution;[20] e. g., the χ2 ICDF for p = 0.05 and df = 7 yields 14.06714 ≈ 14.07 as in the table above.



History and name


This distribution was first described by the German statistician Friedrich Robert Helmert in papers of 1875–6,[21][22] where he computed the sampling distribution of the sample variance of a normal population. Thus in German this was traditionally known as the Helmert'sche ("Helmertian") or "Helmert distribution".


The distribution was independently rediscovered by the English mathematician Karl Pearson in the context of goodness of fit, for which he developed his Pearson's chi-squared test, published in 1900, with computed table of values published in (Elderton 1902), collected in (Pearson 1914, pp. xxxi–xxxiii, 26–28, Table XII).
The name "chi-squared" ultimately derives from Pearson's shorthand for the exponent in a multivariate normal distribution with the Greek letter Chi, writing
−½χ2 for what would appear in modern notation as −½xTΣ−1x (Σ being the covariance matrix).[23] The idea of a family of "chi-squared distributions", however, is not due to Pearson but arose as a further development due to Fisher in the 1920s.[21]



See also




  • Chi distribution

  • Cochran's theorem


  • F-distribution


  • Fisher's method for combining independent tests of significance

  • Gamma distribution

  • Generalized chi-squared distribution

  • Hotelling's T-squared distribution

  • Noncentral chi-squared distribution

  • Pearson's chi-squared test

  • Reduced chi-squared statistic

  • Student's t-distribution

  • Wilks's lambda distribution

  • Wishart distribution



References




  1. ^ M.A. Sanders. "Characteristic function of the central chi-squared distribution" (PDF). Archived from the original (PDF) on 2011-07-15. Retrieved 2009-03-06..mw-parser-output cite.citationfont-style:inherit.mw-parser-output .citation qquotes:"""""""'""'".mw-parser-output .citation .cs1-lock-free abackground:url("//upload.wikimedia.org/wikipedia/commons/thumb/6/65/Lock-green.svg/9px-Lock-green.svg.png")no-repeat;background-position:right .1em center.mw-parser-output .citation .cs1-lock-limited a,.mw-parser-output .citation .cs1-lock-registration abackground:url("//upload.wikimedia.org/wikipedia/commons/thumb/d/d6/Lock-gray-alt-2.svg/9px-Lock-gray-alt-2.svg.png")no-repeat;background-position:right .1em center.mw-parser-output .citation .cs1-lock-subscription abackground:url("//upload.wikimedia.org/wikipedia/commons/thumb/a/aa/Lock-red-alt-2.svg/9px-Lock-red-alt-2.svg.png")no-repeat;background-position:right .1em center.mw-parser-output .cs1-subscription,.mw-parser-output .cs1-registrationcolor:#555.mw-parser-output .cs1-subscription span,.mw-parser-output .cs1-registration spanborder-bottom:1px dotted;cursor:help.mw-parser-output .cs1-ws-icon abackground:url("//upload.wikimedia.org/wikipedia/commons/thumb/4/4c/Wikisource-logo.svg/12px-Wikisource-logo.svg.png")no-repeat;background-position:right .1em center.mw-parser-output code.cs1-codecolor:inherit;background:inherit;border:inherit;padding:inherit.mw-parser-output .cs1-hidden-errordisplay:none;font-size:100%.mw-parser-output .cs1-visible-errorfont-size:100%.mw-parser-output .cs1-maintdisplay:none;color:#33aa33;margin-left:0.3em.mw-parser-output .cs1-subscription,.mw-parser-output .cs1-registration,.mw-parser-output .cs1-formatfont-size:95%.mw-parser-output .cs1-kern-left,.mw-parser-output .cs1-kern-wl-leftpadding-left:0.2em.mw-parser-output .cs1-kern-right,.mw-parser-output .cs1-kern-wl-rightpadding-right:0.2em


  2. ^ Abramowitz, Milton; Stegun, Irene Ann, eds. (1983) [June 1964]. "Chapter 26". Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Applied Mathematics Series. 55 (Ninth reprint with additional corrections of tenth original printing with corrections (December 1972); first ed.). Washington D.C.; New York: United States Department of Commerce, National Bureau of Standards; Dover Publications. p. 940. ISBN 978-0-486-61272-0. LCCN 64-60036. MR 0167642. LCCN 65-12253.


  3. ^ NIST (2006). Engineering Statistics Handbook – Chi-Squared Distribution


  4. ^ abc Johnson, N. L.; Kotz, S.; Balakrishnan, N. (1994). "Chi-Squared Distributions including Chi and Rayleigh". Continuous Univariate Distributions. 1 (Second ed.). John Wiley and Sons. pp. 415–493. ISBN 978-0-471-58495-7.


  5. ^ Mood, Alexander; Graybill, Franklin A.; Boes, Duane C. (1974). Introduction to the Theory of Statistics (Third ed.). McGraw-Hill. pp. 241–246. ISBN 978-0-07-042864-5.


  6. ^ Westfall, Peter H. (2013). Understanding Advanced Statistical Methods. Boca Raton, FL: CRC Press. ISBN 978-1-4665-1210-8.


  7. ^ Ramsey, PH (1988). "Evaluating the Normal Approximation to the Binomial Test". Journal of Educational Statistics. 13 (2): 173–82. doi:10.2307/1164752. JSTOR 1164752.


  8. ^ ab Lancaster, H.O. (1969), The Chi-squared Distribution, Wiley


  9. ^ Dasgupta, Sanjoy D. A.; Gupta, Anupam K. (January 2003). "An Elementary Proof of a Theorem of Johnson and Lindenstrauss" (PDF). Random Structures and Algorithms. 22 (1): 60–65. doi:10.1002/rsa.10073. Retrieved 2012-05-01.


  10. ^ Chi-squared distribution, from MathWorld, retrieved Feb. 11, 2009


  11. ^ M. K. Simon, Probability Distributions Involving Gaussian Random Variables, New York: Springer, 2002, eq. (2.35),
    ISBN 978-0-387-34657-1



  12. ^ Box, Hunter and Hunter (1978). Statistics for experimenters. Wiley. p. 118. ISBN 978-0471093152.


  13. ^ Bartlett, M. S.; Kendall, D. G. (1946). "The Statistical Analysis of Variance-Heterogeneity and the Logarithmic Transformation". Supplement to the Journal of the Royal Statistical Society. 8 (1): 128–138. doi:10.2307/2983618. JSTOR 2983618.


  14. ^ ab Pillai, Natesh S. (2016). "An unexpected encounter with Cauchy and Lévy". Annals of Statistics. 44 (5): 2089–2097. arXiv:1505.01957. doi:10.1214/15-aos1407.


  15. ^ Wilson, E. B.; Hilferty, M. M. (1931). "The distribution of chi-squared". Proc. Natl. Acad. Sci. USA. 17 (12): 684–688. Bibcode:1931PNAS...17..684W. doi:10.1073/pnas.17.12.684. PMC 1076144. PMID 16577411.


  16. ^ Bäckström, T.; Fischer, J. (January 2018). "Fast Randomization for Distributed Low-Bitrate Coding of Speech and Audio". IEEE/ACM Transactions on Audio, Speech, and Language Processing. 26 (1): 19–30. doi:10.1109/TASLP.2017.2757601.


  17. ^ Bausch, J. (2013). "On the Efficient Calculation of a Linear Combination of Chi-Square Random Variables with an Application in Counting String Vacua". J. Phys. A: Math. Theor. 46 (50): 505202. arXiv:1208.2691. Bibcode:2013JPhA...46X5202B. doi:10.1088/1751-8113/46/50/505202.


  18. ^ den Dekker A. J., Sijbers J., (2014) "Data distributions in magnetic resonance images: a review", Physica Medica, [1]


  19. ^ Chi-Squared Test Table B.2. Dr. Jacqueline S. McLaughlin at The Pennsylvania State University. In turn citing: R. A. Fisher and F. Yates, Statistical Tables for Biological Agricultural and Medical Research, 6th ed., Table IV. Two values have been corrected, 7.82 with 7.81 and 4.60 with 4.61


  20. ^ R Tutorial: Chi-squared Distribution


  21. ^ ab Hald 1998, pp. 633–692, 27. Sampling Distributions under Normality.


  22. ^ F. R. Helmert, "Ueber die Wahrscheinlichkeit der Potenzsummen der Beobachtungsfehler und über einige damit im Zusammenhange stehende Fragen", Zeitschrift für Mathematik und Physik 21, 1876, pp. 102–219


  23. ^
    R. L. Plackett, Karl Pearson and the Chi-Squared Test, International Statistical Review, 1983, 61f.
    See also Jeff Miller, Earliest Known Uses of Some of the Words of Mathematics.




Further reading


.mw-parser-output .refbeginfont-size:90%;margin-bottom:0.5em.mw-parser-output .refbegin-hanging-indents>ullist-style-type:none;margin-left:0.mw-parser-output .refbegin-hanging-indents>ul>li,.mw-parser-output .refbegin-hanging-indents>dl>ddmargin-left:0;padding-left:3.2em;text-indent:-3.2em;list-style:none.mw-parser-output .refbegin-100font-size:100%


  • Hald, Anders (1998). A history of mathematical statistics from 1750 to 1930. New York: Wiley. ISBN 978-0-471-17912-2.


  • Elderton, William Palin (1902). "Tables for Testing the Goodness of Fit of Theory to Observation". Biometrika. 1 (2): 155–163. doi:10.1093/biomet/1.2.155.


  • Hazewinkel, Michiel, ed. (2001) [1994], "Chi-squared distribution", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4



External links


  • Earliest Uses of Some of the Words of Mathematics: entry on Chi squared has a brief history


  • Course notes on Chi-Squared Goodness of Fit Testing from Yale University Stats 101 class.


  • Mathematica demonstration showing the chi-squared sampling distribution of various statistics, e. g. Σx², for a normal population

  • Simple algorithm for approximating cdf and inverse cdf for the chi-squared distribution with a pocket calculator







Popular posts from this blog

𛂒𛀶,𛀽𛀑𛂀𛃧𛂓𛀙𛃆𛃑𛃷𛂟𛁡𛀢𛀟𛁤𛂽𛁕𛁪𛂟𛂯,𛁞𛂧𛀴𛁄𛁠𛁼𛂿𛀤 𛂘,𛁺𛂾𛃭𛃭𛃵𛀺,𛂣𛃍𛂖𛃶 𛀸𛃀𛂖𛁶𛁏𛁚 𛂢𛂞 𛁰𛂆𛀔,𛁸𛀽𛁓𛃋𛂇𛃧𛀧𛃣𛂐𛃇,𛂂𛃻𛃲𛁬𛃞𛀧𛃃𛀅 𛂭𛁠𛁡𛃇𛀷𛃓𛁥,𛁙𛁘𛁞𛃸𛁸𛃣𛁜,𛂛,𛃿,𛁯𛂘𛂌𛃛𛁱𛃌𛂈𛂇 𛁊𛃲,𛀕𛃴𛀜 𛀶𛂆𛀶𛃟𛂉𛀣,𛂐𛁞𛁾 𛁷𛂑𛁳𛂯𛀬𛃅,𛃶𛁼

Edmonton

Crossroads (UK TV series)