Natural exponential family

From Wikipedia, the free encyclopedia

In probability and statistics, a natural exponential family (NEF) is a class of probability distributions that is a special case of an exponential family (EF). Many common distributions are members of the natural exponential family, and the use of such distributions simplifies the theory and computation of generalized linear models.

1 Definition
- 1.1 Univariate case
- 1.2 Multivariate case
2 Examples
3 Properties
4 Quadratic Variance Functions
5 References

[edit] Definition

[edit] Univariate case

A univariate natural exponential family is a set of probability distributions whose probability density function (or probability mass function, for the case of a discrete distribution) can be expressed in the form

$f_X(x; \theta) = h(x) \exp(\theta x - A(\theta)) \,\!$

where $h (x)$ and $A (θ)$ are known functions, and θ is the parameter.

Note the similarity to an exponential family:

$f_X(x; \theta) = h(x) \exp(\eta(\theta) T(x) - A(\theta)) \,\!$

Thus a natural exponential family is an exponential family in which the natural parameter η and the natural statistic T are both the identity.

[edit] Multivariate case

Suppose that $\mathbf{x} \in \mathcal{X} \subseteq \mathbb{R}^p$ , then a natural exponential family of order p has density or mass function of the form:

$f_X(\mathbf{x};\mathbf{\theta}) = h(\mathbf{x}) \exp(\theta^\top \mathbf{x} - A(\theta)) \,\!$

where in this case the parameter $\theta \in \mathbb{R}^p$

[edit] Examples

Examples of univariate cases include:

Poisson distribution
exponential distribution
binomial distribution with known number of trials
normal distribution with known variance
gamma distribution with known shape parameter
negative binomial distribution with known $r$

Note that most of these distributions will require a different parameterization than is listed in the above linked pages. For example in the Poisson case, the two parameterizations are related by $θ = logλ$ , where λ is the mean parameter, and so that the density may be written as

$f(k;\theta) = \frac{1}{k!} \exp \{ \theta k - \exp \theta \} . \$

for $\theta \in \mathbb{R}$ , so

$h(k) = \frac{1}{k!} \$ , and $A(\theta) = \exp \theta . \$

An example of the multivariate case is the multinomial distribution (again with known number of trials).

[edit] Properties

[edit] Moment and cumulant generating function

A member of a natural exponential family will have moment generating function of the form:

$M_Y(t) = \exp \{ A(\theta + t) - A(\theta) \} \,$

and so cumulant generating function:

$K_Y(t) = A(\theta + t) - A(\theta) \,$

[edit] Mean and variance

From the cumulant generating function, it follows that in the univariate case, the distribution has mean and variance:

$\mathrm{E}[Y] = K'_Y(0) = A'(\theta)\,$ and $\mathrm{Var}[Y] = K''_Y(0) = A''(\theta)\,$

In the multivariate case, the mean vector and covariance matrix are thus:

$\mathrm{E}[Y] = \nabla A(\theta)\,$ and $\mathrm{Cov}[Y] = \nabla \nabla^\top A(\theta)\,$

where $\nabla$ is the gradient and $\nabla \nabla^\top$ is the Hessian.

[edit] Additivity

The convolution densities from a natural exponential family are a natural exponential family. If $X_1,\ldots,X_n$ are independent identically distributed from a natural exponential family, then:

$\sum_{i=1}^n X_i\,$

is from a natural exponential family. This follows from the properties of the cumulant generating function.

[edit] Other

The variance function for random variables with an NEF distribution can be written in terms of the mean.

V a r (X) = V (μ)

The first two moments of a distribution uniquely characterize it.

X ˜ N E F [μ, V (μ)]

[edit] Quadratic Variance Functions

Six NEFs have quadratic variance functions (QVF) in which the variance of the distribution can be written as a quadratic function of the mean. These are called NEF-QVF. The properties of these distributions was described by Carl Morris.

V a r (X) = V (μ) = ν 0 + ν 1 μ + ν 2 μ 2 .

The six QVF-NEFs are:

1. A subset of all Normal distributions $N (μ,σ 2)$ has constant variance $V (μ) = σ 2 .$

2. Poisson $P o i s (μ)$ with linear $V (μ) = μ.$

3. Gamma $G a m (r,λ)$ with $μ = r λ$ and $V (μ) = μ 2 / r .$

4. Binomial $B i n (n, p)$ with $μ = n p$ and $V (μ) = - μ 2 / n + μ.$

5. Negative binomial $N e g B i n (n, p)$ with $μ = n p / (1 - p)$ and $V (μ) = μ 2 / n + μ.$

6. The (not very famous) distribution generated by the generalized hyperbolic secant distribution (NEF-GHS) has $V (μ) = μ 2 / n + n$ and $μ > 0.$

One property of the NEF-QVFs is that they are closed under linear transformations.

[edit] References

Morris C. "Natural exponential families", Encyclopedia of Statistical Sciences.
Morris C. (1982) "Natural exponential families with quadratic variance functions". Ann. Statist., 10(1), 65--80.
Morris C. (1982) Natural exponential families with quadratic variance functions: statistical theory. Dept of mathematics, Institute of Statistics, University of Texas, Austin.

v • d • e

Probability distributions

Discrete univariate with finite support

Benford · Bernoulli · binomial · categorical · hypergeometric · Rademacher · discrete uniform · Zipf · Zipf-Mandelbrot

Discrete univariate with infinite support

Boltzmann · Conway-Maxwell-Poisson · compound Poisson · discrete phase-type · extended negative binomial · Gauss-Kuzmin · geometric · logarithmic · negative binomial · parabolic fractal · Poisson · Skellam · Yule-Simon · zeta

Continuous univariate supported on a bounded interval

Beta · Kumaraswamy · raised cosine · triangular · U-quadratic · uniform · Wigner semicircle

Continuous univariate supported on a semi-infinite interval

Beta prime · Burr · chi-square · Coxian · Erlang · exponential · F · Fermi-Dirac · folded normal · Fréchet · Gamma · generalized extreme value · generalized inverse Gaussian · half-logistic · half-normal · Hotelling's T-square · hyper-exponential · hypoexponential · inverse chi-square (scale inverse chi-square) · inverse Gaussian · inverse gamma · Lévy · log-normal · log-logistic · Maxwell-Boltzmann · Maxwell speed · Nakagami · noncentral chi-square · Pareto · phase-type · Rayleigh · relativistic Breit–Wigner · Rice · Rosin–Rammler · shifted Gompertz · truncated normal · type-2 Gumbel · Weibull · Wilks' lambda

Continuous univariate supported on the whole real line

Cauchy · extreme value · exponential power · Fisher's z · Fisher-Tippett · generalized hyperbolic · hyperbolic secant · Landau · Laplace · Lévy skew alpha-stable · logistic · normal (Gaussian) · normal inverse Gaussian · skew normal · Student's t · type-1 Gumbel · Variance-Gamma · Voigt

Multivariate (joint)

Discrete: Ewens · multinomial · multivariate Polya Continuous: Dirichlet · Generalized Dirichlet · multivariate normal · multivariate Student · normal-scaled inverse gamma · normal-gamma Matrix-valued: inverse-Wishart · matrix normal · Wishart

Directional, degenerate, and singular

Directional: Kent · von Mises · von Mises–Fisher Degenerate: discrete degenerate · Dirac delta function Singular: Cantor

Families

exponential · natural exponential · location-scale · maximum entropy · Pearson · Tweedie

Categories: Exponentials | Continuous distributions | Discrete distributions

See also ebooksgratis.com: no banners, no cookies, totally FREE.