Back

ⓘ Normal-inverse-Wishart distribution. In probability theory and statistics, the normal-inverse-Wishart distribution is a multivariate four-parameter family of co ..




                                     

ⓘ Normal-inverse-Wishart distribution

In probability theory and statistics, the normal-inverse-Wishart distribution is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and covariance matrix.

                                     

1. Definition

Suppose

μ | μ 0, λ, Σ ∼ N μ | μ 0, 1 λ Σ {\displaystyle {\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},\lambda,{\boldsymbol {\Sigma }}\sim {\mathcal {N}}\left{\boldsymbol {\mu }}{\Big |}{\boldsymbol {\mu }}_{0},{\frac {1}{\lambda }}{\boldsymbol {\Sigma }}\right}

has a multivariate normal distribution with mean μ 0 {\displaystyle {\boldsymbol {\mu }}_{0}} and covariance matrix 1 λ Σ {\displaystyle {\tfrac {1}{\lambda }}{\boldsymbol {\Sigma }}}, where

Σ | Ψ, ν ∼ W − 1 Σ | Ψ, ν {\displaystyle {\boldsymbol {\Sigma }}|{\boldsymbol {\Psi }},\nu \sim {\mathcal {W}}^{-1}{\boldsymbol {\Sigma }}|{\boldsymbol {\Psi }},\nu}

has an inverse Wishart distribution. Then μ, Σ {\displaystyle {\boldsymbol {\mu }},{\boldsymbol {\Sigma }}} has a normal-inverse-Wishart distribution, denoted as

μ, Σ ∼ N I W μ 0, λ, Ψ, ν. {\displaystyle {\boldsymbol {\mu }},{\boldsymbol {\Sigma }}\sim \mathrm {NIW} {\boldsymbol {\mu }}_{0},\lambda,{\boldsymbol {\Psi }},\nu.}
                                     

2. Characterization

Probability density function

f = N μ | μ 0, 1 λ Σ W − 1 Σ | Ψ, ν {\displaystyle f{\boldsymbol {\mu }},{\boldsymbol {\Sigma }}|{\boldsymbol {\mu }}_{0},\lambda,{\boldsymbol {\Psi }},\nu={\mathcal {N}}\left{\boldsymbol {\mu }}{\Big |}{\boldsymbol {\mu }}_{0},{\frac {1}{\lambda }}{\boldsymbol {\Sigma }}\right{\mathcal {W}}^{-1}{\boldsymbol {\Sigma }}|{\boldsymbol {\Psi }},\nu}
                                     

3. Properties

Marginal distributions

By construction, the marginal distribution over Σ {\displaystyle {\boldsymbol {\Sigma }}} is an inverse Wishart distribution, and the conditional distribution over μ {\displaystyle {\boldsymbol {\mu }}} given Σ {\displaystyle {\boldsymbol {\Sigma }}} is a multivariate normal distribution. The marginal distribution over μ {\displaystyle {\boldsymbol {\mu }}} is a multivariate t-distribution.

                                     

4. Posterior distribution of the parameters

Suppose the sampling density is a multivariate normal distribution

y i | μ, Σ ∼ N p μ, Σ {\displaystyle {\boldsymbol {y_{i}}}|{\boldsymbol {\mu }},{\boldsymbol {\Sigma }}\sim {\mathcal {N}}_{p}{\boldsymbol {\mu }},{\boldsymbol {\Sigma }}}

where y {\displaystyle {\boldsymbol {y}}} is an n × p {\displaystyle n\times p} matrix and y i {\displaystyle {\boldsymbol {y_{i}}}} of length p {\displaystyle p} is row i {\displaystyle i} of the matrix.

With the mean and covariance matrix of the sampling distribution is unknown, we can place a Normal-Inverse-Wishart prior on the mean and covariance parameters jointly

μ, Σ ∼ N I W μ 0, λ, Ψ, ν. {\displaystyle {\boldsymbol {\mu }},{\boldsymbol {\Sigma }}\sim \mathrm {NIW} {\boldsymbol {\mu }}_{0},\lambda,{\boldsymbol {\Psi }},\nu.}

The resulting posterior distribution for the mean and covariance matrix will also be a Normal-Inverse-Wishart

μ, Σ | y ∼ N I W, {\displaystyle {\boldsymbol {\mu }},{\boldsymbol {\Sigma }}|y\sim \mathrm {NIW} {\boldsymbol {\mu }}_{n},\lambda _{n},{\boldsymbol {\Psi }}_{n},\nu _{n},}

where

μ n = λ μ 0 + n y ¯ λ + n {\displaystyle {\boldsymbol {\mu }}_{n}={\frac {\lambda {\boldsymbol {\mu }}_{0}+n{\bar {\boldsymbol {y}}}}{\lambda +n}}} λ n = λ + n {\displaystyle \lambda _{n}=\lambda +n} ν n = ν + n {\displaystyle \nu _{n}=\nu +n} Ψ n = Ψ + S + λ n λ + n y ¯ − μ 0 y ¯ − μ 0 T w i t h, S = ∑ i = 1 n y i − y ¯ y i − y ¯ T {\displaystyle {\boldsymbol {\Psi }}_{n}={\boldsymbol {\Psi +S}}+{\frac {\lambda n}{\lambda +n}}({\boldsymbol.


                                     

5. Generating normal-inverse-Wishart random variates

Generation of random variates is straightforward:

  • Sample μ {\displaystyle {\boldsymbol {\mu }}} from a multivariate normal distribution with mean μ 0 {\displaystyle {\boldsymbol {\mu }}_{0}} and variance 1 λ Σ {\displaystyle {\boldsymbol {\tfrac {1}{\lambda }}}{\boldsymbol {\Sigma }}}
  • Sample Σ {\displaystyle {\boldsymbol {\Sigma }}} from an inverse Wishart distribution with parameters Ψ {\displaystyle {\boldsymbol {\Psi }}} and ν {\displaystyle \nu }
                                     

6. Related distributions

  • The normal-inverse-gamma distribution is the one-dimensional equivalent.
  • The multivariate normal distribution and inverse Wishart distribution are the component distributions out of which this distribution is made.
  • The normal-Wishart distribution is essentially the same distribution parameterized by precision rather than variance. If μ, Σ ∼ N I W μ 0, λ, Ψ, ν {\displaystyle {\boldsymbol {\mu }},{\boldsymbol {\Sigma }}\sim \mathrm {NIW} {\boldsymbol {\mu }}_{0},\lambda,{\boldsymbol {\Psi }},\nu} then μ, Σ − 1 ∼ N W {\displaystyle {\boldsymbol {\mu }},{\boldsymbol {\Sigma }}^{-1}\sim \mathrm {NW} {\boldsymbol {\mu }}_{0},\lambda,{\boldsymbol {\Psi }}^{-1},\nu}.
                                     
  • elliptical distribution is any member of a broad family of probability distributions that generalize the multivariate normal distribution Intuitively
  • transform sampling Inverse - variance weighting Inverse - Wishart distribution Iris flower data set Irwin Hall distribution Isomap Isotonic regression Isserlis theorem
  • probability density function of the Wishart and inverse Wishart distributions and the matrix variate beta distribution It has two equivalent definitions
  • series. These product distributions are somewhat comparable to the Wishart distribution The latter is the joint distribution of the four elements actually
  • normality assumptions about error distributions the counterpart of the sum of squares due to error has a Wishart distribution MANOVA is based on the product
  • normal exponential gamma chi - squared beta Dirichlet Bernoulli categorical Poisson Wishart inverse Wishart geometric A number of common distributions are
  • semicircle distribution 1: C Wishart distribution F: C Zeta distribution 1: D Zipf s law  1: D Donsker s theorem  LU: C Empirical distribution function
  • addition, if the random variable has normal distribution the sample covariance matrix has Wishart distribution and a slightly differently scaled version
  • 1365 - 2311.1927.tb00074.x. hdl: 2440 15100. with J Wishart Fisher, R. A. Wishart J. 1927 On the Distribution of the Error of an Interpolated Value, and
  • the Wishart distribution which is the conjugate prior of the precision matrix inverse covariance matrix for a multivariate Gaussian distribution Mult

Users also searched:

...
...
...