Joint, Marginal & Conditional Gaussian

Learn More

Joint, marginal and conditional density

For two random variables $X$ and $Y$:

$$ \text{joint:} \quad f(x, y) \ge 0 \enspace, \quad \int_{\mathbb{R}} \int_{\mathbb{R}} f(x, y) \,\mathrm{d}x \,\mathrm{d}y = 1 \enspace, \quad P\big((X, Y) \in A\big) = \iint_A f(x, y) \,\mathrm{d}x \,\mathrm{d}y \enspace, $$

$$ \text{marginal:} \quad f(x) = \int_{\mathbb{R}} f(x, y) \,\mathrm{d}y \enspace, $$

$$ \text{conditional:} \quad f(x \,|\, \hat y) = \frac{f(x, \hat y)}{f(\hat y)} = \frac{f(x, \hat y)}{\int_{\mathbb{R}} f(x, \hat y) \,\mathrm{d}x} \enspace. $$

The slice $f(x, \hat y)$ of the joint at $y = \hat y$ has the shape of the conditional; dividing by its area $f(\hat y)$ normalizes it.

2D Gaussian

Interactive visualizaton of the 2D Gaussian and its marginal and conditional density.

$$ f(\underline x) = \mathcal{N}(\underline x; \underline \mu, \textbf{C}) = \frac{1}{2\pi \sqrt{\det(\textbf{C})}} \cdot \exp\!\left\{ -\frac{1}{2} \cdot (\underline x - \underline \mu)^\top \textbf{C}^{-1} (\underline x - \underline \mu) \right\} \enspace, \quad \underline{x}\in \mathbb{R}^2 \enspace, \quad \textbf{C} \enspace \text{positive semidefinite} \enspace. $$

Formulas and Literature

The Gaussian parameters are restricted to $$ \underline \mu = \begin{bmatrix}0 \\ 0\end{bmatrix}\,, \quad \textbf{C} = \begin{bmatrix}1 & \rho \\ \rho & 1\end{bmatrix} \enspace. $$

Then marginal and conditional are again Gaussian [MatrixCookbook]:

$$ f(x) = \mathcal{N}(x; 0, 1) \enspace, \quad f(x \,|\, \hat y) = \mathcal{N}(x; \rho \hat y, 1 - \rho^2) \enspace. $$

For $\rho = \pm 1$, $\textbf{C}$ is singular: all mass lies on the line $y = \rho x$, where the joint is the 1D Gaussian $\mathcal{N}(x; 0, 1)$, and the conditional becomes the Dirac delta $\delta(x - \rho \hat y)$.

Note that the 1D and 2D densities are scaled with respect to each other such that 2D joint and 1D marginal have the same height and therefore the same shape along the $x$ axis.