A univariate density of the exponential family has the form
$$p(x) = \frac{1}{Z(\underline{\theta})} \exp\Big( \sum_{k} \theta_k \, T_k(x) \Big), \qquad Z(\underline{\theta}) = \int_{a}^{b} \exp\Big( \sum_{k} \theta_k \, T_k(x) \Big) \, \mathrm{d}x ,$$
with the sufficient statistics (features) $T_k(x)$, the natural parameters (weights) $\theta_k$ and the normalization constant (partition function) $Z$. Its logarithm $\log Z$ is the log-partition function. The log-density is a weighted sum of the features, shifted by $-\log Z$.
$Z$ must be finite. On $\mathbb{R}$ this requires the exponent to decrease fast enough towards $\pm\infty$. For example, a polynomial exponent must have an even highest power with a negative weight. This demo always normalizes on the bounded domain $[a, b]$, so every choice gives a valid (truncated) density. A warning is shown if the density does not vanish at the boundary.
The density with the largest entropy among all densities with given expectations $\mathrm{E}\{T_k(x)\}$ is the exponential family density with features $T_k$. Therefore, the Gaussian is the maximum entropy density for a given mean and variance. The dashed line shows the Gaussian with the same mean and variance as $p(x)$; its entropy is always at least the entropy of $p(x)$.
The derivatives of the log-partition function give the moments of the sufficient statistics:
$$\frac{\partial \log Z}{\partial \theta_k} = \mathrm{E}\{T_k(x)\}, \qquad \frac{\partial^2 \log Z}{\partial \theta_k \, \partial \theta_l} = \mathrm{Cov}\{T_k(x), T_l(x)\} .$$
$p(x)$, $\log Z$, the moments and the entropy are computed with the midpoint rule on 2000 points of the domain.