密度估计公式

1. 极大似然估计：

$$y=p(x_1,x_2,x_3,...,x_n)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_1-\mu {)}^2}{2{\sigma }^2}}\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_2-\mu {)}^2}{2{\sigma }^2}}...\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_n-\mu {)}^2}{2{\sigma }^2}}$$

$$\begin{gathered} lny=lnp(x_1,x_2,x_3,...,x_n)=ln(\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_1-\mu {)}^2}{2{\sigma }^2}}\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_2-\mu {)}^2}{2{\sigma }^2}}...\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_n-\mu {)}^2}{2{\sigma }^2}})= \\ -nln(\sqrt{2\pi }\sigma )-\sum _{i=1}^n\frac{(x_i-\mu {)}^2}{2{\sigma }^2} \end{gathered}$$

要求y的极限值（将$\mu 和\sigma$视为变量，x视为常量），只需要对上述等式两边对x求导并令导数为0：

$$\frac{\partial \ln y}{\partial \mu }=\sum _{i=1}^n\frac{(x_i-\mu )}{{\sigma }^2}=0$$
即：$\mu =\frac{1}{n}{\sum }_{i=1}^n(x_i)$

$$\frac{\partial \ln y}{\partial \sigma }=-n\frac{1}{\sigma }+\sum _{i=1}^n\frac{(x_i-\mu {)}^2}{{\sigma }^3}=0$$
即：${\sigma }^2={\sum }_{i=1}^n\frac{(x_i-\mu {)}^2}{n}$

2. 后验估计（贝叶斯估计）：

$$y=p(x_1,x_2,x_3,...,x_n;{\theta }_0)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_1-\mu {)}^2}{2{\sigma }^2}}\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_2-\mu {)}^2}{2{\sigma }^2}}...\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_n-\mu {)}^2}{2{\sigma }^2}}\frac{1}{\sqrt{2\pi }{\sigma }_0}{e}^{-\frac{({\mu }_0-\mu {)}^2}{2{\sigma }_0^2}}$$

$$\begin{gathered} lny=lnp(x_1,x_2,x_3,...,x_n;{\theta }_0)=ln(\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_1-\mu {)}^2}{2{\sigma }^2}}\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_2-\mu {)}^2}{2{\sigma }^2}}...\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x_n-\mu {)}^2}{2{\sigma }^2}}\frac{1}{\sqrt{2\pi }{\sigma }_0}{e}^{-\frac{({\mu }_0-\mu {)}^2}{2{\sigma }_0^2}})= \\ -nln(\sqrt{2\pi }\sigma )-\sum _{i=1}^n\frac{(x_i-\mu {)}^2}{2{\sigma }^2}-ln(\sqrt{2\pi }{\sigma }_0)-\frac{({\mu }_0-\mu {)}^2}{2{\sigma }_0^2} \end{gathered}$$

要求y的极限值（将$\mu 和\sigma$视为变量，x视为常量），只需要对上述等式两边对x求导并令导数为0：

$$\frac{\partial \ln y}{\partial \mu }=\sum _{i=1}^n\frac{(x_i-\mu )}{{\sigma }^2}+\frac{({\mu }_0-\mu )}{{\sigma }_0^2}=0$$

即：
$$\frac{\partial \ln y}{\partial \mu }=\frac{1}{{\sigma }^2}\sum _{i=1}^n{x}_i-\frac{n\mu }{{\sigma }^2}+\frac{{\mu }_0}{{\sigma }_0^2}-\frac{\mu }{{\sigma }_0^2}=0$$

$$\frac{1}{{\sigma }^2}\sum _{i=1}^n{x}_i+\frac{{\mu }_0}{{\sigma }_0^2}=(\frac{n}{{\sigma }^2}+\frac{1}{{\sigma }_0^2})\mu$$

$$\mu =\frac{\frac{1}{{\sigma }^2}{\sum }_{i=1}^n{x}_i+\frac{{\mu }_0}{{\sigma }_0^2}}{\frac{n}{{\sigma }^2}+\frac{1}{{\sigma }_0^2}}$$

这里要注意的是，贝叶斯估计在${\theta }_0$处的先验概率的计算方式，此时要将$u_0和{\sigma }_0当作先验参数$