groupnorm_backward反向公式推导

前向

• 均值
  $${\mu }_{ng}=\frac{{\sum }_{i=1}^M(X^i)}{M}\text{\hspace{1em}(1)}$$

• 方差
  $${\sigma }_{ng}^2=\frac{{\sum }_{i=1}^M(X^i-{\mu }_{ng})}{M}\text{\hspace{1em}(2)}$$

• 归一化：

  令
  $$rsig=\frac{1}{\sqrt{{\sigma }_{ng}^2+\epsilon }}\text{\hspace{1em}(3)}$$
  则：
  $$Y=\gamma \ast (X-\mu )\ast rsig+\beta =\gamma \ast X\ast rsig+\beta -\gamma \ast \mu \ast rsig\text{\hspace{1em}(4)}$$

反向

令
$$S=\gamma \ast rsig\text{\hspace{1em}(5)}$$

$$B=\beta -\gamma \ast \mu \ast rsig\text{\hspace{1em}(6)}$$

则
$$Y=S\ast X+B$$
令
$$M=K\times H\times W（K=C/Group）\text{\hspace{1em}(7)}$$
由链式法则：
$$\frac{dL}{dX}=\frac{dL}{dY}\ast \frac{dY}{dX}=\frac{dL}{dY}\ast (\frac{d(S\ast X)}{dX}+\frac{dB}{dX})\text{\hspace{1em}(8)}$$
其中：

$$\frac{d(S\ast X)}{dX}=S+X\ast \frac{dS}{dX}=S+X\ast \gamma \ast \frac{drsig}{dX}\text{\hspace{1em}(9)}$$

$$\frac{dB}{dX}=-\gamma \ast \mu \ast \frac{drsig}{dX}-\gamma \ast rsig\ast \frac{d\mu }{dX}$$

$$\frac{drsig}{dX}=-rsi{g}^3\ast \frac{(X-\mu )}{M}\text{\hspace{1em}(10)}$$

$$\frac{d\mu }{dX}=\frac{1}{M}\text{\hspace{1em}(11)}$$

由(5),(8)(9)(10)(11)得：
$$\begin{gathered} \frac{dL}{dX}=dy\ast (S+X\ast \gamma \ast rsi{g}^3\ast \frac{(\mu -X)}{M}+\gamma \ast \mu \ast rsi{g}^3\ast \frac{(X-\mu )}{M}-\frac{\gamma \ast rsig}{M}) \\ =dy\ast S+dy\ast \gamma \ast rsi{g}^3\ast \frac{(u-X)}{M}\ast (X-\mu )-dy\ast \frac{\gamma \ast rsig}{M}\text{\hspace{1em}(12)} \end{gathered}$$
令
$$\begin{gathered} C_1=S=\gamma \ast rsig \\ {C}_2=dy\ast \gamma \ast rsi{g}^3\ast \frac{\mu -X}{M} \\ {C}_3=-C_2\ast \mu -\frac{dy\ast \gamma \ast rsig}{M}\text{\hspace{1em}(13)} \end{gathered}$$
得：
$$dx=C_1\ast dy+C_2\ast X+C_3\text{\hspace{1em}(14)}$$