隐藏层误差
在每层两个节点的情况下，即网络是
2
∗
2
∗
2
2 * 2*2
2∗2∗2
隐藏层第一个节点的误差计算，由输出层误差按权重分配
e
h
1
=
e
o
1
⋅
(
w
1
,
1
w
1
,
1
+
w
2
,
1
)
+
e
o
2
⋅
(
w
1
,
2
w
1
,
2
+
w
2
,
2
)
e_{h1}=e_{o1}\cdot (\frac {w_{1,1}}{w_{1,1}+w_{2,1}})+e_{o2}\cdot (\frac {w_{1,2}}{w_{1,2}+w_{2,2}})
eh1=eo1⋅(w1,1+w2,1w1,1)+eo2⋅(w1,2+w2,2w1,2)
相应的：
e
h
2
=
e
o
1
⋅
(
w
2
,
1
w
1
,
1
+
w
2
,
1
)
+
e
o
2
⋅
(
w
2
,
2
w
2
,
1
+
w
2
,
2
)
e_{h2}=e_{o1}\cdot (\frac {w_{2,1}}{w_{1,1}+w_{2,1}})+e_{o2}\cdot (\frac {w_{2,2}}{w_{2,1}+w_{2,2}})
eh2=eo1⋅(w1,1+w2,1w2,1)+eo2⋅(w2,1+w2,2w2,2)
因矩阵相乘的规则是
[
a
b
c
d
]
⋅
[
e
f
]
=
[
a
⋅
e
+
b
⋅
f
c
⋅
e
+
d
⋅
f
]
\left[\begin{array}{clr} a&b\\ c&d\\ \end{array}\right] \cdot \left[\begin{array}{clr} e\\ f\\ \end{array}\right]= \left[\begin{array}{clr} a\cdot e+b\cdot f\\ c\cdot e + d\cdot f\\ \end{array}\right]
[acbd]⋅[ef]=[a⋅e+b⋅fc⋅e+d⋅f]
所以反推得到关于第一个节点的隐藏层矩阵的点乘式
和第二个节点的隐藏层矩阵的点乘式
e
h
1
=
[
w
1
,
1
w
1
,
1
+
w
2
,
1
w
1
,
2
w
2
,
1
+
w
2
,
2
]
⋅
[
e
o
1
e
o
2
]
e_{h1}= \left[\begin{array}{clr} \frac {w_{1,1}}{w_{1,1}+w_{2,1}} & \frac {w_{1,2}}{w_{2,1}+w_{2,2}}\\ \end{array}\right] \cdot \left[\begin{array}{clr} e_{o1} \\ e_{o2} \\ \end{array}\right]
eh1=[w1,1+w2,1w1,1w2,1+w2,2w1,2]⋅[eo1eo2]
e h 2 = [ w 2 , 1 w 1 , 1 + w 2 , 1 w 2 , 2 w 2 , 1 + w 2 , 2 ] ⋅ [ e o 1 e o 2 ] e_{h2}= \left[\begin{array}{clr} \frac {w_{2,1}}{w_{1,1}+w_{2,1}} & \frac {w_{2,2}}{w_{2,1}+w_{2,2}}\\ \end{array}\right] \cdot \left[\begin{array}{clr} e_{o1} \\ e_{o2} \\ \end{array}\right] eh2=[w1,1+w2,1w2,1w2,1+w2,2w2,2]⋅[eo1eo2]
综上，存在误差隐藏层矩阵点乘式是
e h i d d e n = [ w 1 , 1 w 1 , 1 + w 2 , 1 w 1 , 2 w 2 , 1 + w 2 , 2 w 2 , 1 w 1 , 1 + w 2 , 1 w 2 , 2 w 2 , 1 + w 2 , 2 ] ⋅ [ e o 1 e o 2 ] e_{hidden}= \left[\begin{array}{clr} \frac {w_{1,1}}{w_{1,1}+w_{2,1}} & \frac {w_{1,2}}{w_{2,1}+w_{2,2}}\\ \\ \frac {w_{2,1}}{w_{1,1}+w_{2,1}} & \frac {w_{2,2}}{w_{2,1}+w_{2,2}}\\ \end{array}\right] \cdot \left[\begin{array}{clr} e_{o1} \\ e_{o2} \\ \end{array}\right] ehidden=⎣⎡w1,1+w2,1w1,1w1,1+w2,1w2,1w2,1+w2,2w1,2w2,1+w2,2w2,2⎦⎤⋅[eo1eo2]
这些分数难以处理，所以可以丢掉分母，弊端是失去后馈误差的大小
e
h
i
d
d
e
n
=
[
w
1
,
1
w
1
,
2
w
2
,
1
w
2
,
2
]
⋅
[
e
o
1
e
o
2
]
e_{hidden}= \left[\begin{array}{clr} {w_{1,1}} & {w_{1,2}}\\ \\ {w_{2,1}} & {w_{2,2}}\\ \end{array}\right] \cdot \left[\begin{array}{clr} e_{o1} \\ e_{o2} \\ \end{array}\right]
ehidden=⎣⎡w1,1w2,1w1,2w2,2⎦⎤⋅[eo1eo2]
这样，第一个矩阵就变成了原来隐藏层权重矩阵的转置矩阵
w
T
w^{T}
wT
于是，反向传播误差为：
e
h
i
d
d
e
n
=
w
T
⋅
e
o
u
t
p
u
t
e_{hidden}=w^{T} \cdot e_{output}
ehidden=wT⋅eoutput