【李沐深度学习笔记】矩阵计算（4）

课程地址和说明

线性代数实现p4
本系列文章是我学习李沐老师深度学习系列课程的学习笔记，可能会对李沐老师上课没讲到的进行补充。
本节是第四篇，由于CSDN限制，只能被迫拆分

矩阵计算

矩阵的导数运算

向量对向量求导的基本运算规则

已知向量函数$\overrightarrow{y}=\overrightarrow{f}(\overrightarrow{x})$与向量$\overrightarrow{x}={\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_m\end{array}\right]}_{m\times 1}$

• 当$\overrightarrow{y}=\overrightarrow{a}$，且$\overrightarrow{a}$不是$\overrightarrow{x}$的函数（即$\overrightarrow{a}$中没有分量和$\overrightarrow{x}$相关）时，则有：
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=\left[\begin{array}{c}\frac{\partial f(\overrightarrow{x})}{\partial x_1}\\ \frac{\partial f(\overrightarrow{x})}{\partial x_2}\\ ⋮\\ \frac{\partial f(\overrightarrow{x})}{\partial x_m}\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right]=\overrightarrow{0}$$
• 当$\overrightarrow{y}=\overrightarrow{x}$时，即$\overrightarrow{y}=\left[\begin{array}{c}{f}_1(\overrightarrow{x})\\ f_2(\overrightarrow{x})\\ ⋮\\ f_m(\overrightarrow{x})\end{array}\right]=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_m\end{array}\right]$，则有：
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=\left[\begin{array}{c}\frac{\partial f(\overrightarrow{x})}{\partial x_1}\\ \frac{\partial f(\overrightarrow{x})}{\partial x_2}\\ ⋮\\ \frac{\partial f(\overrightarrow{x})}{\partial x_m}\end{array}\right]={\left[\begin{array}{cccc}\frac{\partial f_1(\overrightarrow{x})}{\partial x_1}& \frac{\partial f_2(\overrightarrow{x})}{\partial x_1}& \dots & \frac{\partial f_n(\overrightarrow{x})}{\partial x_1}\\ \frac{\partial f_1(\overrightarrow{x})}{\partial x_2}& \frac{\partial f_2(\overrightarrow{x})}{\partial x_2}& \dots & \frac{\partial f_n(\overrightarrow{x})}{\partial x_2}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{\partial f_1(\overrightarrow{x})}{\partial x_m}& \frac{\partial f_2(\overrightarrow{x})}{\partial x_m}& \dots & \frac{\partial f_n(\overrightarrow{x})}{\partial x_m}\end{array}\right]}_{m\times n}=\left[\begin{array}{cccc}1& 0& \dots & 0\\ 0& 1& \dots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & 1\end{array}\right]=\mathbf{I}或\mathbf{E}（单位矩阵的两种不同记号，含义一致）$$
• 当$\overrightarrow{y}=\mathbf{A}\overrightarrow{x}$，$\mathbf{A}=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1m}\\ a_{21}& a_{22}& \cdots & a_{2m}\\ ⋮& ⋮& \ddots & ⋮\\ a_{m1}& a_{m2}& \cdots & a_{mm}\end{array}\right]$，则有：
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=\frac{\partial \mathbf{A}\overrightarrow{x}}{\partial \overrightarrow{x}}={\mathbf{A}}^T（按分母布局）$$
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=\frac{\partial \mathbf{A}\overrightarrow{x}}{\partial \overrightarrow{x}}=\mathbf{A}（按分子布局）$$
  （证明见本节第三篇）
• 当$\overrightarrow{y}={\overrightarrow{x}}^T\mathbf{A}$，$\mathbf{A}=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1m}\\ a_{21}& a_{22}& \cdots & a_{2m}\\ ⋮& ⋮& \ddots & ⋮\\ a_{m1}& a_{m2}& \cdots & a_{mm}\end{array}\right]$，
  $\overrightarrow{y}={\overrightarrow{x}}^T\mathbf{A}=\left[\begin{array}{cccc}{x}_1,& x_2,& \dots ,& x_m\end{array}\right]\cdot \left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1m}\\ a_{21}& a_{22}& \cdots & a_{2m}\\ ⋮& ⋮& \ddots & ⋮\\ a_{m1}& a_{m2}& \cdots & a_{mm}\end{array}\right]=\left[\begin{array}{cccc}{a}_{11}{x}_1+a_{21}{x}_2+\cdots +a_{m1}{x}_m,& a_{12}{x}_1+a_{22}{x}_2+\cdots +a_{m2}{x}_m,& \dots ,& a_{1m}{x}_1+a_{2m}{x}_2+\cdots +a_{mm}{x}_m\end{array}\right]$，所以按一一对应法则只能理解成（这里行向量列向量混用了，没办法）$\overrightarrow{y}=\left[\begin{array}{c}{f}_1(\overrightarrow{x})\\ f_2(\overrightarrow{x})\\ ⋮\\ f_m(\overrightarrow{x})\end{array}\right]=\left[\begin{array}{c}{a}_{11}{x}_1+a_{21}{x}_2+\cdots +a_{m1}{x}_m\\ a_{12}{x}_1+a_{22}{x}_2+\cdots +a_{m2}{x}_m\\ ⋮\\ a_{1m}{x}_1+a_{2m}{x}_2+\cdots +a_{mm}{x}_m\end{array}\right]$，则有：
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=\frac{\partial {\overrightarrow{x}}^T\mathbf{A}}{\partial \overrightarrow{x}}=\left[\begin{array}{cccc}\frac{\partial f_1(\overrightarrow{x})}{\partial x_1}& \frac{\partial f_2(\overrightarrow{x})}{\partial x_1}& \dots & \frac{\partial f_n(\overrightarrow{x})}{\partial x_1}\\ \frac{\partial f_1(\overrightarrow{x})}{\partial x_2}& \frac{\partial f_2(\overrightarrow{x})}{\partial x_2}& \dots & \frac{\partial f_n(\overrightarrow{x})}{\partial x_2}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{\partial f_1(\overrightarrow{x})}{\partial x_m}& \frac{\partial f_2(\overrightarrow{x})}{\partial x_m}& \dots & \frac{\partial f_n(\overrightarrow{x})}{\partial x_m}\end{array}\right]=\left[\begin{array}{cccc}{a}_{11}& a_{21}& \dots & a_{m1}\\ a_{12}& a_{22}& \dots & a_{m2}\\ ⋮& ⋮& \ddots & ⋮\\ a_{1m}& a_{2m}& \dots & a_{mm}\end{array}\right]={\mathbf{A}}^T$$
• 当$\overrightarrow{y}=a\overrightarrow{u}$，$a$是任意常数，$\overrightarrow{u}=\overrightarrow{u}(\overrightarrow{x})$，则有：
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=a\frac{\partial \overrightarrow{u}}{\partial \overrightarrow{x}}$$
• 当$\overrightarrow{y}=\mathbf{A}\overrightarrow{u}$，$\overrightarrow{u}=\overrightarrow{u}(\overrightarrow{x})$，$\mathbf{A}$中的元素与$\overrightarrow{x}$中的元素无关系，则有：
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=\mathbf{A}\frac{\partial \overrightarrow{u}}{\partial \overrightarrow{x}}$$
• 当$\overrightarrow{y}=\overrightarrow{u}+\overrightarrow{v}$时，$\overrightarrow{u}=\overrightarrow{u}(\overrightarrow{x}),\overrightarrow{v}=\overrightarrow{v}(\overrightarrow{x})$，则有：
  $$\frac{\partial \overrightarrow{y}}{\partial \overrightarrow{x}}=\frac{\partial \overrightarrow{u}}{\partial \overrightarrow{x}}+\frac{\partial \overrightarrow{v}}{\partial \overrightarrow{x}}$$

拓展到矩阵

就是升维度，升到了四维空间，矩阵可以相当于四维空间里的向量，反正挺难懂的，我看个乐hhhhhhhh