【白板推导系列笔记】线性分类-线性判别分析（Fisher）-模型定义

线性判别分析的思想是，找的一个方向$\omega$，将样本向这个方向做投影，投影后的数据尽可能的满足

1. 相同类内部的样本的投影尽可能接近
2. 不同类之间的距离尽可能较大

总结为类内小，类间大

X = ( x 1 x 2 ⋯ x N ) T = ( x 1 T x 2 T ⋮ x N T ) N × p , Y = ( y 1 y 2 ⋮ y N ) N × 1 { ( x i , y i ) } i = 1 N , x i ∈ R p , y i ∈ { + 1 , − 1 } x C 1 = { x i ∣ y i = + 1 } , x C 2 = { x i ∣ y i = − 1 } ∣ x C 1 ∣ = N 1 , ∣ x C 2 ∣ = N 2 , N 1 + N 2 = N

X=(x1​​x2​​⋯​xN​​)T=⎝ ⎛​x1T​x2T​⋮xNT​​⎠ ⎞​N×p​,Y=⎝ ⎛​y1​y2​⋮yN​​⎠ ⎞​N×1​{ (xi​,yi​)}i=1N​,xi​∈Rp,yi​∈{ +1,−1}xC1​​={ xi​∣yi​=+1},xC2​​={ xi​∣yi​=−1}∣xC1​​∣=N1​,∣xC2​​∣=N2​,N1​+N2​=N​
设
$$z_i={\omega }^T{x}_i$$
显然这是个实数，可以看做$x_i$在$\omega$上的投影
模型要求类内小，可以用方差矩阵来衡量类内样本的聚散程度
z ˉ = 1 N ∑ i = 1 N z i = 1 N ∑ i = 1 N ω T x i C 1 : z 1 ˉ = 1 N 1 ∑ i = 1 N 1 ω T x i S 1 = 1 N 1 ∑ i = 1 N 1 ( ω T x i − z 1 ˉ ) ( ω T x i − z 1 ˉ ) T = 1 N 1 ∑ i = 1 N 1 ( ω T x i − 1 N 1 ∑ j = 1 N 1 ω T x j ) ( ω T x i − 1 N 1 ∑ j = 1 N 1 ω T x j ) T 这里定义 1 N 1 ∑ j = 1 N 1 x j = x C 1 ‾ = 1 N 1 ∑ i = 1 N 1 ω T ( x i − x C 1 ‾ ) ( x i − x C 1 ‾ ) T ω = ω T ( 1 N 1 ∑ i = 1 N 1 ( x i − x C 1 ‾ ) ( x i − x C 1 ‾ ) T ) ω 这里定义 1 N 1 ∑ i = 1 N 1 ( x i − x C 1 ‾ ) ( x i − x C 1 ‾ ) T = S C 1 = ω T S C 1 ω C 2 : z 2 ˉ = 1 N 2 ∑ i = 1 N 2 ω T x i S 2 = ω T S C 2 ω

z¯=1N∑i=1Nzi=1N∑i=1NωTxiC1:z1¯=1N1∑i=1N1ωTxiS1=1N1∑i=1N1(ωTxi−z1¯)(ωTxi−z1¯)T=1N1∑i=1N1(ωTxi−1N1∑j=1N1ωTxj)(ωTxi−1N1∑j=1N1ωTxj)T这里定义1N1∑j=1N1xj=xC1¯=1N1∑i=1N1ωT(xi−xC1¯)(xi−xC1¯)Tω=ωT(1N1∑i=1N1(xi−xC1¯)(xi−xC1¯)T)ω这里定义1N1∑i=1N1(xi−xC1¯)(xi−xC1¯)T=SC1=ωTSC1ωC2:z2¯=1N2∑i=1N2ωTxiS2=ωTSC2ω" role="presentation">z¯C1:z1¯S1C2:z2¯S2=1N∑i=1Nzi=1N∑i=1NωTxi=1N1∑i=1N1ωTxi=1N1∑i=1N1(ωTxi−z1¯)(ωTxi−z1¯)T=1N1∑i=1N1(ωTxi−1N1∑j=1N1ωTxj)(ωTxi−1N1∑j=1N1ωTxj)T这里定义1N1∑j=1N1xj=xC1¯¯¯¯¯¯¯=1N1∑i=1N1ωT(xi−xC1¯¯¯¯¯¯¯)(xi−xC1¯¯¯¯¯¯¯)Tω=ωT(1N1∑i=1N1(xi−xC1¯¯¯¯¯¯¯)(xi−xC1¯¯¯¯¯¯¯)T)ω这里定义1N1∑i=1N1(xi−xC1¯¯¯¯¯¯¯)(xi−xC1¯¯¯¯¯¯¯)T=SC1=ωTSC1ω=1N2∑i=1N2ωTxi=ωTSC2ω$$\begin{array}{rl}\overline{z}& =\frac{1}{N}\sum _{i=1}^N{z}_i=\frac{1}{N}\sum _{i=1}^N{\omega }^T{x}_i\\ C_1:\overline{z_1}& =\frac{1}{N_1}\sum _{i=1}^{N_1}{\omega }^T{x}_i\\ S_1& =\frac{1}{N_1}\sum _{i=1}^{N_1}({\omega }^T{x}_i-\overline{z_1})({\omega }^T{x}_i-\overline{z_1}{)}^T\\ & =\frac{1}{N_1}\sum _{i=1}^{N_1}({\omega }^T{x}_i-\frac{1}{N_1}\sum _{j=1}^{N_1}{\omega }^T{x}_j)({\omega }^T{x}_i-\frac{1}{N_1}\sum _{j=1}^{N_1}{\omega }^T{x}_j{)}^T\\ & 这里定义\frac{1}{N_1}\sum _{j=1}^{N_1}{x}_j=\overline{x_{C_1}}\\ & =\frac{1}{N_1}\sum _{i=1}^{N_1}{\omega }^T(x_i-\overline{x_{C_1}})(x_i-\overline{x_{C_1}}{)}^T\omega \\ & ={\omega }^T\left(\frac{1}{N_1}\sum _{i=1}^{N_1}(x_i-\overline{x_{C_1}})(x_i-\overline{x_{C_1}}{)}^T\right)\omega \\ & 这里定义\frac{1}{N_1}\sum _{i=1}^{N_1}(x_i-\overline{x_{C_1}})(x_i-\overline{x_{C_1}}{)}^T=S_{C_1}\\ & ={\omega }^T{S}_{C_1}\omega \\ C_2:\overline{z_2}& =\frac{1}{N_2}\sum _{i=1}^{N_2}{\omega }^T{x}_i\\ S_2& ={\omega }^T{S}_{C_2}\omega \end{array}$$

zˉC1​:z1​