极限的意义 重要极限
lim
x
→
0
sin
x
x
\lim\limits_{x \to 0} \frac{\sin x}{x}
x→0limxsinx和
lim
x
→
∞
(
1
+
1
x
)
x
\lim\limits_{x \to \infty} (1+\frac{1}{x})^x
x→∞lim(1+x1)x
闭区间连续函数的性质 (有界性,最值性,介值性)
微分与积分
微分的意义 曲率
κ
=
d
α
d
s
=
d
arctan
d
y
/
d
x
(
d
x
)
2
+
(
d
y
)
2
\kappa = \frac{\mathrm{d} \alpha}{\mathrm{d} s} = \frac{\mathrm{d} \arctan \mathrm{d} y / \mathrm{d} x}{\sqrt{(\mathrm{d} x)^2 + (\mathrm{d} y)^2}}
κ=dsdα=(dx)2+(dy)2darctandy/dx
微分的运算 反函数
d
x
d
y
=
(
d
y
d
x
)
−
1
\frac{\mathrm{d} x}{\mathrm{d} y} = (\frac{\mathrm{d} y}{\mathrm{d} x})^{-1}
dydx=(dxdy)−1 链式法则
d
y
d
x
=
∑
u
d
y
d
u
d
u
d
x
\frac{\mathrm{d} y}{\mathrm{d} x} = \sum\limits_{u} \frac{\mathrm{d} y}{\mathrm{d} u} \frac{\mathrm{d} u}{\mathrm{d} x}
dxdy=u∑dudydxdu
积分的运算 换元
∬
f
(
x
,
y
)
d
x
d
y
=
∬
f
(
x
(
u
,
v
)
,
y
(
u
,
v
)
)
∣
∂
(
x
,
y
)
∂
(
u
,
v
)
∣
d
u
d
v
\iint f(x,y) \mathrm{d} x \mathrm{d} y = \iint f(x(u,v),y(u,v)) |\frac{\partial (x,y)}{\partial (u,v)}| \mathrm{d} u \mathrm{d} v
∬f(x,y)dxdy=∬f(x(u,v),y(u,v))∣∂(u,v)∂(x,y)∣dudv 概率密度函数中的换元
q
(
u
,
v
)
=
p
(
x
(
u
,
v
)
,
y
(
u
,
v
)
)
∣
∂
(
x
,
y
)
∂
(
u
,
v
)
∣
q(u,v) = p(x(u,v),y(u,v)) |\frac{\partial (x,y)}{\partial (u,v)}|
q(u,v)=p(x(u,v),y(u,v))∣∂(u,v)∂(x,y)∣
D
[
X
]
=
E
[
(
X
−
E
[
X
]
)
2
]
=
E
[
X
2
]
−
E
2
[
X
]
⩾
0
D[X] = E[(X-E[X])^2] = E[X^2] - E^2[X] \geqslant 0
D[X]=E[(X−E[X])2]=E[X2]−E2[X]⩾0 (期望的线性性质,期望的Jensen不等式)
E
[
(
X
−
e
)
2
]
=
E
[
(
X
−
E
[
X
]
)
2
]
+
(
E
[
X
]
−
e
)
2
E[(X-e)^2] = E[(X-E[X])^2] + (E[X]-e)^2
E[(X−e)2]=E[(X−E[X])2]+(E[X]−e)2 (期望的线性性质)
D
[
c
X
+
d
]
=
c
2
D
[
X
]
D[cX+d] = c^2 D[X]
D[cX+d]=c2D[X] (期望的线性性质)
D
[
X
]
⩽
(
b
−
E
[
X
]
)
(
E
[
X
]
−
a
)
⩽
1
4
(
b
−
a
)
2
D[X] \leqslant (b-E[X])(E[X]-a) \leqslant \frac{1}{4} (b-a)^2
D[X]⩽(b−E[X])(E[X]−a)⩽41(b−a)2 (期望的线性性质)
协方差
C
o
v
(
X
,
Y
)
=
E
[
(
X
−
E
[
X
]
)
(
Y
−
E
[
Y
]
)
]
=
E
[
X
Y
]
−
E
[
X
]
E
[
Y
]
\mathrm{Cov}(X,Y) = E[(X-E[X])(Y-E[Y])] = E[XY]-E[X]E[Y]
Cov(X,Y)=E[(X−E[X])(Y−E[Y])]=E[XY]−E[X]E[Y]
协方差矩阵
Σ
i
j
=
C
o
v
(
X
i
,
X
j
)
\Sigma_{ij} = \mathrm{Cov}(X_i,X_j)
Σij=Cov(Xi,Xj)
条件期望
条件概率
全概率公式,贝叶斯公式
独立性
E
[
X
Y
]
⩽
E
[
X
2
]
E
[
Y
2
]
E[XY] \leqslant \sqrt{E[X^2]E[Y^2]}
E[XY]⩽E[X2]E[Y2],
C
o
v
(
X
,
Y
)
⩽
D
[
X
]
D
[
Y
]
\mathrm{Cov}(X,Y) \leqslant \sqrt{D[X]D[Y]}
Cov(X,Y)⩽D[X]D[Y]
ρ
(
X
,
Y
)
=
C
o
v
(
X
,
Y
)
/
D
[
X
]
D
[
Y
]
\rho(X,Y) = \mathrm{Cov}(X,Y) / \sqrt{D[X]D[Y]}
ρ(X,Y)=Cov(X,Y)/D[X]D[Y]
离散随机变量
01分布,
X
∼
B
(
1
,
p
)
X \sim B(1,p)
X∼B(1,p),
E
[
X
]
=
p
E[X]=p
E[X]=p,
D
[
x
]
=
p
(
1
−
p
)
D[x]=p(1-p)
D[x]=p(1−p)
二项分布,
X
∼
B
(
n
,
p
)
X \sim B(n,p)
X∼B(n,p),
Pr
(
X
=
k
)
=
(
n
k
,
n
−
k
)
p
k
(
1
−
p
)
n
−
k
\Pr(X=k)=\binom{n}{k,n-k}p^k(1-p)^{n-k}
Pr(X=k)=(k,n−kn)pk(1−p)n−k,
E
[
X
]
=
n
p
E[X]=np
E[X]=np,
D
[
x
]
=
n
p
(
1
−
p
)
D[x]=np(1-p)
D[x]=np(1−p)
几何分布,
X
∼
G
(
p
)
X \sim G(p)
X∼G(p),
Pr
(
X
=
k
)
=
(
1
−
p
)
k
−
1
p
\Pr(X=k)=(1-p)^{k-1}p
Pr(X=k)=(1−p)k−1p,
E
[
X
]
=
1
p
E[X]=\frac{1}{p}
E[X]=p1,
D
[
x
]
=
1
p
(
1
p
−
1
)
D[x]=\frac{1}{p}(\frac{1}{p}-1)
D[x]=p1(p1−1)
泊松分布,
X
∼
P
(
λ
)
X \sim P(\lambda)
X∼P(λ),
Pr
(
X
=
k
)
=
λ
k
k
!
e
−
λ
\Pr(X=k)=\frac{\lambda^k}{k!}e^{-\lambda}
Pr(X=k)=k!λke−λ,
E
[
X
]
=
λ
E[X]=\lambda
E[X]=λ,
D
[
x
]
=
λ
D[x]=\lambda
D[x]=λ
n
p
=
λ
np=\lambda
np=λ,
n
→
∞
n \to \infty
n→∞,
p
→
0
p \to 0
p→0,
(
n
k
,
n
−
k
)
p
k
(
1
−
p
)
n
−
k
→
λ
k
k
!
e
−
λ
\binom{n}{k,n-k}p^k(1-p)^{n-k} \to \frac{\lambda^k}{k!}e^{-\lambda}
(k,n−kn)pk(1−p)n−k→k!λke−λ
连续随机变量
均匀分布,
X
∼
U
(
a
,
b
)
X \sim U(a,b)
X∼U(a,b),
p
(
x
)
=
δ
(
a
⩽
x
⩽
b
)
1
b
−
a
p(x)=\delta(a \leqslant x \leqslant b) \frac{1}{b-a}
p(x)=δ(a⩽x⩽b)b−a1,
E
[
X
]
=
1
2
(
a
+
b
)
E[X]=\frac{1}{2}(a+b)
E[X]=21(a+b),
D
[
x
]
=
1
12
(
b
−
a
)
2
D[x]=\frac{1}{12}(b-a)^2
D[x]=121(b−a)2
指数分布,
X
∼
e
(
β
)
X \sim e(\beta)
X∼e(β),
p
(
x
)
=
δ
(
x
>
0
)
1
β
e
−
1
β
x
p(x)=\delta(x>0) \frac{1}{\beta} e^{- \frac{1}{\beta} x}
p(x)=δ(x>0)β1e−β1x,
E
[
X
]
=
β
E[X]=\beta
E[X]=β,
D
[
x
]
=
β
2
D[x]=\beta^2
D[x]=β2
正态分布,
X
∼
N
(
μ
,
σ
2
)
X \sim \mathcal{N}(\mu,\sigma^2)
X∼N(μ,σ2),
p
(
x
)
=
1
2
π
σ
2
exp
(
−
1
2
σ
2
(
x
−
μ
)
2
)
p(x)=\frac{1}{\sqrt{2\pi\sigma^2}} \exp(-\frac{1}{2\sigma^2}(x-\mu)^2)
p(x)=2πσ21exp(−2σ21(x−μ)2),
E
[
X
]
=
μ
E[X]=\mu
E[X]=μ,
D
[
x
]
=
σ
2
D[x]=\sigma^2
D[x]=σ2
Markov不等式
Pr
(
X
⩾
a
)
⩽
E
[
X
]
a
\Pr(X \geqslant a) \leqslant \frac{E[X]}{a}
Pr(X⩾a)⩽aE[X] (
X
⩾
0
X \geqslant 0
X⩾0)
Chebyshev不等式
Pr
(
(
X
−
μ
)
2
⩾
σ
2
)
⩽
D
[
X
]
σ
2
\Pr((X-\mu)^2 \geqslant \sigma^2) \leqslant \frac{D[X]}{\sigma^2}
Pr((X−μ)2⩾σ2)⩽σ2D[X] 单边Chebyshev不等式
Pr
(
X
−
μ
⩾
σ
)
,
Pr
(
X
−
μ
⩽
−
σ
)
⩽
D
[
X
]
D
[
X
]
+
σ
2
\Pr(X-\mu \geqslant \sigma),\Pr(X-\mu \leqslant -\sigma) \leqslant \frac{D[X]}{D[X]+\sigma^2}
Pr(X−μ⩾σ),Pr(X−μ⩽−σ)⩽D[X]+σ2D[X]
Chernoff不等式
Pr
(
e
t
X
⩾
e
t
a
)
⩽
E
[
e
t
X
]
e
t
a
\Pr(e^{tX} \geqslant e^{ta}) \leqslant \frac{E[e^{tX}]}{e^{ta}}
Pr(etX⩾eta)⩽etaE[etX]
01分布独立
Pr
(
X
ˉ
⩾
(
1
+
ϵ
)
p
ˉ
)
⩽
(
e
ϵ
(
1
+
ϵ
)
(
1
+
ϵ
)
)
p
ˉ
⩽
(
e
−
ϵ
2
/
3
)
p
ˉ
\Pr(\bar{X} \geqslant (1+\epsilon) \bar{p}) \leqslant (\frac{e^\epsilon}{(1+\epsilon)^{(1+\epsilon)}})^{\bar{p}} \leqslant (e^{-\epsilon^2/3})^{\bar{p}}
Pr(Xˉ⩾(1+ϵ)pˉ)⩽((1+ϵ)(1+ϵ)eϵ)pˉ⩽(e−ϵ2/3)pˉ
E
[
e
t
X
]
=
(
1
−
p
)
+
p
e
t
=
1
+
p
(
e
t
−
1
)
⩽
e
p
(
e
t
−
1
)
E[e^{tX}] = (1-p) + pe^t = 1 + p(e^t-1) \leqslant e^{p(e^t-1)}
E[etX]=(1−p)+pet=1+p(et−1)⩽ep(et−1)
Pr
(
X
ˉ
⩾
(
1
−
ϵ
)
p
ˉ
)
⩽
(
e
ϵ
(
1
−
ϵ
)
(
1
−
ϵ
)
)
p
ˉ
⩽
(
e
−
ϵ
2
/
2
)
p
ˉ
\Pr(\bar{X} \geqslant (1-\epsilon) \bar{p}) \leqslant (\frac{e^\epsilon}{(1-\epsilon)^{(1-\epsilon)}})^{\bar{p}} \leqslant (e^{-\epsilon^2/2})^{\bar{p}}
Pr(Xˉ⩾(1−ϵ)pˉ)⩽((1−ϵ)(1−ϵ)eϵ)pˉ⩽(e−ϵ2/2)pˉ
[0,1]上分布独立
Pr
(
X
ˉ
⩾
μ
ˉ
+
ϵ
)
⩽
e
−
2
n
ϵ
2
\Pr(\bar{X} \geqslant \bar{\mu}+\epsilon) \leqslant e^{- 2n \epsilon^2}
Pr(Xˉ⩾μˉ+ϵ)⩽e−2nϵ2
E
[
e
t
X
]
⩽
(
1
−
μ
)
+
μ
e
t
⩽
e
t
μ
+
t
2
/
8
E[e^{tX}] \leqslant (1 - \mu) + \mu e^t \leqslant e^{t \mu + t^2 / 8}
E[etX]⩽(1−μ)+μet⩽etμ+t2/8
e
t
X
=
e
X
t
+
(
1
−
X
)
0
⩽
X
e
t
+
(
1
−
X
)
e^{tX} = e^{Xt + (1-X)0} \leqslant Xe^t + (1-X)
etX=eXt+(1−X)0⩽Xet+(1−X)
𝒩(0,1)独立
Pr
(
X
ˉ
⩾
ϵ
)
⩽
1
2
e
−
n
ϵ
2
/
2
\Pr(\bar{X} \geqslant \epsilon) \leqslant \frac{1}{2} e^{-n\epsilon^2/2}
Pr(Xˉ⩾ϵ)⩽21e−nϵ2/2
X
ˉ
∼
N
(
0
,
1
n
)
\bar{X} \sim \mathcal{N}(0,\frac{1}{n})
Xˉ∼N(0,n1)
Bennet和Bernstein不等式 (引入方差和更多约束)
大数定律
1
n
∑
i
=
1
n
X
i
→
Pr
1
n
∑
i
=
1
n
E
[
X
i
]
\frac{1}{n} \sum\limits_{i=1}^{n} X_i \stackrel{\Pr}{\to} \frac{1}{n} \sum\limits_{i=1}^{n} E[X_i]
n1i=1∑nXi→Prn1i=1∑nE[Xi]
Markov大数定律
1
n
2
D
[
∑
i
=
1
n
X
i
]
→
0
\frac{1}{n^2}D[\sum\limits_{i=1}^{n} X_i] \to 0
n21D[i=1∑nXi]→0
Chebyshev大数定律
D
[
X
i
]
⩽
C
o
n
s
t
D[X_i] \leqslant \mathrm{Const}
D[Xi]⩽Const (
ρ
(
X
i
,
X
j
)
=
0
\rho(X_i,X_j)=0
ρ(Xi,Xj)=0)
Khintchine大数定律
E
[
X
i
]
<
∞
E[X_i]<\infty
E[Xi]<∞ (i.i.d.)
Bernoulli大数定律
中心极限定理
Lindburg-Levy中心极限定理
(
∑
i
=
1
n
X
i
−
n
μ
)
/
n
σ
2
→
d
N
(
0
,
1
)
(\sum\limits_{i=1}^{n} X_i - n \mu) / \sqrt{n\sigma^2} \stackrel{d}{\to} \mathcal{N}(0,1)
(i=1∑nXi−nμ)/nσ2→dN(0,1) (i.i.d.)
De Moivre-Laplace中心极限定理
X
i
∼
B
(
1
,
p
)
X_i \sim B(1,p)
Xi∼B(1,p) (
Y
n
=
∑
i
=
1
n
X
i
∼
B
(
n
,
p
)
Y_n = \sum\limits_{i=1}^{n} X_i \sim B(n,p)
Yn=i=1∑nXi∼B(n,p))
Lyapunov中心极限定理
(
∑
i
=
1
n
X
i
−
∑
i
=
1
n
μ
i
)
/
∑
i
=
1
n
σ
i
2
→
d
N
(
0
,
1
)
\left. \left(\sum\limits_{i=1}^{n} X_i - \sum\limits_{i=1}^{n} \mu_i\right) \middle/ \sqrt{\sum\limits_{i=1}^{n} \sigma_i^2} \right. \stackrel{d}{\to} \mathcal{N}(0,1)
(i=1∑nXi−i=1∑nμi)/i=1∑nσi2→dN(0,1) (独立) (
(
∑
i
=
1
n
E
[
∣
X
i
−
μ
i
∣
2
+
δ
]
)
1
2
+
δ
/
∑
i
=
1
n
σ
i
2
→
0
\left. \left(\sum\limits_{i=1}^{n} E[|X_i-\mu_i|^{2+\delta}]\right)^{\frac{1}{2+\delta}} \middle/ \sqrt{\sum\limits_{i=1}^{n} \sigma_i^2} \right. \to 0
(i=1∑nE[∣Xi−μi∣2+δ])2+δ1/i=1∑nσi2→0)
统计与检验
样本k阶原点矩
A
k
=
1
n
∑
i
=
1
n
X
i
k
A_k = \frac{1}{n} \sum\limits_{i=1}^{n} X_i^k
Ak=n1i=1∑nXik,样本k阶中心矩
B
k
=
1
n
∑
i
=
1
n
(
X
i
−
X
ˉ
)
k
B_k = \frac{1}{n} \sum\limits_{i=1}^{n} (X_i-\bar{X})^k
Bk=n1i=1∑n(Xi−Xˉ)k
样本方差
E
[
s
0
2
]
=
(
1
−
1
n
)
σ
2
E[s_0^2] = (1-\frac{1}{n}) \sigma^2
E[s02]=(1−n1)σ2
最小值,最大值,极差,次序统计量
P
k
(
x
)
=
∑
i
=
k
n
(
n
i
,
n
−
i
)
(
P
(
x
)
)
i
(
1
−
P
(
x
)
)
n
−
i
P_k(x) = \sum\limits_{i=k}^{n} \binom{n}{i,n-i} (P(x))^i (1-P(x))^{n-i}
Pk(x)=i=k∑n(i,n−in)(P(x))i(1−P(x))n−i
∑
i
=
k
n
(
n
i
,
n
−
i
)
p
i
(
1
−
p
)
n
−
i
=
n
(
n
−
1
k
−
1
,
(
n
−
1
)
−
(
k
−
1
)
)
∫
0
p
t
k
−
1
(
1
−
t
)
(
n
−
1
)
−
(
k
−
1
)
d
t
\sum\limits_{i=k}^{n} \binom{n}{i,n-i} p^i (1-p)^{n-i} = n \binom{n-1}{k-1,(n-1)-(k-1)} \int\limits_{0}^{p} t^{k-1}(1-t)^{(n-1)-(k-1)} \mathrm{d}t
i=k∑n(i,n−in)pi(1−p)n−i=n(k−1,(n−1)−(k−1)n−1)0∫ptk−1(1−t)(n−1)−(k−1)dt
p
(
x
)
←
Pr
(
X
∈
(
x
−
ϵ
,
x
+
ϵ
)
)
p(x) \gets \Pr(X \in (x-\epsilon,x+\epsilon))
p(x)←Pr(X∈(x−ϵ,x+ϵ))
Beta分布,Gamma分布,Dirichlet分布
正态总体抽象分布
χ2分布(卡方分布)
Y
=
∑
i
=
1
n
X
i
2
Y = \sum\limits_{i=1}^{n} X_i^2
Y=i=1∑nXi2 (样本方差)
t分布(学生分布)
Y
=
X
/
1
n
∑
i
=
1
n
X
i
2
Y = \left. X \middle/ \sqrt{\frac{1}{n} \sum\limits_{i=1}^{n} X_i^2} \right.
Y=X/n1i=1∑nXi2 (均值和样本方差)
F分布
Y
=
(
1
m
∑
j
=
1
m
X
j
2
)
/
(
1
n
∑
i
=
1
n
X
i
2
)
Y = \left. \left(\frac{1}{m} \sum\limits_{j=1}^{m} X_j^2\right) \middle/ \left(\frac{1}{n} \sum\limits_{i=1}^{n} X_i^2\right) \right.
Y=(m1j=1∑mXj2)/(n1i=1∑nXi2) (两个正态分布总体)
X
ˉ
\bar{X}
Xˉ与
s
0
2
s_0^2
s02独立
分位数(分位点)
参数估计
区间估计
枢轴变量(正态分布总体)
非正态分布总体
单侧置信区间
点估计
矩估计法
极大似然估计法
一致性,无偏性,有效性
假设检验(根据统计量和可接受置信度(显著性水平)计算置信区间(拒绝域))
方差已知的单个正态总体的期望检验(Z检验)
独立性检验
n
∑
i
,
j
(
p
^
i
j
−
p
^
i
p
^
j
)
2
p
^
i
p
^
j
∼
χ
2
(
(
#
i
−
1
)
(
#
j
−
1
)
)
n \sum\limits_{i,j} \frac{(\hat{p}_{ij} - \hat{p}_i\hat{p}_j)^2}{\hat{p}_i\hat{p}_j} \sim \chi^2((\#i-1)(\#j-1))
ni,j∑p^ip^j(p^ij−p^ip^j)2∼χ2((#i−1)(#j−1))
凸优化
Young不等式
a
b
⩽
1
p
a
p
+
1
q
b
q
ab \leqslant \frac{1}{p}a^p + \frac{1}{q}b^q
ab⩽p1ap+q1bq (
a
,
b
>
0
a,b>0
a,b>0,
p
,
q
>
0
p,q>0
p,q>0,
1
p
+
1
q
=
1
\frac{1}{p}+\frac{1}{q}=1
p1+q1=1) Holder不等式
E
[
X
Y
]
⩽
E
[
X
p
]
1
p
E
[
Y
q
]
1
q
E[XY] \leqslant E[X^p]^{\frac{1}{p}}E[Y^q]^{\frac{1}{q}}
E[XY]⩽E[Xp]p1E[Yq]q1 (
X
,
Y
>
0
X,Y>0
X,Y>0,
p
,
q
>
0
p,q>0
p,q>0,
1
p
+
1
q
=
1
\frac{1}{p}+\frac{1}{q}=1
p1+q1=1)