nn.functional.normalize

torch.nn.functional.normalize

官方文档: normalize

Performs $L_p$ normalization of inputs over specified dimension.

作用: 在指定的维度计算$p$范数，默认的计算是2范数

计算公式如下

$$v=\frac{v}{max(||v|{|}_p,ϵ)}$$

下面对函数的参数进行解释

torch.nn.functional.normalize(input, p=2.0, dim=1, eps=1e-12, out=None)

• input: 输入tensor
• p: 归一化的范数，默认为2范数
• dim: 计算维度 (the dimension to reduce)
• eps: 防止分母为0的一个很小得数
• out: 输出tensor，如果指定了这个tensor，操作会变得不可微

下面举两个例子，一个二维一个三维

二维例子

input2 = torch.randn((3, 4))
output2_1 = nn.functional.normalize(input2)
print(output2_1)

'''
input2:
tensor([[-0.9216,  0.2382,  0.0036,  0.1124],
        [ 0.6481,  0.0569,  2.6192,  0.6064],
        [-0.5110, -0.4260,  1.5873, -0.3685]])

output2_1
tensor([[-0.9615,  0.2485,  0.0037,  0.1173],
        [ 0.2343,  0.0206,  0.9469,  0.2192],
        [-0.2903, -0.2420,  0.9018, -0.2094]])
'''
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

normalize默认是计算dim=1的，输入矩阵是(3,4)，计算第1维就是4那一维(dim=0代表3那一维)。如下图所示，dim=1代表4那一维度，就是红色的那四个元素计算归一化，蓝色同理

即

− 0.9615 = − 0.9216 ( − 0.9216 ) 2 + ( 0.2382 ) 2 + ( 0.0036 ) 2 + ( 0.1124 ) 2 0.2485 = 0.2382 ( − 0.9216 ) 2 + ( 0.2382 ) 2 + ( 0.0036 ) 2 + ( 0.1124 ) 2 0.0037 = 0.0036 ( − 0.9216 ) 2 + ( 0.2382 ) 2 + ( 0.0036 ) 2 + ( 0.1124 ) 2 0.1173 = 0.1124 ( − 0.9216 ) 2 + ( 0.2382 ) 2 + ( 0.0036 ) 2 + ( 0.1124 ) 2

−0.96150.24850.00370.1173​=(−0.9216)2+(0.2382)2+(0.0036)2+(0.1124)2 ​−0.9216​=(−0.9216)2+(0.2382)2+(0.0036)2+(0.1124)2 ​0.2382​=(−0.9216)2+(0.2382)2+(0.0036)2+(0.1124)2 ​0.0036​=(−0.9216)2+(0.2382)2+(0.0036)2+(0.1124)2 ​0.1124​​

三维例子

input3 = torch.randn((2, 3, 4))
output3_1 = nn.functional.normalize(input3)
print(output3_1)

'''
input3
tensor([[[ 0.0125,  0.0806,  1.0750, -0.9401],
         [ 0.1409,  2.8251, -0.2162,  0.9788],
         [-1.1588,  0.7537, -0.0691, -0.6371]],

        [[-0.2402, -0.8545,  1.2948, -1.4992],
         [-0.1676, -1.0821,  1.7947, -1.5175],
         [-0.1969,  0.3079, -1.2304, -0.7987]]])

output3_1
tensor([[[ 0.0107,  0.0276,  0.9784, -0.6270],
         [ 0.1207,  0.9658, -0.1968,  0.6529],
         [-0.9926,  0.2577, -0.0629, -0.4250]],

        [[-0.6805, -0.6048,  0.5114, -0.6582],
         [-0.4749, -0.7659,  0.7088, -0.6662],
         [-0.5580,  0.2180, -0.4859, -0.3507]]])
'''
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

这里dim=1也就是3那一维，所以对3个数进行归一化，这里以0.0107为例

$$0.0107=\frac{0.0125}{\sqrt{(0.0125{)}^2+(0.1409{)}^2+(-1.1588{)}^2}}$$