Zlib和Zstd 性能对比评测

Zstandard 是facebook厂商提供的一种高压缩率的实时压缩算法，他在压缩比和压缩速度上做了很好的平衡，同时还提供高速的解码器，针对小数据，也提供了一种叫做字典压缩算法，从样本中可以创建出字典集合数据，Zstandard库以BSD协议方式开源。
该算法是由Facebook的Yann Collet开发的一个无损数据压缩算法。Zstandard在设计上与DEFLATE（.zip、gzip）算法有着差不多的压缩比，但有更高的压缩和解压缩速度。
Zstandard使用字典算法（LZ77）结合熵编码法的有限状态熵（tANS）。—–Wikipedia

Zlib 就不介绍了，大家都很熟悉了，一套古老的压缩算法。性能对比，两年前做过简单对比，Zstandard （zstd）压缩算法在JAVA上的使用
这次再做一次深入的评测和对比，其中，实验样本用的是我们游戏协议的文本样本，测试结果见文章结尾：

和zlib对比

1.在小数据量下（256），level=3,相对zlib,!!#ff0000 压缩效率大幅提高!!(94230.272 -> 153682 , 163.09%)，表现更加稳定(± 7958.782 ops/s–>±1801.146 ops/s)，解压缩效率小幅降低（608231.351 -> 565699.876 , -7%），并且表现更加稳定± 312460.294 ops/s -->± 12919.602 ops/s)， 但压缩比会小幅降低（18.0%–>17.97%）
2.在大数据量下（8192），level=3,相对zlib!!#ff0000 ,压缩效率大幅提高!!(3939.309-> 29897.221 ,758.99%)，稳定性小幅降低(45.24/3939.309=1.14%->513.773/ 29897.221 =1.71%)，解压缩效率大幅提高（30828.328 -> 82746.561 , 268.41%），稳定性小幅提高(（4388.578/30828.328=14.23%–>8866.313/ 82746.561=10.71%)， 压缩比基本持平（ 63.5%–>63.48%）

总结 1，从数据上看出，zstandard比zlib,同样的压缩比情况下，速度高于zlib,同样的速度情况下，压缩比要高于zlib

自身不同level 对比

3.小数据量下(256 bytes)，随着level增大，压缩率会小幅提高，提升不明显，在level>=2 下 ，!!#ff0000 压缩效率会大幅降低，!!解压缩效率也会小幅降低。

压缩（level=3）

其中,ratio=1-compress/size

Benchmark                                   (level)  (size)   Mode  Cnt       Score       Error  Units   compress    ratio
Compress.ZstdLevelBenchMark.zstdCompress          1     256  thrpt   10  158247.321      2539.721  ops/s  219       14.45%
Compress.ZstdLevelBenchMark.zstdCompress          2     256  thrpt   10  160513.122      1055.204  ops/s   217      15.23%
Compress.ZstdLevelBenchMark.zstdCompress          3     256  thrpt   10  153682.210      1801.146  ops/s   210     	17.97%
Compress.ZstdLevelBenchMark.zstdCompress          4     256  thrpt   10  133600.466      1974.326  ops/s   206	    19.53%
Compress.ZstdLevelBenchMark.zstdCompress          5     256  thrpt   10  124519.999      3317.794  ops/s   204	    20.31%
Compress.ZstdLevelBenchMark.zstdCompress          6     256  thrpt   10  122500.287      1030.539  ops/s   204	    20.31%
1
2
3
4
5
6
7

解压（level=3）

Benchmark                                   (level)  (size)   Mode  Cnt       Score       Error  Units   
Compress.ZstdLevelBenchMark.zstdUncompress        1     256  thrpt   10  582084.385     7778.949  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2     256  thrpt   10  580312.371     6809.382  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3     256  thrpt   10  565699.876     12919.602  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4     256  thrpt   10  562818.616      5036.532  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5     256  thrpt   10  501616.311      5596.259  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6     256  thrpt   10  494291.843      7102.794  ops/s
1
2
3
4
5
6
7

4.大数据量下(8192 bytes)，随着level增大，压缩率会小幅提高，提升不明显，!!#ff0000 压缩效率会大幅降低!!，解压缩效率会小幅提高

压缩（level=6）

Benchmark                                   (level)  (size)   Mode  Cnt       Score       Error  Units   compress    ratio
Compress.ZstdLevelBenchMark.zstdCompress          1    8192  thrpt   10   34586.557      302.868   ops/s   3091     62.27%
Compress.ZstdLevelBenchMark.zstdCompress          2    8192  thrpt   10   34113.274      243.294  ops/s   3035      62.95%
Compress.ZstdLevelBenchMark.zstdCompress          3    8192  thrpt   10   29897.221       513.773  ops/s   2992	    63.48%
Compress.ZstdLevelBenchMark.zstdCompress          4    8192  thrpt   10   18699.637       176.416  ops/s   2948	    64.01%
Compress.ZstdLevelBenchMark.zstdCompress          5    8192  thrpt   10   13744.149        76.323  ops/s   2885	    64.78%
Compress.ZstdLevelBenchMark.zstdCompress          6    8192  thrpt   10    9931.221        79.205  ops/s   2886	    64.77%
1
2
3
4
5
6
7

解压（level=6）

Benchmark                                   (level)  (size)   Mode  Cnt       Score       Error  Units 
Compress.ZstdLevelBenchMark.zstdUncompress        1    8192  thrpt   10   83840.509     9960.244  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2    8192  thrpt   10   83274.771     6297.771  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3    8192  thrpt   10   82746.561      8866.313  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4    8192  thrpt   10   84652.976      6150.251  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5    8192  thrpt   10   84665.619      6465.654  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6    8192  thrpt   10   85139.016      5582.793  ops/s
1
2
3
4
5
6
7

总结 2，从以上3，4点对比，默认值3 ，或者 2 可以认为是相对较优解。

综上所述，
在协议这块，服务器用压缩API的频率远大于解压缩API频率， 在由zlib替换了level=3的Zstd后，压缩效率会大幅提高，表现也更加稳定，压缩率会有小幅降低。
在level的调整方面，随着level增大，压缩率会有小幅提高，但压缩效率会大幅降低，得不偿失，所以保持默认值3或者2是比较合适的。如果追求压缩/解压缩效率则选择2，追求压缩比，则选择3

##测试结果

zlib

Benchmark                                   (size)   Mode  Cnt       Score        Error  Units   compress    ratio
Compress.ZstdLevelBenchMark.zlibCompress       256  thrpt   10   94230.272 ±   7958.782  ops/s   197          18.0%
Compress.ZstdLevelBenchMark.zlibCompress       512  thrpt   10   67053.071 ±   5606.840  ops/s   308		  39.3%
Compress.ZstdLevelBenchMark.zlibCompress      1024  thrpt   10   32361.968 ±   1426.980  ops/s   448          58.5%
Compress.ZstdLevelBenchMark.zlibCompress      2048  thrpt   10   15301.493 ±    354.073  ops/s   733          64.3%
Compress.ZstdLevelBenchMark.zlibCompress      4096  thrpt   10    6656.437 ±     96.692  ops/s   1359         68.4%
Compress.ZstdLevelBenchMark.zlibCompress      8192  thrpt   10    3939.309 ±     45.240  ops/s   3126         63.5%
Compress.ZstdLevelBenchMark.zlibUncompress     256  thrpt   10  608231.351 ± 312460.294  ops/s
Compress.ZstdLevelBenchMark.zlibUncompress     512  thrpt   10  460449.016 ± 206413.902  ops/s
Compress.ZstdLevelBenchMark.zlibUncompress    1024  thrpt   10  169738.557 ±  33999.682  ops/s
Compress.ZstdLevelBenchMark.zlibUncompress    2048  thrpt   10  110027.019 ±  17135.330  ops/s
Compress.ZstdLevelBenchMark.zlibUncompress    4096  thrpt   10   65892.491 ±  11272.236  ops/s
Compress.ZstdLevelBenchMark.zlibUncompress    8192  thrpt   10   30828.328 ±   4388.578  ops/s
1
2
3
4
5
6
7
8
9
10
11
12
13

Zstd

Benchmark                                   (level)  (size)   Mode  Cnt       Score       Error  Units   compress     ratio
Compress.ZstdLevelBenchMark.zstdCompress          1     256  thrpt   10  158247.321      2539.721  ops/s  219         14.45%
Compress.ZstdLevelBenchMark.zstdCompress          1     512  thrpt   10  138946.669      707.322  ops/s   314         38.67%
Compress.ZstdLevelBenchMark.zstdCompress          1    1024  thrpt   10  116642.039      1013.260  ops/s  443         56.74%
Compress.ZstdLevelBenchMark.zstdCompress          1    2048  thrpt   10   84307.054      1845.538  ops/s  738         63.96%
Compress.ZstdLevelBenchMark.zstdCompress          1    4096  thrpt   10   57226.220      277.580   ops/s   1304       68.16%
Compress.ZstdLevelBenchMark.zstdCompress          1    8192  thrpt   10   34586.557      302.868   ops/s   3091       62.27%
Compress.ZstdLevelBenchMark.zstdCompress          2     256  thrpt   10  160513.122      1055.204  ops/s   217        15.23%
Compress.ZstdLevelBenchMark.zstdCompress          2     512  thrpt   10  139264.767      1479.608  ops/s   311        39.26%
Compress.ZstdLevelBenchMark.zstdCompress          2    1024  thrpt   10  114781.873      1121.017  ops/s   411        59.86%
Compress.ZstdLevelBenchMark.zstdCompress          2    2048  thrpt   10   82449.922      681.039  ops/s   704         65.63%
Compress.ZstdLevelBenchMark.zstdCompress          2    4096  thrpt   10   55644.063      749.733  ops/s   1329        67.55%
Compress.ZstdLevelBenchMark.zstdCompress          2    8192  thrpt   10   34113.274      243.294  ops/s   3035        62.95%
Compress.ZstdLevelBenchMark.zstdCompress          3     256  thrpt   10  153682.210      1801.146  ops/s   210     	17.97%
Compress.ZstdLevelBenchMark.zstdCompress          3     512  thrpt   10  132749.271      1685.915  ops/s   311     	39.26%
Compress.ZstdLevelBenchMark.zstdCompress          3    1024  thrpt   10  109401.965      1214.010  ops/s   425	    58.50%
Compress.ZstdLevelBenchMark.zstdCompress          3    2048  thrpt   10   81395.267      1234.244  ops/s   732	    64.26%
Compress.ZstdLevelBenchMark.zstdCompress          3    4096  thrpt   10   52683.335       485.770  ops/s   1295	    68.38%
Compress.ZstdLevelBenchMark.zstdCompress          3    8192  thrpt   10   29897.221       513.773  ops/s   2992	    63.48%
Compress.ZstdLevelBenchMark.zstdCompress          4     256  thrpt   10  133600.466      1974.326  ops/s   206	    19.53%
Compress.ZstdLevelBenchMark.zstdCompress          4     512  thrpt   10  112088.964      4874.136  ops/s   307	    40.04%
Compress.ZstdLevelBenchMark.zstdCompress          4    1024  thrpt   10   88978.838      1492.176  ops/s   420	    58.98%
Compress.ZstdLevelBenchMark.zstdCompress          4    2048  thrpt   10   62289.020      1805.546  ops/s   724	    64.65%
Compress.ZstdLevelBenchMark.zstdCompress          4    4096  thrpt   10   38117.885       496.076  ops/s   1288	    68.55%
Compress.ZstdLevelBenchMark.zstdCompress          4    8192  thrpt   10   18699.637       176.416  ops/s   2948	    64.01%
Compress.ZstdLevelBenchMark.zstdCompress          5     256  thrpt   10  124519.999      3317.794  ops/s   204	    20.31%
Compress.ZstdLevelBenchMark.zstdCompress          5     512  thrpt   10  101343.380      1078.978  ops/s   293	    42.77%
Compress.ZstdLevelBenchMark.zstdCompress          5    1024  thrpt   10   75868.554       405.721  ops/s   420	    58.98%
Compress.ZstdLevelBenchMark.zstdCompress          5    2048  thrpt   10   50464.417       497.478  ops/s   714	    65.14%
Compress.ZstdLevelBenchMark.zstdCompress          5    4096  thrpt   10   29152.233       320.349  ops/s   1243	    69.65%
Compress.ZstdLevelBenchMark.zstdCompress          5    8192  thrpt   10   13744.149        76.323  ops/s   2885	    64.78%
Compress.ZstdLevelBenchMark.zstdCompress          6     256  thrpt   10  122500.287      1030.539  ops/s   204	    20.31%
Compress.ZstdLevelBenchMark.zstdCompress          6     512  thrpt   10   96711.036       965.084  ops/s   293	    42.77%
Compress.ZstdLevelBenchMark.zstdCompress          6    1024  thrpt   10   63150.268      2078.721  ops/s   420	    58.98%
Compress.ZstdLevelBenchMark.zstdCompress          6    2048  thrpt   10   38628.981       669.073  ops/s   714	    65.14%
Compress.ZstdLevelBenchMark.zstdCompress          6    4096  thrpt   10   19968.703       271.908  ops/s   1240	    69.73%
Compress.ZstdLevelBenchMark.zstdCompress          6    8192  thrpt   10    9931.221        79.205  ops/s   2886	    64.77%
Compress.ZstdLevelBenchMark.zstdUncompress        1     256  thrpt   10  582084.385     7778.949  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        1     512  thrpt   10  480672.486     2697.970  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        1    1024  thrpt   10  361403.668     3443.690  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        1    2048  thrpt   10  189317.885     1558.473  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        1    4096  thrpt   10  134702.624     1711.808  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        1    8192  thrpt   10   83840.509     9960.244  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2     256  thrpt   10  580312.371     6809.382  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2     512  thrpt   10  487137.103     3394.255  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2    1024  thrpt   10  364243.783     3039.647  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2    2048  thrpt   10  266417.312     11819.619  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2    4096  thrpt   10  128809.854     5231.345  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        2    8192  thrpt   10   83274.771     6297.771  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3     256  thrpt   10  565699.876     12919.602  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3     512  thrpt   10  484201.159      8496.238  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3    1024  thrpt   10  351868.427      6034.207  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3    2048  thrpt   10  264361.085     13951.503  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3    4096  thrpt   10  134640.265      1068.485  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        3    8192  thrpt   10   82746.561      8866.313  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4     256  thrpt   10  562818.616      5036.532  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4     512  thrpt   10  461458.897      5798.547  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4    1024  thrpt   10  349583.096      2818.184  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4    2048  thrpt   10  265175.187     10967.808  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4    4096  thrpt   10  132574.256      1409.214  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        4    8192  thrpt   10   84652.976      6150.251  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5     256  thrpt   10  501616.311      5596.259  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5     512  thrpt   10  419099.670      1799.619  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5    1024  thrpt   10  353087.876      6711.421  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5    2048  thrpt   10  264624.591      2530.357  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5    4096  thrpt   10  133355.669      1524.036  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        5    8192  thrpt   10   84665.619      6465.654  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6     256  thrpt   10  494291.843      7102.794  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6     512  thrpt   10  418350.498      2466.316  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6    1024  thrpt   10  355302.788      2142.318  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6    2048  thrpt   10  265958.624     11417.774  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6    4096  thrpt   10  131764.090      6460.557  ops/s
Compress.ZstdLevelBenchMark.zstdUncompress        6    8192  thrpt   10   85139.016      5582.793  ops/s
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73