mongo+ycsb性能测试及线程数分析

一、mongo安装及压测ycsb安装

mongodb下载官网
https://www.mongodb.com/

mongo下载完对应tar包后，通过tar -xvf命令解压至/usr/local下

mv tar包名称 mongodb
cd /mongodb
mkdir  /data/db
mkdir  /data/log
vi mongodb.conf
#新建配置文件，配置如下
dbpath=/usr/local/mongodb/data/db
logpath=/usr/local/mongodb/data/log/mongodb.log
logappend=true
port=27017
auth=false
fork=true
bind_ip=0.0.0.0
1
2
3
4
5
6
7
8
9
10
11
12
13

启动mongo

/usr/local/mongod/bin/mongod --config /usr/local/mongod/conf/mongodb.conf 
1

ycsb0.17下载地址

https://download.csdn.net/download/dong0928/12285659?utm_source=iteye_new

二、ycsb加载与负载测试

#官方文档 Now you are ready to run! First, use the asynchronous driver to load the data:

解压后ycsb后

cd ycsb/workloads
1

不同的工作负载模式不同对应增删改查的操作所占百分比不同
ycsb的workloads目录下保存了6种不同的workload类型，代表了不同的压测负载类型，详细的介绍列在下面：
workloada：混合了50%的读和50%的写； workloadb：Read mostly
workload，混合了95%的读和5%的写，该workload侧重于测试集群的读能力；
workloadc：Readonly，100%只读
workloadd：Read latest workload，插入数据，接着就读取这些新插入的数据
workloade：Short ranges，短范围scan，不同于随机读，每个测试线程都会去scan一段数据
workloadf：Read-modiy-wirte，读改写，客户端读出一个记录，修改它并将被修改的记录返回

mv workloada back_workloada
vi workloada
#配置如下：

##连接的url及db地址
mongodb.url=mongodb://10.37.0.101/
mongodb.database=ycsb

##初始加载数据数，第一次可以100000次少点便于验证
recordcount=100000000

##操作次数
operationcount=1000000

##（要使用的工作负载类）
workload=site.ycsb.workloads.CoreWorkload

##（默认值是1，应该读取读取所有字段（true），只读取一个（false））
readallfields=true

##查询操作所占百分比
readproportion=0.5

##更新操作所占百分比
updateproportion=0.5

##扫描操作所占百分比
scanproportion=0

##写入操作所占百分比
insertproportion=0

requestdistribution=zipfian

threadcount=100

fieldlength=200

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

首先进行加载数据
./bin/ycsb load mongodb -s -P workloads/workloada > outputLoad.txt
Then, run the workload:
./bin/ycsb run mongodb -s -P workloads/workloada > outputRun.txt

避免重复写入主键冲突
-p mongodb.upsert=true
线程数
-p threads 10
1
2
3
4

mongo库已插入千万条数据

测试结果分析：

#花费时间3470.464s
[OVERALL], RunTime(ms), 3470464
#数据库每秒操作次数为2881.4590786707486
[OVERALL], Throughput(ops/sec), 2881.4590786707486
[TOTAL_GCS_PS_Scavenge], Count, 113
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 16027
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.4618114465385608
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 113
[TOTAL_GC_TIME], Time(ms), 16027
[TOTAL_GC_TIME_%], Time(%), 0.4618114465385608
#清理操作操作数、单平均延迟、最小延迟、最大延迟95/99百分位
[CLEANUP], Operations, 100
[CLEANUP], AverageLatency(us), 3373.35
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 336895
[CLEANUP], 95thPercentileLatency(us), 14
[CLEANUP], 99thPercentileLatency(us), 90
#插入操作操作数、平均延迟、最小延迟、最大延迟，95/99百分位
[INSERT], Operations, 10000000
[INSERT], AverageLatency(us), 28241.2937591
[INSERT], MinLatency(us), 120
[INSERT], MaxLatency(us), 298057727
[INSERT], 95thPercentileLatency(us), 58623
[INSERT], 99thPercentileLatency(us), 391423
[INSERT], Return=OK, 10000000
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

load完后，现在进行run测试

./bin/ycsb run mongodb -P workloads/workloada > run.log
vi run.log
1
2

结果如下：

[OVERALL], RunTime(ms), 8125
[OVERALL], Throughput(ops/sec), 12307.692307692309
[TOTAL_GCS_PS_Scavenge], Count, 6
[TOTAL_GC_TIME], Time(ms), 224
[TOTAL_GC_TIME_%], Time(%), 2.756923076923077
[READ], Operations, 49931
[READ], AverageLatency(us), 5412.5758146241815
[READ], MinLatency(us), 192
[READ], MaxLatency(us), 412927
[READ], 95thPercentileLatency(us), 15295
[READ], 99thPercentileLatency(us), 89919
[READ], Return=OK, 49931
[CLEANUP], Operations, 100
[CLEANUP], AverageLatency(us), 1566.47
[CLEANUP], MinLatency(us), 0
[CLEANUP], MaxLatency(us), 156287
[CLEANUP], 95thPercentileLatency(us), 3
[CLEANUP], 99thPercentileLatency(us), 298
[UPDATE], Operations, 50069
[UPDATE], AverageLatency(us), 4083.3765204018455
[UPDATE], MinLatency(us), 223
[UPDATE], MaxLatency(us), 420095
[UPDATE], 95thPercentileLatency(us), 11199
[UPDATE], 99thPercentileLatency(us), 61503
[UPDATE], Return=OK, 50069
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

可以看出load进行插入的操作，run进行read、update的操作，并且按照配置文件的0.5%即5+5=10万条查询及更新吞吐量为12307

这时候我们去改变线程数量再次以此配置进行测试

最佳线程数目 = （（线程等待时间+线程CPU时间）/线程CPU时间 ）* CPU数目

./bin/ycsb run mongodb -P workloads/workloada -threads 100
1

[OVERALL], RunTime(ms), 264990
[OVERALL], Throughput(ops/sec), 377.3727310464546
[TOTAL_GCS_PS_Scavenge], Count, 6
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 148
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.05585116419487528
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 6
[TOTAL_GC_TIME], Time(ms), 148
[TOTAL_GC_TIME_%], Time(%), 0.05585116419487528
[READ], Operations, 50167
[READ], AverageLatency(us), 8718.358542468157
[READ], MinLatency(us), 246
[READ], MaxLatency(us), 5517311
[READ], 95thPercentileLatency(us), 12839
[READ], 99thPercentileLatency(us), 40255
[READ], Return=OK, 50167
[CLEANUP], Operations, 100
[CLEANUP], AverageLatency(us), 220.4
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 21711
[CLEANUP], 95thPercentileLatency(us), 5
[CLEANUP], 99thPercentileLatency(us), 154
[UPDATE], Operations, 49833
[UPDATE], AverageLatency(us), 11637.081632653062
[UPDATE], MinLatency(us), 259
[UPDATE], MaxLatency(us), 5713919
[UPDATE], 95thPercentileLatency(us), 8879
[UPDATE], 99thPercentileLatency(us), 42143
[UPDATE], Return=OK, 49833
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

线程数为3时3380

./bin/ycsb run mongodb -P workloads/workloada -threads 3
1

[OVERALL], RunTime(ms), 29581
[OVERALL], Throughput(ops/sec), 3380.548324938305
[TOTAL_GCS_PS_Scavenge], Count, 4
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 60
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.2028328994962983
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 4
[TOTAL_GC_TIME], Time(ms), 60
[TOTAL_GC_TIME_%], Time(%), 0.2028328994962983
[READ], Operations, 50235
[READ], AverageLatency(us), 590.9062605752961
[READ], MinLatency(us), 188
[READ], MaxLatency(us), 834559
[READ], 95thPercentileLatency(us), 617
[READ], 99thPercentileLatency(us), 1039
[READ], Return=OK, 50235
[CLEANUP], Operations, 3
[CLEANUP], AverageLatency(us), 329483.0
[CLEANUP], MinLatency(us), 4
[CLEANUP], MaxLatency(us), 988671
[CLEANUP], 95thPercentileLatency(us), 988671
[CLEANUP], 99thPercentileLatency(us), 988671
[UPDATE], Operations, 49765
[UPDATE], AverageLatency(us), 626.2020697277203
[UPDATE], MinLatency(us), 217
[UPDATE], MaxLatency(us), 1181695
[UPDATE], 95thPercentileLatency(us), 634
[UPDATE], 99thPercentileLatency(us), 1002
[UPDATE], Return=OK, 49765
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

线程数为4时5535

./bin/ycsb run mongodb -P workloads/workloada -threads 4
1

[OVERALL], RunTime(ms), 18065
[OVERALL], Throughput(ops/sec), 5535.566011624689
[TOTAL_GCS_PS_Scavenge], Count, 4
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 66
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.36534735676722946
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 4
[TOTAL_GC_TIME], Time(ms), 66
[TOTAL_GC_TIME_%], Time(%), 0.36534735676722946
[READ], Operations, 49992
[READ], AverageLatency(us), 460.93799007841255
[READ], MinLatency(us), 185
[READ], MaxLatency(us), 268287
[READ], 95thPercentileLatency(us), 560
[READ], 99thPercentileLatency(us), 926
[READ], Return=OK, 49992
[CLEANUP], Operations, 4
[CLEANUP], AverageLatency(us), 1521.75
[CLEANUP], MinLatency(us), 5
[CLEANUP], MaxLatency(us), 6039
[CLEANUP], 95thPercentileLatency(us), 6039
[CLEANUP], 99thPercentileLatency(us), 6039
[UPDATE], Operations, 50008
[UPDATE], AverageLatency(us), 496.3823788193889
[UPDATE], MinLatency(us), 216
[UPDATE], MaxLatency(us), 828927
[UPDATE], 95thPercentileLatency(us), 582
[UPDATE], 99thPercentileLatency(us), 953
[UPDATE], Return=OK, 50008
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

那么线程设置为5呢,2277

./bin/ycsb run mongodb -P workloads/workloada -threads 5
1

[OVERALL], RunTime(ms), 43899
[OVERALL], Throughput(ops/sec), 2277.956217681496
[TOTAL_GCS_PS_Scavenge], Count, 4
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 184
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.4191439440533953
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 4
[TOTAL_GC_TIME], Time(ms), 184
[TOTAL_GC_TIME_%], Time(%), 0.4191439440533953
[READ], Operations, 49986
[READ], AverageLatency(us), 1507.791441603649
[READ], MinLatency(us), 185
[READ], MaxLatency(us), 2721791
[READ], 95thPercentileLatency(us), 518
[READ], 99thPercentileLatency(us), 859
[READ], Return=OK, 49986
[CLEANUP], Operations, 5
[CLEANUP], AverageLatency(us), 653.6
[CLEANUP], MinLatency(us), 3
[CLEANUP], MaxLatency(us), 3235
[CLEANUP], 95thPercentileLatency(us), 3235
[CLEANUP], 99thPercentileLatency(us), 3235
[UPDATE], Operations, 50014
[UPDATE], AverageLatency(us), 2588.507597872596
[UPDATE], MinLatency(us), 210
[UPDATE], MaxLatency(us), 2373631
[UPDATE], 95thPercentileLatency(us), 529
[UPDATE], 99thPercentileLatency(us), 885
[UPDATE], Return=OK, 50014
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

二分法查找，当然因为是单核低配虚拟机，直接按照2个thread来测就可以了，实际物理机会有明显的线程数量差量变化，会呈现高斯分布，中间有一个最适合的线程数

./bin/ycsb run mongodb -P workloads/workloada -threads 2
1

[OVERALL], RunTime(ms), 32385
[OVERALL], Throughput(ops/sec), 3087.8493129535277
[TOTAL_GCS_PS_Scavenge], Count, 4
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 84
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.25937934228809634
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 4
[TOTAL_GC_TIME], Time(ms), 84
[TOTAL_GC_TIME_%], Time(%), 0.25937934228809634
[READ], Operations, 49916
[READ], AverageLatency(us), 515.6664596522157
[READ], MinLatency(us), 172
[READ], MaxLatency(us), 694271
[READ], 95thPercentileLatency(us), 523
[READ], 99thPercentileLatency(us), 945
[READ], Return=OK, 49916
[CLEANUP], Operations, 2
[CLEANUP], AverageLatency(us), 1568.0
[CLEANUP], MinLatency(us), 21
[CLEANUP], MaxLatency(us), 3115
[CLEANUP], 95thPercentileLatency(us), 3115
[CLEANUP], 99thPercentileLatency(us), 3115
[UPDATE], Operations, 50084
[UPDATE], AverageLatency(us), 665.4541370497564
[UPDATE], MinLatency(us), 203
[UPDATE], MaxLatency(us), 1405951
[UPDATE], 95thPercentileLatency(us), 550
[UPDATE], 99thPercentileLatency(us), 955
[UPDATE], Return=OK, 50084
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

线程数为6时

[OVERALL], RunTime(ms), 52916
[OVERALL], Throughput(ops/sec), 1889.7875878751229
[TOTAL_GCS_PS_Scavenge], Count, 4
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 114
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.215435785017764
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 4
[TOTAL_GC_TIME], Time(ms), 114
[TOTAL_GC_TIME_%], Time(%), 0.215435785017764
[READ], Operations, 50322
[READ], AverageLatency(us), 1727.1597313302334
[READ], MinLatency(us), 182
[READ], MaxLatency(us), 2369535
[READ], 95thPercentileLatency(us), 603
[READ], 99thPercentileLatency(us), 1041
[READ], Return=OK, 50322
[CLEANUP], Operations, 6
[CLEANUP], AverageLatency(us), 972.6666666666666
[CLEANUP], MinLatency(us), 3
[CLEANUP], MaxLatency(us), 5783
[CLEANUP], 95thPercentileLatency(us), 5783
[CLEANUP], 99thPercentileLatency(us), 5783
[UPDATE], Operations, 49678
[UPDATE], AverageLatency(us), 3466.9299488707275
[UPDATE], MinLatency(us), 203
[UPDATE], MaxLatency(us), 4734975
[UPDATE], 95thPercentileLatency(us), 618
[UPDATE], 99thPercentileLatency(us), 1067
[UPDATE], Return=OK, 49678
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

结论分析：

当前的单核虚拟机，4个线程可以带来最大的0.5%update，0.5%read，线程数为4时候，吞吐量达到最大

tips：测试数据量一定要够大够多，也就是操作的次数和操作增删改查比例要设置大点，否则会有明显误差