ES集群安装

1，集群规划

ES冷热数据分离

• 热数据节点：node.data: true, node.attr.box_type: hot
• 冷数据节点：node.data: true, node.attr.box_type: warm
• 参考 https://www.cnblogs.com/hapjin/p/11314492.html
  

2，配置集群，并启动

• hadoop01 : es master

[wang@hadoop01 elasticsearch-7.16.3]$ cat config/elasticsearch.yml
cluster.name: es7
node.name: 192.168.56.172
path.data: /opt/elasticsearch-7.16.3/data/
network.host: 0.0.0.0

# 是不是有资格成为主节点
node.master: true 
node.data: false
http.port: 29200
transport.port: 29300

#集群间，主机通信地址
discovery.seed_hosts: ["192.168.56.172:29300","192.168.56.171:29300"]
# 与node.master 配置的ip或 主机名保持一致 (当node.master使用主机名时, 此处需填主机名, 写ip可能无效）
cluster.initial_master_nodes: ["192.168.56.172"]

cluster.routing.allocation.same_shard.host: true 
cluster.max_shards_per_node: 10000
script.max_compilations_rate: 5000/5m
#xpack.security.enabled: true 
#xpack.security.transport.ssl.enabled: true 
#xpack.security.transport.ssl.verification_mode: certificate
#xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 
#xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
xpack.monitoring.collection.enabled: true
cluster.fault_detection.follower_check.interval: 5s
cluster.fault_detection.follower_check.timeout: 30s
cluster.fault_detection.follower_check.retry_count: 5
cluster.fault_detection.leader_check.interval: 5s
cluster.fault_detection.leader_check.timeout: 30s
cluster.fault_detection.leader_check.retry_count: 5
cluster.follower_lag.timeout: 120s
cluster.join.timeout: 90s
cluster.publish.timeout: 60s
action.destructive_requires_name: false
transport.connect_timeout: 60s
transport.compress: true
cluster.info.update.interval: 300s
cluster.info.update.timeout: 60s
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

• hadoop02 : es data

[wang@hadoop02 elasticsearch-7.16.3]$ cat config/elasticsearch.yml
cluster.name: es7
node.name: 192.168.56.171
path.data: /opt/elasticsearch-7.16.3/data/
network.host: 0.0.0.0

# 是不是有资格成为主节点
node.master: false
node.data: true
http.port: 29200
transport.port: 29300

discovery.seed_hosts: ["192.168.56.172:29300","192.168.56.171:29300"]
cluster.initial_master_nodes: ["192.168.56.172"]

cluster.routing.allocation.same_shard.host: true 
cluster.max_shards_per_node: 10000
script.max_compilations_rate: 5000/5m
#xpack.security.enabled: true 
#xpack.security.transport.ssl.enabled: true 
#xpack.security.transport.ssl.verification_mode: certificate
#xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 
#xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

启动集群

cat >> /etc/security/limits.conf <<EOF
* soft nofile 262144
* hard nofile 262144
* soft nproc 65536  
* hard nproc 65536 
EOF
echo 'session required  pam_limits.so' >>  /etc/pam.d/login 

echo 'vm.max_map_count=262144' >>  /etc/sysctl.conf 

##使得配置生效( ctl + c, 退出重新登录终端)
sysctl -p

## 启动es集群
[wang@hadoop01 elasticsearch-7.16.3]$ ./bin/elasticsearch -d
[wang@hadoop02 elasticsearch-7.16.3]$ ./bin/elasticsearch -d

## 查看es日志
tail -f logs/es7.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

3，设置访问密码

• 解开上面elasticsearch.yml 的注释，完成下面操作后，重启服务即可通过密码访问es集群
• 密码文件路径：config/elastic-certificates.p12

# 任意es节点上执行
[wang@hadoop01 ~]$ bin/elasticsearch-certutil ca
提示输入路径跟密码，直接回车，生成一个文件：elastic-stack-ca.p12

# 在生成证书的机器上执行
[wang@hadoop01 ~]$ bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 
 提示输入刚才设置的密码，路径,生成一个文件：elastic-certificates.p12 (es节点通信的凭证了), 拷贝此文件到其他es节点
[wang@hadoop01 ~]$ mv elastic-certificates.p12 config/
[wang@hadoop01 ~]$ scp elastic-certificates.p12 hadoop02:/opt/ elasticsearch-7.1.1/config/


# 在各个节点上给证书添加密码 ，密码需要和创建证书的密码一致 (没设置密码就略过）
[wang@hadoop01 ~]$ echo 'wang@123'| bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password 
[wang@hadoop01 ~]$ echo 'wang@123'| bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
[wang@hadoop01 ~]$ scp elasticsearch.keystore hadoop02:/opt/elasticsearch-7.1.1/config/
#将elasticsearch.keystore拷贝到各个节点
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

重启集群，验证访问

#无设置密码
[wang@hadoop02 elasticsearch-7.16.3]$ curl -u 'elastic:passwd' localhost:29200/_cat/health
1669260365 03:26:05 es7 green 2 1 4 4 0 0 0 0 - 100.0%

#设置密码后
# 任意节点执行（初次设置密码才需要执行）
[wang@hadoop02 elasticsearch-7.16.3]$ bin/elasticsearch-setup-passwords interactive
根据提示输入es各用户的密码

[wang@hadoop02 elasticsearch-7.16.3]$ curl -u 'elastic:passwd' localhost:29200/_cat/nodes
192.168.56.172 30 94 34 4.36 4.29 4.18 ilmr       * 192.168.56.172
192.168.56.171 32 98  0 0.05 0.04 0.09 cdfhilrstw - 192.168.56.171
1
2
3
4
5
6
7
8
9
10
11
12