【2022】二进制方式部署etcd高可用集群

安装部分

二进制方式部署ETCD高可用集群

• 环境准备：
  

• 在etcd-01主机上下载安装包，并解压至/usr/local/bin目录下

[root@etcd-01 ~]# wget https://github.com/etcd-io/etcd/releases/download/v3.5.5/etcd-v3.5.5-linux-amd64.tar.gz
[root@etcd-01 ~]# tar zxf etcd-v3.5.5-linux-amd64.tar.gz --strip-components=1 -C /usr/local/bin etcd-v3.5.5-linux-amd64/etcd{,ctl}

• 操作成功即为安装成功（也可以查看都有什么内容）；可以查看版本

[root@etcd-01 ~]# etcdctl version
etcdctl version: 3.5.5
API version: 3.5

• 将组件分发至其他节点（因为内容一样，所以没必要在下载一次），注意如果使用节点名要提前做好hosts解析

[root@etcd-01 ~]# scp /usr/local/bin/* etcd-02:/usr/local/bin/
[root@etcd-01 ~]# scp /usr/local/bin/* etcd-03:/usr/local/bin/

生成证书

• 本例使用cfssl工具生成证书，所以下载cfssl工具

[root@etcd-01 ~]# wget "https://github.com/cloudflare/cfssl/releases/download/v1.6.2/cfssl_1.6.2_linux_amd64" -O /usr/local/bin/cfssl
[root@etcd-01 ~]# wget "https://github.com/cloudflare/cfssl/releases/download/v1.6.2/cfssljson_1.6.2_linux_amd64" -O /usr/local/bin/cfssljson
[root@etcd-01 ~]# chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson 
[root@etcd-01 ~]# cfssl version
Version: 1.6.2
Runtime: go1.18

• 创建一个证书存放目录，这一步需要所有节点都创建

[root@etcd-01 ~]# mkdir -p /etc/etcd/ssl
[root@etcd-02 ~]# mkdir -p /etc/etcd/ssl
[root@etcd-03 ~]# mkdir -p /etc/etcd/ssl

• 创建一个用于证书的json文件

[root@etcd-01 ~]# vim ca-config.json
{
  "signing": {
    "default": {
      "expiry": "876000h"
    },
    "profiles": {
      "etcd": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "876000h"
      }
    }
  }
}

• 创建一个用于生成CA证书和key的json文件

[root@etcd-01 ~]# cat etcd-ca-csr.json 
{
  "CN": "etcd",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "etcd",
      "OU": "Etcd Security"
    }
  ],
  "ca": {
    "expiry": "876000h"
  }
}

• 生成CA证书和证书的key

[root@etcd-01 ~]# cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca
[root@etcd-01 ~]# ls /etc/etcd/ssl/
etcd-ca.csr  etcd-ca-key.pem  etcd-ca.pem

• 创建用于etcd服务证书的json

[root@etcd-01 ~]# cat etcd-csr.json 
{
  "CN": "etcd",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "etcd",
      "OU": "Etcd Security"
    }
  ]
}

• 生成ETCD服务证书

[root@etcd-01 ~]# cfssl gencert -ca=/etc/etcd/ssl/etcd-ca.pem -ca-key=/etc/etcd/ssl/etcd-ca-key.pem -config=ca-config.json -hostname=127.0.0.1,etcd-01,etcd-02,etcd-03,192.168.10.3,192.168.10.4,192.168.10.5 -profile=etcd etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd
[root@etcd-01 ~]# ls /etc/etcd/ssl/
etcd-ca.csr  etcd-ca-key.pem  etcd-ca.pem  etcd.csr  etcd-key.pem  etcd.pem

• 将证书复制到其他节点

[root@etcd-01 ssl]# for FILE in etcd-ca-key.pem  etcd-ca.pem  etcd-key.pem  etcd.pem; do        scp /etc/etcd/ssl/${FILE} etcd-02:/etc/etcd/ssl/${FILE};      done
[root@etcd-01 ssl]# for FILE in etcd-ca-key.pem  etcd-ca.pem  etcd-key.pem  etcd.pem; do        scp /etc/etcd/ssl/${FILE} etcd-03:/etc/etcd/ssl/${FILE};      done

ETCD配置

• 创建etcd配置文件

[root@etcd-01 ~]# cat /etc/etcd/etcd.config.yml 
name: 'etcd-01'
data-dir: /var/lib/etcd
wal-dir: /var/lib/etcd/wal
snapshot-count: 5000
heartbeat-interval: 100
election-timeout: 1000
quota-backend-bytes: 0
listen-peer-urls: 'https://192.168.10.3:2380'
listen-client-urls: 'https://192.168.10.3:2379,http://127.0.0.1:2379'
max-snapshots: 3
max-wals: 5
cors:
initial-advertise-peer-urls: 'https://192.168.10.3:2380'
advertise-client-urls: 'https://192.168.10.3:2379'
discovery:
discovery-fallback: 'proxy'
discovery-proxy:
discovery-srv:
initial-cluster: 'etcd-01=https://192.168.10.3:2380,etcd-02=https://192.168.10.4:2380,etcd-03=https://192.168.10.5:2380'
initial-cluster-token: 'etcd-cluster'
initial-cluster-state: 'new'
strict-reconfig-check: false
enable-v2: true
enable-pprof: true
proxy: 'off'
proxy-failure-wait: 5000
proxy-refresh-interval: 30000
proxy-dial-timeout: 1000
proxy-write-timeout: 5000
proxy-read-timeout: 0
client-transport-security:
  cert-file: '/etc/etcd/ssl/etcd.pem'
  key-file: '/etc/etcd/ssl/etcd-key.pem'
  client-cert-auth: true
  trusted-ca-file: '/etc/etcd/ssl/etcd-ca.pem'
  auto-tls: true
peer-transport-security:
  cert-file: '/etc/etcd/ssl/etcd.pem'
  key-file: '/etc/etcd/ssl/etcd-key.pem'
  peer-client-cert-auth: true
  trusted-ca-file: '/etc/etcd/ssl/etcd-ca.pem'
  auto-tls: true
debug: false
log-package-levels:
log-outputs: [default]
force-new-cluster: false

• 其他两个节点配置文件只需要修改各自的节点名和IP即可

• 创建service文件

[root@etcd-01 ~]# cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Service
Documentation=https://coreos.com/etcd/docs/latest/
After=network.target

[Service]
Type=notify
ExecStart=/usr/local/bin/etcd --config-file=/etc/etcd/etcd.config.yml
Restart=on-failure
RestartSec=10
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
Alias=etcd3.service

• 其他两个节点配置一样，复制即可
• 所有启动etcd服务

systemctl daemon-reload
systemctl enable --now etcd

• 查看ETCD状态

[root@etcd-01 ~]# export ETCDCTL_API=3
[root@etcd-01 ~]# etcdctl --endpoints="192.168.10.5:2379,192.168.10.4:2379,192.168.10.3:2379" --cacert=/etc/etcd/ssl/etcd-ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem  endpoint status --write-out=table
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|     ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 192.168.10.5:2379 | 4ee1cc1544fd02a3 |   3.5.5 |   20 kB |     false |      false |         2 |          9 |                  9 |        |
| 192.168.10.4:2379 | 2af255134b508f21 |   3.5.5 |   20 kB |     false |      false |         2 |          9 |                  9 |        |
| 192.168.10.3:2379 | 86ef4da6f07b0d20 |   3.5.5 |   20 kB |      true |      false |         2 |          9 |                  9 |        |
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

到这里二进制部署etcd集群就结束了！