kubeadm下进行etcd数据备份与还原

---

在kubernetes集群中，etcd存储了集群的元数据信息，为防止信息的丢失及还原数据，我们需要对etcd数据进行备份。
本文使用的kubernetes集群采用kubeadm部署的

一、单master的etcd的数据备份与还原

1.从容器中把etcdctl工具复制出来到主机中的/usr/local/bin目录下

docker cp $(docker ps  |  grep -v etcd-mirror | grep -w etcd | awk '{print $1}'):/usr/local/bin/etcdctl /usr/local/bin/
1

2.备份etcd数据

etcdctl --endpoints 127.0.0.1:2379  \
--cert="/etc/kubernetes/pki/etcd/server.crt"  \
--key="/etc/kubernetes/pki/etcd/server.key"  \
--cacert="/etc/kubernetes/pki/etcd/ca.crt"   \
snapshot save etcd_snap_save.db

1
2
3
4
5
6

3.还原etcd数据

####移除配置和删除数据
mv /etc/kubernetes/manifests/etcd.yaml /opt/
rm -rf /var/lib/etcd/

####通过命令恢复
etcdctl snapshot restore etcd_snap_save.db  \
--cacert=/etc/kubernetes/pki/etcd/ca.crt  \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key  \
--data-dir=/var/lib/etcd/   \
--endpoints=https://127.0.0.1:2379

####配置还原
mv /opt/etcd.yaml /etc/kubernetes/manifests/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

二、多master的etcd的数据备份与还原

1.从容器中把etcdctl工具复制出来到主机中的/usr/local/bin目录下

docker cp $(docker ps  |  grep -v etcd-mirror | grep -w etcd | awk '{print $1}'):/usr/local/bin/etcdctl /usr/local/bin/
1

2.备份etcd数据

etcdctl --endpoints 127.0.0.1:2379  \
--cert="/etc/kubernetes/pki/etcd/server.crt"  \
--key="/etc/kubernetes/pki/etcd/server.key"  \
--cacert="/etc/kubernetes/pki/etcd/ca.crt"   \
snapshot save etcd_snap_save.db

1
2
3
4
5
6

3.还原etcd数据

三台master上都需要操作

k8s-master-1

####移除配置和删除数据
mv /etc/kubernetes/manifests/etcd.yaml /opt/
rm -rf /var/lib/etcd/

####通过命令恢复
ETCDCTL_API=3 etcdctl snapshot restore  snap-save.db  \
--cacert=/etc/kubernetes/pki/etcd/ca.crt  \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key  \
--data-dir=/var/lib/etcd/   \
--endpoints=https://127.0.0.1:2379 \
--initial-cluster=k8s-master-1=https://192.168.56.101:2380,k8s-master-2=https://192.168.56.102:2380,k8s-master-3=https://192.168.56.103:2380 \
--name=k8s-master-1 \
--initial-advertise-peer-urls=https://192.168.56.101:2380

####配置还原
mv /opt/etcd.yaml /etc/kubernetes/manifests/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

k8s-master-2

####移除配置和删除数据
mv /etc/kubernetes/manifests/etcd.yaml /opt/
rm -rf /var/lib/etcd/

####通过命令恢复
ETCDCTL_API=3 etcdctl snapshot restore  snap-save.db  \
--cacert=/etc/kubernetes/pki/etcd/ca.crt  \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key  \
--data-dir=/var/lib/etcd/   \
--endpoints=https://127.0.0.1:2379 \
--initial-cluster=k8s-master-1=https://192.168.56.101:2380,k8s-master-2=https://192.168.56.102:2380,k8s-master-3=https://192.168.56.103:2380 \
--name=k8s-master-2 \
--initial-advertise-peer-urls=https://192.168.56.102:2380

####配置还原
mv /opt/etcd.yaml /etc/kubernetes/manifests/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

k8s-master-3

####移除配置和删除数据
mv /etc/kubernetes/manifests/etcd.yaml /opt/
rm -rf /var/lib/etcd/

####通过命令恢复
ETCDCTL_API=3 etcdctl snapshot restore  snap-save.db  \
--cacert=/etc/kubernetes/pki/etcd/ca.crt  \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key  \
--data-dir=/var/lib/etcd/   \
--endpoints=https://127.0.0.1:2379 \
--initial-cluster=k8s-master-1=https://192.168.56.101:2380,k8s-master-2=https://192.168.56.102:2380,k8s-master-3=https://192.168.56.103:2380 \
--name=k8s-master-3 \
--initial-advertise-peer-urls=https://192.168.56.103:2380

####配置还原
mv /opt/etcd.yaml /etc/kubernetes/manifests/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

三、k8s中其他etcd命令汇总

###查看etcd 集群成员列表
etcdctl --endpoints=https://192.168.56.101:2379,https://192.168.56.102:2379,https://192.168.56.103:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key member list

###查看endpoints 状态
etcdctl --endpoints=https://192.168.56.101:2379,https://192.168.56.102:2379,https://192.168.56.103:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key --write-out=table endpoint status

###查看etcd endpoints的健康
etcdctl --endpoints=https://192.168.56.101:2379,https://192.168.56.102:2379,https://192.168.56.103:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key --write-out=table endpoint health

###查看所有的key
ETCDCTL_API=3 etcdctl --endpoints=https://192.168.56.101:2379,https://192.168.56.102:2379,https://192.168.56.103:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key get / --prefix=true --keys-only

###查看某个key具体的内容（会有乱码，因为是序列化后的）
ETCDCTL_API=3 etcdctl --endpoints=https://192.168.56.101:2379,https://192.168.56.102:2379,https://192.168.56.103:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key get /registry/services/specs/liutest/nginx
1
2
3
4
5
6
7
8
9
10
11
12
13
14

以上为kubeadm下进行etcd数据备份的记录。

参考文档：
https://blog.csdn.net/qq_40573385/article/details/123206645
https://blog.51cto.com/lidabai/5224219