Redis集群模式说明
Redis支持以集群模式运行，在该模式下，Redis将所有存储空间分为16384个哈希槽，集群中的每个Master节点负责N个哈希槽（一个数据分片），当用户写入一条数据时，Redis计算其哈希槽，然后将数据写在负责该哈希槽的节点上。且每个Master节点可以添加一个或多个Slave节点，当某个Master节点不可用时，其Slave节点自动代替Master节点继续工作
由此可见，在Redis集群模式下，我们可获得更高的吞吐量，和一定程度的可用性。需要注意的是，在集群模式下，Redis仍不能保证数据零丢失
基本原理
本文以Redis官方镜像6.0.8版本作为示例，创建一个6个节点的Redis集群，其中3个Master节点，3个Slave节点。因为每个节点有自己的状态和标识，所以使用Statefulset来创建Pod，此外需要为每个节点挂载一个云盘，用以持久化节点数据。建议选择较新Redis版本，如果Redis版本低于5.0，则初始化集群所用的命令可能会有所不同
1、创建6个副本的集群（三主三从）
[root@kubernetes-master ~]# cat redis_cluster.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-cluster
labels:
middleware: redis-cluster
app: redis-cluster
spec:
serviceName: redis-cluster
replicas: 6
selector:
matchLabels:
app: redis-cluster
middleware: redis-cluster
template:
metadata:
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9121"
labels:
middleware: redis-cluster
app: redis-cluster
spec:
containers:
- name: redis-cluster
image: redis:5.0.7
imagePullPolicy: IfNotPresent
command:
- /bin/sh
- -c
- |
set -x
redis-server /conf/redis.conf --cluster-announce-ip "$(MY_POD_IP)"
env:
- name: MY_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
ports:
- containerPort: 6379
name: client
volumeMounts:
- name: conf
mountPath: /conf
- name: redis-exporter
image: bitnami/redis-exporter:latest
args: ["--redis.password=12345678"]
ports:
- containerPort: 9121
initContainers:
- name: init-sysctl
image: busybox:1.28
imagePullPolicy: IfNotPresent
command:
- /bin/sh
- -c
- sysctl -w net.core.somaxconn=4096
- chmod 755 /conf/cluster.sh
securityContext:
privileged: true
volumes:
- name: conf
configMap:
name: redis-cluster
defaultMode: 0755
---
apiVersion: v1
kind: Service
metadata:
name: redis-cluster
labels:
middleware: redis-cluster
app: redis-cluster
spec:
type: NodePort
ports:
- protocol: TCP
port: 6379
name: client
selector:
app: redis-cluster
middleware: redis-cluster
---
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-cluster
labels:
middleware: redis-cluster
app: redis-cluster
data:
redis.conf: |+
port 6379
daemonize no
dir /data
cluster-enabled yes
cluster-config-file /data/nodes.conf
cluster-node-timeout 15000
appendonly yes
#masterauth foo123bared
#requirepass foo123bared
requirepass 12345678
#tcp-backlog 4095
maxmemory 500mb
maxmemory-policy volatile-lru
cluster-require-full-coverage no
cluster-migration-barrier 1
#protected-mode no
cluster.sh: |+
#! /bin/sh
set -x
echo yes | redis-cli -a 12345678 --cluster create --cluster-replicas 1 $1
2、查看是否创建成功
[root@kubernetes-master ~]# kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/redis-cluster-0 2/2 Running 0 16h
pod/redis-cluster-1 2/2 Running 0 16h
pod/redis-cluster-2 2/2 Running 0 16h
pod/redis-cluster-3 2/2 Running 0 16h
pod/redis-cluster-4 2/2 Running 0 16h
pod/redis-cluster-5 2/2 Running 0 16h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis-cluster NodePort 10.100.188.235 6379:31427/TCP 16h
说明：出现以上信息说明集群正常运行
3、初始化集群，目前Redis还不支持以hostname方式初始化集群，所以先获取每个节点的IP地址
[root@kubernetes-master ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
redis-cluster-0 2/2 Running 0 16h 10.244.129.115 kubernetes-node1
redis-cluster-1 2/2 Running 0 16h 10.244.22.81 kubernetes-node2
redis-cluster-2 2/2 Running 0 16h 10.244.129.116 kubernetes-node1
redis-cluster-3 2/2 Running 0 16h 10.244.22.82 kubernetes-node2
redis-cluster-4 2/2 Running 0 16h 10.244.129.114 kubernetes-node1
redis-cluster-5 2/2 Running 0 16h 10.244.22.83 kubernetes-node2
3.1、登录到其中一个Redis节点
[root@kubernetes-master ~]# kubectl exec -it redis-cluster-1 -- bash
root@redis-cluster-1:/data#
3.2、执行初始化命令，共有6个节点，当选项–cluster-replicas指定为1时，表示为每个Master节点分配一个Slave节点，这样集群中刚好3个Master节点和3个Slave节点
redis-cli -a 123456 --cluster create 10.244.129.115:6379 10.244.22.81:6379 10.244.129.116:6379 10.244.22.82:6379 10.244.129.114:6379 10.244.22.83:6379 --cluster-replicas 1
3.3、输出类似以下信息表示初始化成功
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
4、使用Redis，现在进入集群中的任意一个Pod中都可以访问Redis服务，前面我们创建了一个headless类型的Service，kubernetes集群会为该服务分配一个DNS A记录，格式为my-svc.my-namespace.svc.cluster-domain.example，每次访问该服务名时，将随机解析到其中一个Redis节点上。
我们进入redis-cluster-1节点中，尝试登录到Redis。
root@redis-cluster-1:/data# redis-cli -a 123456 -c -h redis-cluster.default.svc.cluster.local -p 6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
redis-cluster.default.svc.cluster.local:6379>
redis-cluster.default.svc.cluster.local:6379>
redis-cluster.default.svc.cluster.local:6379> set k8 v8
-> Redirected to slot [8331] located at 10.244.22.81:6379
OK
10.244.22.81:6379> get k8
"v8"
5、可以参考此文档进行集群的扩缩容操作
https://help.aliyun.com/document_detail/185957.html