-
【云原生 | Kubernetes 系列】----HPA自动伸缩
HPA自动伸缩
HorizontalPodAutoscaler（简称 HPA ） 自动更新工作负载资源（例如 Deployment 或者 StatefulSet）， 目的是自动扩缩工作负载以满足需求。
水平扩缩意味着对增加的负载的响应是部署更多的 Pod。 这与 “垂直（Vertical）” 扩缩不同，对于 Kubernetes， 垂直扩缩意味着将更多资源（例如：内存或 CPU）分配给已经为工作负载运行的 Pod。
如果负载减少，并且 Pod 的数量高于配置的最小值， HorizontalPodAutoscaler 会指示工作负载资源（Deployment、StatefulSet 或其他类似资源）缩减。1. 手动调整副本数
1.1 修改yaml中的replicas个数
root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl get deployments.apps wework-tomcat-app1-deployment -n wework NAME READY UP-TO-DATE AVAILABLE AGE wework-tomcat-app1-deployment 1/1 1 1 5d- 1
- 2
- 3
将replicas改为2
kind: Deployment apiVersion: apps/v1 metadata: labels: app: wework-tomcat-app1-deployment-label name: wework-tomcat-app1-deployment namespace: wework spec: replicas: 2 selector: matchLabels: app: wework-tomcat-app1-selector template: metadata: labels: app: wework-tomcat-app1-selector spec: containers: - name: wework-tomcat-app1-container image: harbor.intra.com/wework/tomcat-app1:v3 ports: - containerPort: 8080 protocol: TCP name: http env: - name: "password" value: "123456" - name: "age" value: "18" resources: limits: cpu: 1 memory: "512Mi" requests: cpu: 500m memory: "512Mi" volumeMounts: - name: wework-images mountPath: /usr/local/nginx/html/webapp/images readOnly: false - name: wework-static mountPath: /usr/local/nginx/html/webapp/static readOnly: false volumes: - name: wework-images nfs: server: 192.168.31.109 path: /data/k8s/wework/images - name: wework-static nfs: server: 192.168.31.104 path: /data/k8s/wework/static --- kind: Service apiVersion: v1 metadata: labels: app: wework-tomcat-app1-service-label name: wework-tomcat-app1-service namespace: wework spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: 8080 nodePort: 30092 selector: app: wework-tomcat-app1-selector- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
生效后deployment的pod数量变成了2个
root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl apply -f tomcat-app1.yaml deployment.apps/wework-tomcat-app1-deployment configured service/wework-tomcat-app1-service unchanged root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl get deployments.apps wework-tomcat-app1-deployment -n wework NAME READY UP-TO-DATE AVAILABLE AGE wework-tomcat-app1-deployment 2/2 2 2 5d- 1
- 2
- 3
- 4
- 5
- 6
1.2 命令行调整
通过命令kubectl scale --replicas=指定deployment中pod的数量
root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl scale --replicas=3 deployments.apps wework-tomcat-app1-deployment -n wework deployment.apps/wework-tomcat-app1-deployment scaled root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl get deployments.apps wework-tomcat-app1-deployment -n wework NAME READY UP-TO-DATE AVAILABLE AGE wework-tomcat-app1-deployment 2/3 3 2 5d root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl get deploy -n wework wework-tomcat-app1-deployment NAME READY UP-TO-DATE AVAILABLE AGE wework-tomcat-app1-deployment 3/3 3 3 5d- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
1.3 通过命令修改指定yaml创建pod数量
通过命令行kubectl scale --replicas=副本个数 -f 配置文件
root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl scale --replicas=4 -f tomcat-app1.yaml deployment.apps/wework-tomcat-app1-deployment scaled root@k8s-master-01:/opt/k8s-data/yaml/wework/tomcat-app1# kubectl get pods -n wework |grep app1 wework-tomcat-app1-deployment-98df9d6f8-2sf4d 1/1 Running 0 3m33s wework-tomcat-app1-deployment-98df9d6f8-d9pn5 1/1 Running 0 4h17m wework-tomcat-app1-deployment-98df9d6f8-fnjlx 1/1 Running 0 8s wework-tomcat-app1-deployment-98df9d6f8-ghw76 1/1 Running 0 11m- 1
- 2
- 3
- 4
- 5
- 6
- 7
1.4 其他
# Scale a replica set named 'foo' to 3 kubectl scale --replicas=3 rs/foo # Scale a resource identified by type and name specified in "foo.yaml" to 3 kubectl scale --replicas=3 -f foo.yaml # If the deployment named mysql's current size is 2, scale mysql to 3 kubectl scale --current-replicas=2 --replicas=3 deployment/mysql # Scale multiple replication controllers kubectl scale --replicas=5 rc/foo rc/bar rc/baz # Scale stateful set named 'web' to 3 kubectl scale --replicas=3 statefulset/web- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
2. 通过HPA调整副本数
K8s从1.1版本开始支持了HPA(Horizontal Pod Autoscaler)控制器.用于基于Pod中CPU/Memory资源利用率进行Pod的自动扩缩容.使用Metrices Server进行数据采集,通过API将数据提供给HPA控制器,实现基于某个资源利用率对Pod进行扩缩容.
2.1 metrics-server
使用HPA需要先部署metrics-server
部署详见: https://github.com/kubernetes-sigs/metrics-server
部署完后就可以看到node的CPU,MEM使用率root@k8s-master-01:~/metrics# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% 192.168.31.101 89m 4% 929Mi 28% 192.168.31.102 212m 10% 2378Mi 73% 192.168.31.103 111m 5% 1897Mi 58% 192.168.31.111 128m 3% 1494Mi 20% 192.168.31.112 131m 6% 2524Mi 78% 192.168.31.113 153m 7% 1091Mi 33%- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
2.2 controller-manager启动参数
参数 含义 –horizontal-pod-autoscaler-sync-period duration 定义Pod水平伸缩时间间隔,默认15秒 –horizontal-pod-autoscaler-cpu-initialization-period Pod初始化时间,在此期间内Pod的Cpu资源指标将不会被采纳,默认5分钟 –horizontal-pod-autoscaler-initial-readiness-delay 用于设置pod准备时间,在此期间内Pod统统被认为未就绪不采集数据,默认为30秒 kube-controller的默认配置位于:/etc/systemd/system/kube-controller-manager.service ,可以按需求进行修改
[Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] ExecStart=/usr/bin/kube-controller-manager \ --bind-address=192.168.31.101 \ --allocate-node-cidrs=true \ --cluster-cidr=172.100.0.0/16 \ --cluster-name=kubernetes \ --cluster-signing-cert-file=/etc/kubernetes/ssl/ca.pem \ --cluster-signing-key-file=/etc/kubernetes/ssl/ca-key.pem \ --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \ --leader-elect=true \ --node-cidr-mask-size=24 \ --root-ca-file=/etc/kubernetes/ssl/ca.pem \ --service-account-private-key-file=/etc/kubernetes/ssl/ca-key.pem \ --service-cluster-ip-range=10.200.0.0/16 \ --use-service-account-credentials=true \ --v=2 Restart=always RestartSec=5 [Install] WantedBy=multi-user.target- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
2.3 创建HPA
2.3.1 通过命令行创建
kubectl autoscale deployment -n wework wework-tomcat-app1 --min=2 --max=5 --cpu-percent=60- 1
2.3.2 通过yml创建
apiVersion: autoscaling/v1 kind: HorizontalPodAutoscaler metadata: namespace: wework name: wework-tomcat-app1-podautoscaler labels: app: wework-tomcat-app1-selector version: v2beta1 spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: wework-tomcat-app1-deployment minReplicas: 2 maxReplicas: 10 targetCPUUtilizationPercentage: 30- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
部署hpa,正常情况下TARGETS是 ??%/30%
root@k8s-master-01:~/metrics# kubectl apply -f hpa.yaml horizontalpodautoscaler.autoscaling/wework-tomcat-app1-podautoscaler created root@k8s-master-01:~/metrics# kubectl get hpa -n wework NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 0%/30% 2 10 4 8m5s- 1
- 2
- 3
- 4
- 5
如果是unknown状态,就是hpa.yaml配置有问题,需要再次核对标签和name等信息.
root@k8s-master-01:~/metrics# kubectl get hpa -n wework NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1 <unknown>/30% 2 10 0 2m14s- 1
- 2
- 3
默认horizontal-pod-autoscaler-cpu-initialization-period 5分钟,所以暂时不会有影响Pod数量
root@k8s-master-01:~/metrics# kubectl get pods -n wework |grep app1 wework-tomcat-app1-deployment-98df9d6f8-2sf4d 1/1 Running 0 94m wework-tomcat-app1-deployment-98df9d6f8-d9pn5 1/1 Running 0 5h48m wework-tomcat-app1-deployment-98df9d6f8-fnjlx 1/1 Running 0 90m wework-tomcat-app1-deployment-98df9d6f8-ghw76 1/1 Running 0 102m root@k8s-master-01:~/metrics# kubectl get deploy -n wework wework-tomcat-app1-deployment NAME READY UP-TO-DATE AVAILABLE AGE wework-tomcat-app1-deployment 4/4 4 4 5d1h- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
5分钟之后,由于cpu使用率小于30%,于是hpa进行了收缩
root@k8s-master-01:~/metrics# kubectl get deployments.apps -n wework wework-tomcat-app1-deployment NAME READY UP-TO-DATE AVAILABLE AGE wework-tomcat-app1-deployment 2/2 2 2 5d1h root@k8s-master-01:~/metrics# kubectl get pods -n wework |grep wework-tomcat-app1 wework-tomcat-app1-deployment-98df9d6f8-d9pn5 1/1 Running 0 5h52m wework-tomcat-app1-deployment-98df9d6f8-ghw76 1/1 Running 0 106m- 1
- 2
- 3
- 4
- 5
- 6
2.4 测试自动伸缩
2.4.1 自动扩展
进入一个pod,使用openssl命令跑高CPU
[root@wework-tomcat-app1-deployment-98df9d6f8-ghw76 /]# openssl speed Doing md2 for 3s on 16 size blocks: 696894 md2's in 2.85s Doing md2 for 3s on 64 size blocks: 355911 md2's in 2.86s Doing md2 for 3s on 256 size blocks: 122354 md2's in 2.91s Doing md2 for 3s on 1024 size blocks: 33355 md2's in 2.91s Doing md2 for 3s on 8192 size blocks: 4360 md2's in 2.97s Doing md4 for 3s on 16 size blocks: 22608001 md4's in 2.94s Doing md4 for 3s on 64 size blocks: 16352809 md4's in 2.91s Doing md4 for 3s on 256 size blocks: 9126019 md4's in 2.81s Doing md4 for 3s on 1024 size blocks: 3463848 md4's in 2.96s Doing md4 for 3s on 8192 size blocks: 484363 md4's in 2.85s Doing md5 for 3s on 16 size blocks: 16087992 md5's in 2.80s Doing md5 for 3s on 64 size blocks: 11538806 md5's in 2.94s Doing md5 for 3s on 256 size blocks: 6081728 md5's in 2.97s Doing md5 for 3s on 1024 size blocks: 2075725 md5's in 2.93s- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
此时HPA会自动扩展POD数量
root@k8s-master-01:~/metrics# kubectl get hpa -n wework --watch NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 2%/30% 2 10 2 46m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 8%/30% 2 10 2 46m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 5%/30% 2 10 2 47m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 5%/30% 2 10 2 48m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 102%/30% 2 10 2 49m yawework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 102%/30% 2 10 4 49m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 102%/30% 2 10 7 49m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 2%/30% 2 10 7 50m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 22%/30% 2 10 7 51m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 34%/30% 2 10 7 52m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 34%/30% 2 10 7 53m- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
查看HPA详细信息可以看到,POD数量从2个扩展到4个,又从4个扩展到7个.
root@k8s-master-01:~# kubectl describe hpa -n wework wework-tomcat-app1-podautoscaler Name: wework-tomcat-app1-podautoscaler Namespace: wework Labels: app=wework-tomcat-app1-selector version=v2beta1 Annotations: <none> CreationTimestamp: Tue, 23 Aug 2022 16:01:06 +0800 Reference: Deployment/wework-tomcat-app1-deployment Metrics: ( current / target ) resource cpu on pods (as a percentage of request): 34% (171m) / 30% Min replicas: 2 Max replicas: 10 Deployment pods: 7 current / 7 desired Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale True ReadyForNewScale recommended size matches current size ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request) ScalingLimited False DesiredWithinRange the desired count is within the acceptable range Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulRescale 41m horizontal-pod-autoscaler New size: 2; reason: All metrics below target Normal SuccessfulRescale 5m37s horizontal-pod-autoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target Normal SuccessfulRescale 5m22s horizontal-pod-autoscaler New size: 7; reason: cpu resource utilization (percentage of request) above target root@k8s-master-01:~/metrics# kubectl get pods -n wework |grep wework-tomcat-app1 wework-tomcat-app1-deployment-98df9d6f8-6px56 1/1 Running 0 95s wework-tomcat-app1-deployment-98df9d6f8-d9pn5 1/1 Running 0 6h30m wework-tomcat-app1-deployment-98df9d6f8-dt29q 1/1 Running 0 95s wework-tomcat-app1-deployment-98df9d6f8-fpzmp 1/1 Running 0 95s wework-tomcat-app1-deployment-98df9d6f8-ghw76 1/1 Running 0 144m wework-tomcat-app1-deployment-98df9d6f8-vxrlx 1/1 Running 0 110s- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
2.4.2 自动收缩
此时停止CPU负载
oing camellia-128 cbc for 3s on 8192 size blocks: 88823 camellia-128 cbc's in 2.97s Doing camellia-192 cbc for 3s on 16 size blocks: 28821766 camellia-192 cbc's in 2.91s Doing camellia-192 cbc for 3s on 64 size blocks: 8081136 camellia-192 cbc's in 2.92s Doing camellia-192 cbc for 3s on 256 size blocks: ^C [root@wework-tomcat-app1-deployment-98df9d6f8-ghw76 /]#- 1
- 2
- 3
- 4
- 5
停止后由于hpa获取到CPU使用率的下降,也开始了收缩POD的数量
root@k8s-master-01:~/metrics# kubectl get hpa -n wework --watch NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 略 wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 33%/30% 2 10 7 55m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 27%/30% 2 10 7 56m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 0%/30% 2 10 7 57m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 0%/30% 2 10 7 58m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 0%/30% 2 10 7 59m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 0%/30% 2 10 7 60m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 0%/30% 2 10 7 62m wework-tomcat-app1-podautoscaler Deployment/wework-tomcat-app1-deployment 0%/30% 2 10 2 62m root@k8s-master-01:~/metrics# kubectl get pods -n wework |grep wework-tomcat-app1 wework-tomcat-app1-deployment-98df9d6f8-d9pn5 1/1 Running 0 6h42m wework-tomcat-app1-deployment-98df9d6f8-ghw76 1/1 Running 0 156m- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
此时,Pod数量又恢复到了2个.HPA自动伸缩测试完毕
-
相关阅读:
【PX4】Ubuntu20.04+ROS Noetic 配置PX4-v1.12.2和Gazebo11联合仿真环境【教程】
武汉新时标文化传媒有限公司抖音小店无货源真的靠谱吗？
TF-IDF算法：从文本中发现价值的黄金法则
字符加密A--E,B-F,W--A
Java 栈帧的一个面试题
Redis 复制(replica)
[Linux Review-1] Linux OS fundamental #101
springboot2.7.10升级到3.0.8版本
《Redis设计与实现》阅读总结-4
JAVA计算机毕业设计作业管理系统Mybatis+系统+数据库+调试部署
- 原文地址：https://blog.csdn.net/qq_29974229/article/details/126488937
