• Kubernetes(k8s)的流量负载组件Ingress安装与使用

    目录

    1. Ingress介绍

    Service对集群之外暴露服务的主要方式有两种：NodePort和LoadBalancer，但这两种方式都有缺点：

    • NodePort方式会占用集群每个节点的很多端口
    • LoadBalancer每个Service都需要一个LB，并且需要kubernetes之外的设备支持

    Ingress只需要一个NodePort或者一个LB就可以满足暴露多个Service的需求，工作机制大致如下图所示

    Ingress

    Ingress相当于一个七层的负载均衡器，是kubernetes对反向代理的一个抽象，它的工作原理类似于Nginx

    原理是在Ingress Service中定义哪个域名对应kubernetes集群中的哪个Service，Ingress Controller通过监听这些映射规则并转化为Nginx(或Contour、Haproxy)的反向代理配置，并将反向代理配置写入这些服务，最后由Nginx(或Contour、Haproxy)对外提供服务

    nginx代理

    2. ingress-nginx Controller安装

    更多细节参考github kubernetes/ingress-nginx

    这里安装的是ingress-nginx 1.2.0版本，适用于Kubernetes 1.23，使用的Nginx版本1.19.10

    1. 下载Deployent的yaml文件
    [root@k8s-master ~]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.2.0/deploy/static/provider/cloud/deploy.yaml

    • 1
    1. 在所有node节点上，拉取国内的ingress-nginx/controller:v1.2.0镜像源
    [root@k8s-node1 ~]# crictl pull dyrnq/controller:v1.2.0

    • 1
    1. 修改yaml文件内容，将下面两处
    image: k8s.gcr.io/ingress-nginx/controller:v1.2.0@sha256:d8196e3bc1e72547c5dec66d6556c0ff92a23f6d0919b206be170bc90d5f9185

image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660

image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660

    • 1
    • 2
    • 3
    • 4
    • 5

    修改成

    image: docker.io/dyrnq/controller:v1.2.0@sha256:d8196e3bc1e72547c5dec66d6556c0ff92a23f6d0919b206be170bc90d5f9185

image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660

image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660

    • 1
    • 2
    • 3
    • 4
    • 5
    1. 运行deploy
    [root@k8s-master ~]# kubectl apply -f deploy.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    1. 查看ingress-nginx的pod
    [root@k8s-master ~]# kubectl get pod -n ingress-nginx -o wide
NAME                                        READY   STATUS      RESTARTS       AGE   IP               NODE        NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-crt25        0/1     Completed   0              37h   10.244.169.172   k8s-node2              
ingress-nginx-admission-patch-9nn5g         0/1     Completed   1              37h   10.244.169.170   k8s-node2              
ingress-nginx-controller-77d4dc6978-cksgf   1/1     Running     1 (151m ago)   37h   10.244.169.175   k8s-node2              
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    1. 查看ingress-nginx的Service
    [root@k8s-master ~]# kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.96.68.173        80:31238/TCP,443:30986/TCP   37h
ingress-nginx-controller-admission   ClusterIP      10.96.185.8            443/TCP                      37h
[root@k8s-master ~]#

    • 1
    • 2
    • 3
    • 4
    • 5

    3. 准备Pod和Service

    3.1 准备Pod

    通过Deployment分别创建3个nginx pod和3个tomcat pod。新建nginx-tomcat-deployment.yaml文件，内容如下。然后创建deployment

    [root@k8s-master ~]# cat nginx-tomcat-deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: dev
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx-pod
  template:
    metadata:
      labels:
        app: nginx-pod
    spec:
      containers:
        - name: nginx-container
          image: nginx:latest
          ports:
            - name: nginx-port
              containerPort: 80
              protocol: TCP

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deployment
  namespace: dev
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat-pod
  template:
    metadata:
      labels:
        app: tomcat-pod
    spec:
      containers:
        - name: tomcat-container
          image: tomcat:8.5-jre10-slim
          ports:
            - name: tomcat-port
              containerPort: 8080
              protocol: TCP
[root@k8s-master ~]# 
[root@k8s-master ~]# kubectl apply -f nginx-tomcat-deployment.yaml 
deployment.apps/nginx-deployment created
deployment.apps/tomcat-deployment created
[root@k8s-master ~]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53

    查看Pod

    [root@k8s-master ~]# kubectl get pod -n dev
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-84d58db8c4-75z5v   1/1     Running   0          24m
nginx-deployment-84d58db8c4-lqpqj   1/1     Running   0          24m
nginx-deployment-84d58db8c4-skcnq   1/1     Running   0          24m
tomcat-deployment-d4b4bdfdf-5jlqv   1/1     Running   0          24m
tomcat-deployment-d4b4bdfdf-lrbpt   1/1     Running   0          24m
tomcat-deployment-d4b4bdfdf-x7c97   1/1     Running   0          24m
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9

    3.2 准备Service

    这里创建HeadLiness类型的Service分别指向nginx-pod和tomcat-pod，直接从Service的域名转发到后端Pod，不经过VIP(虚拟IP)。新建nginx-tomcat-headliness.yaml文件，内容如下。然后创建Service

    [root@k8s-master ~]# cat nginx-tomcat-headliness.yaml 
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: dev
spec:
  selector:
    app: nginx-pod
  type: ClusterIP
  clusterIP: None
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

---


apiVersion: v1
kind: Service
metadata:
  name: tomcat-service
  namespace: dev
spec:
  selector:
    app: tomcat-pod
  type: ClusterIP
  clusterIP: None
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080


[root@k8s-master ~]# 
[root@k8s-master ~]# kubectl apply -f nginx-tomcat-headliness.yaml 
service/nginx-service created
service/tomcat-service created
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40

    查看Service

    [root@k8s-master ~]# kubectl get svc -n dev
NAME             TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
nginx-service    ClusterIP   None                 80/TCP     4m1s
tomcat-service   ClusterIP   None                 8080/TCP   4m1s
[root@k8s-master ~]#

    • 1
    • 2
    • 3
    • 4
    • 5

    4. Ingress的HTTP代理

    4.1 Ingress的创建

    新建ingress-http.yaml文件，内容如下。然后运行ingress

    [root@k8s-master ~]# cat ingress-http.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-http
  namespace: dev
spec:
  rules:
    - host: nginx.bulut.com
      http:
        paths:
          - path: /
            pathType: Exact
            backend:
              service:
                name: nginx-service
                port:
                  number: 80  
    - host: tomcat.bulut.com
      http:
        paths:
          - path: /
            pathType: Exact
            backend:
              service:
                name: tomcat-service
                port:
                  number: 8080
  ingressClassName: nginx
[root@k8s-master ~]# 
[root@k8s-master ~]# kubectl apply -f ingress-https.yaml 
ingress.networking.k8s.io/ingress-http created
[root@k8s-master ~]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33

    4.2 Ingress的查看

    [root@k8s-master ~]# kubectl get ingress ingress-http -n dev
NAME           CLASS   HOSTS                              ADDRESS   PORTS   AGE
ingress-http   nginx   nginx.bulut.com,tomcat.bulut.com             80      64m
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4

    4.3 Ingress详情的查看

    [root@k8s-master ~]# kubectl describe ingress ingress-http -n dev
Name:             ingress-http
Labels:           
Namespace:        dev
Address:          
Ingress Class:    nginx
Default backend:  
Rules:
  Host              Path  Backends
  ----              ----  --------
  nginx.bulut.com   
                    /   nginx-service:80 (10.244.169.178:80,10.244.169.179:80,10.244.36.70:80)
  tomcat.bulut.com  
                    /   tomcat-service:8080 (10.244.169.176:8080,10.244.169.177:8080,10.244.36.69:8080)
Annotations:        
Events:
  Type    Reason  Age                   From                      Message
  ----    ------  ----                  ----                      -------
  Normal  Sync    116s (x3 over 4m34s)  nginx-ingress-controller  Scheduled for sync
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20

    4.4 nginx和tomcat的服务访问

    在要访问的服务器的/etc/hosts文件中，添加IP域名映射。其中IP是Kubernetes运行ingress-nginx-controller pod的节点(这里是192.168.23.162/k8s-node2)，如下所示

    [root@docker ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

......省略部分......

192.168.23.162 nginx.bulut.com
192.168.23.162 tomcat.bulut.com

[root@docker ~]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    然后访问nginx和tomcat

    [root@docker ~]# 
[root@docker ~]# curl http://nginx.bulut.com:31238



Welcome to nginx!




    Welcome to nginx!
    

    If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
    


    For online documentation and support please refer to
nginx.org.
    
Commercial support is available at
nginx.com.
    


    Thank you for using nginx.
    


[root@docker ~]# 
[root@docker ~]# curl http://tomcat.bulut.com:31238






    
        
        Apache Tomcat/8.5.35
        
        
        
    

    
        
    
            
            
    
                
    Apache Tomcat/8.5.35
    
            
    
......省略部分......
        
    
    


[root@docker ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62

    4.5 Client到Pod的服务访问流程说明

    这里以访问nginx服务为例进行说明：

    1. Client访问http://nginx.bulut.com:31238，被解析成访问http://192.168.23.162:31238
    2. 查看ingress-nginx的Service如下
    [root@k8s-master ~]# kubectl get svc -n ingress-nginx 
NAME                                 TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.96.68.173        80:31238/TCP,443:30986/TCP   38h
ingress-nginx-controller-admission   ClusterIP      10.96.185.8            443/TCP                      38h
[root@k8s-master ~]#

    • 1
    • 2
    • 3
    • 4
    • 5

    虽然TYPE为LoadBalancer，但请求还是会转发到http://10.96.68.173:80

    1. 因为我们最先访问的域名是nginx.bulut.com，所以会将此请求转发到http://nginx-service:80
    2. 然后由3个nginx-pod中的一个提供服务

    5. Ingress的HTTPS代理

    5.1 生成证书

    [root@k8s-master ~]# openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 3650 -out tls.crt -subj "/C=CN/ST=HN/L=CS/O=bulut/OU=bulut/CN=bulut.com/emailAddress=bulut@163.com"
Generating a 2048 bit RSA private key
............................+++
......................................................................................................+++
writing new private key to 'tls.key'
-----
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7

    参数说明如下：

    • req：表示证书请求的子命令
    • -newkey rsa:2048：表示生成私钥
    • -nodes：表示私钥不加密
    • -keyout tls.key：表示生成的私钥输出文件
    • -x509：表示输出证书
    • -days 3650：表示证书有效期
    • -out tls.crt：表示生成的证书输出文件
    • -subj “/C=CN/ST=HN/L=CS/O=bulut/OU=bulut/CN=bulut.com/emailAddress=bulut@163.com”：表示自动输入证书拥有者信息，分别表示：国家/省份/城市/公司名/部门名/域名/邮件地址

    5.2 创建密钥

    [root@k8s-master ~]# kubectl create secret tls tls-secret --key tls.key --cert tls.crt
secret/tls-secret created
[root@k8s-master ~]#

    • 1
    • 2
    • 3

    5.1 Ingress的创建

    新建ingress-https.yaml文件，内容如下。然后运行ingress

    [root@k8s-master ~]# cat ingress-https.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-https
  namespace: dev
spec:
  tls:
    - secretName: tls-secret
      hosts:
        - nginx.bulut.com
        - tomcat.bulut.com
  rules:
    - host: nginx.bulut.com
      http:
        paths:
          - path: /
            pathType: Exact
            backend:
              service:
                name: nginx-service
                port:
                  number: 80
    - host: tomcat.bulut.com
      http:
        paths:
          - path: /
            pathType: Exact
            backend:
              service:
                name: tomcat-service
                port:
                  number: 8080

[root@k8s-master ~]#
[root@k8s-master ~]# kubectl apply -f ingress-https.yaml 
ingress.networking.k8s.io/ingress-https created
[root@k8s-master ~]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38

    4.2 Ingress的查看

    [root@k8s-master ~]# kubectl get ingress ingress-https -n dev
NAME            CLASS    HOSTS                              ADDRESS   PORTS     AGE
ingress-https      nginx.bulut.com,tomcat.bulut.com             80, 443   2m38s
[root@k8s-master ~]#

    • 1
    • 2
    • 3
    • 4

    4.3 Ingress详情的查看

    [root@k8s-master ~]# kubectl get ingress ingress-https -n dev
NAME            CLASS    HOSTS                              ADDRESS   PORTS     AGE
ingress-https      nginx.bulut.com,tomcat.bulut.com             80, 443   2m38s
[root@k8s-master ~]# 
[root@k8s-master ~]# kubectl describe ingress ingress-https -n dev
Name:             ingress-https
Labels:           
Namespace:        dev
Address:          
Ingress Class:    
Default backend:  
TLS:
  tls-secret terminates nginx.bulut.com,tomcat.bulut.com
Rules:
  Host              Path  Backends
  ----              ----  --------
  nginx.bulut.com   
                    /   nginx-service:80 (10.244.169.178:80,10.244.169.179:80,10.244.36.70:80)
  tomcat.bulut.com  
                    /   tomcat-service:8080 (10.244.169.176:8080,10.244.169.177:8080,10.244.36.69:8080)
Annotations:        
Events:             
[root@k8s-master ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23

    4.4 nginx和tomcat的服务访问

    在要访问的服务器的/etc/hosts文件中，添加IP域名映射。其中IP是Kubernetes运行ingress-nginx-controller pod的节点(这里是192.168.23.162/k8s-node2)，如下所示

    [root@docker ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

......省略部分......

192.168.23.162 nginx.bulut.com
192.168.23.162 tomcat.bulut.com

[root@docker ~]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    然后访问nginx和tomcat

    [root@docker ~]# 
[root@docker ~]# curl --insecure https://nginx.bulut.com:30986



Welcome to nginx!




    Welcome to nginx!
    

    If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
    


    For online documentation and support please refer to
nginx.org.
    
Commercial support is available at
nginx.com.
    


    Thank you for using nginx.
    


[root@docker ~]# 
[root@docker ~]# curl --insecure https://tomcat.bulut.com:30986





    
        
        Apache Tomcat/8.5.35
        
        
        
    

    
        
    
            
            
    
                
    Apache Tomcat/8.5.35
    
            
    
......省略部分......
        
    
    


[root@docker ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
  • 相关阅读:
    【Rust日报】2023-10-16 为什么要异步 Rust
    基于架构软件设计-架构真题（五十八）
    Failed to start mysql.service Unit not found
    Stretched mesh
    深度学习实战基础案例——卷积神经网络（CNN）基于Xception的猫狗识别|第2例
    变革加速，博睿数据赋能“中国智造”转型升级
    正则表达式
    Elastic Search(一)
    搬砖日记：vue2 用require引入图片项目编译失败
    多级缓存（笔记）
  • 原文地址：https://blog.csdn.net/yy8623977/article/details/124899742