K8s集群实现集群业务是否对外暴露的控制（多LB实施方案）

在这里插入图片描述

架构图：

简述实施步骤：

1.新建集群会自动创建一个公网的LB，需要新创建一个内网LB绑定在此nginx controller 上，实现内网访问pod的目的。

2.新建专属内网的nginx controller 绑定内网LB只对内网提供服务，并且修改默认的ingress.class标签

1.基础Nginx Controller绑定内网LB

参考网址：https://help.aliyun.com/document_detail/151506.html

查看集群现存lb

# kubectl get svc -A |grep LoadBalancer
kube-system   nginx-ingress-lb                     LoadBalancer   192.168.223.150   47.108.153.86   80:30299/TCP,443:31348/TCP   86m
1
2

新建集群LB服务

（推荐）方式一：

集群服务 > 网络 > 服务 > 选择nginx-controller所在命名空间 > 创建 > 负载均衡.内网.新建slb.关联nginx

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-hAkwuUH4-1656317466456)(C:\Users\OPS\Desktop\K8s集群多LB方案\集群多LB方案.assets\image-20220627144407609.png)]$

这种方式是最快捷的，或者也能选择手动创建

方式二：

1.创建负载均衡器 2.在此页面创建时，绑定已有负载均衡即可

或者，当您成功创建一个私网SLB实例后，您可以通过以下示例注解来配置Nginx Ingress Controller使用该SLB实例，

# nginx ingress slb service
apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress-lb
  namespace: kube-system
  labels:
    app: nginx-ingress-lb
  annotations:
    # 指明SLB实例地址类型为私网类型。
    service.beta.kubernetes.io/alicloud-loadbalancer-address-type: intranet
    # 修改为您的私网SLB实例ID。
    service.beta.kubernetes.io/alicloud-loadbalancer-id: <YOUR_INTRANET_SLB_ID>
    # 是否自动创建SLB端口监听（会覆写已有端口监听），也可手动创建端口监听。
    #service.beta.kubernetes.io/alicloud-loadbalancer-force-override-listeners: 'true'
spec:
  type: LoadBalancer
  # route traffic to other nodes
  externalTrafficPolicy: "Cluster"
  ports:
  - port: 80
    name: http
    targetPort: 80
  - port: 443
    name: https
    targetPort: 443
  selector:
    # select app=ingress-nginx pods
    app: ingress-nginx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

查询是否创建成功

# kubectl get svc -A |grep LoadBalancer
kube-system   nginx-ingress-lb                     LoadBalancer   192.168.223.150   47.108.153.86   80:30299/TCP,443:31348/TCP   100m
kube-system   vpc-lb                               LoadBalancer   192.168.37.202    172.24.43.76    80:32532/TCP,443:31330/TCP   3m19s
1
2
3

部署服务测试连通性

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hellok8s-dep
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-kubernetes
  template:
    metadata:
      labels:
        app: hello-kubernetes
    spec:
      containers:
      - name: hello-kubernetes
        image: paulbouwer/hello-kubernetes:1.4
        #image: centos:7
        #args:
        #- sleep
        #- "1000000"
        ports:
        - containerPort: 8080

---
apiVersion: v1
kind: Service
metadata:
  name: hellok8s-svc
  namespace: default
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: hello-kubernetes


---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hellok8s-ingress
  annotations:
    ##用来绑定特定的ingress-nginx
    #kubernetes.io/ingress.class: ack-nginx-vpc    
    ##默认class
    kubernetes.io/ingress.class: nginx
    ##配置了ssl证书即打开此配置
    #nginx.ingress.kubernetes.io/ssl-redirect: 'true'
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/enable-cors: 'true'
    nginx.ingress.kubernetes.io/cors-allow-headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization,Access-Control-Allow-Origin
    nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS, DELETE
    nginx.ingress.kubernetes.io/cors-allow-origin: '*'
    nginx.ingress.kubernetes.io/cors-allow-credentials: 'true'

spec:
  ##按需配置ssl证书
  #tls:
  #- secretName: da-e.top-tls
  #  hosts:
  #  - "*.test.com"
  rules:
  ##按需配置域名
  #- host: www.test.com  
  - http:
      paths:
      - path: /hellok8s
        pathType: Prefix
        backend:
          service:
            name: hellok8s-svc
            port:
              number: 80
      - path: /
        pathType: Prefix
        backend:
          service:
            name: hellok8s-svc

            port:
              number: 80
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

# kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
hellok8s-dep-6588f6bd76-wlh4h   1/1     Running   0          60s
1
2
3

# kubectl get svc -A |grep LoadBalancer
kube-system   nginx-ingress-lb                     LoadBalancer   192.168.223.150   47.108.153.86   80:30299/TCP,443:31348/TCP   100m
kube-system   vpc-lb                               LoadBalancer   192.168.37.202    172.24.43.76    80:32532/TCP,443:31330/TCP   3m19s
1
2
3

验证内外网访问

浏览器访问公网LB地址

http://47.108.153.86/hellok8s
1

同一VPC机器访问内网LB地址

# curl 172.24.43.76 
<!DOCTYPE html>
<html>
<head>
    <title>Hello Kubernetes!</title>
    <link rel="stylesheet" type="text/css" href="/css/main.css">
    <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Ubuntu:300" >
</head>
<body>

  <div class="main">
    <img src="/images/kubernetes.png"/>
    <div class="content">
      <div id="message">
  Hello world!
</div>
<div id="info">
  <table>
    <tr>
      <th>pod:</th>
      <td>hellok8s-dep-6588f6bd76-wlh4h</td>
    </tr>
    <tr>
      <th>node:</th>
      <td>Linux (4.19.91-26.al7.x86_64)</td>
    </tr>
  </table>

</div>

    </div>
  </div>

</body>
</html>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

总结：这样做的好处：同一VPC访问服务时，可以走内网，节省流量

2.集群新建Nginx Controller内网访问LB

参考网址：https://help.aliyun.com/document_detail/151524.html

# kubectl create ns vpc-nginx
namespace/vpc-nginx created
1
2

web页面创建应用nginx controller应用

容器服务》应用市场》ack-ingress-nginx-v1 》选择对应集群和命名空间（找不到命名空间时，刷新网页）
在这里插入图片描述

修改ingressClassResource的name字段

必须配置为专属标识，以区别两个nginx controller

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-P1sUeyhQ-1656317466457)(C:\Users\OPS\Desktop\K8s集群多LB方案\集群多LB方案.assets\image-20220627152037154.png)]$

默认为ack-nginx 修改为 ack-nginx-vpc
$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-WUAuQo2T-1656317466459)(C:\Users\OPS\Desktop\K8s集群多LB方案\集群多LB方案.assets\image-20220627154315485.png)]$

关闭公网访问 controller.service.external.enabled = false

打开内网controller.service.internal.enabled = true

（注意参考文档配置是 ack-ingress-nginx-v1版本）
在这里插入图片描述

等待部署完成

# kubectl -n vpc-nginx get svc |grep LoadBalancer
ack-ingress-nginx-v1-default-controller-internal    LoadBalancer   192.168.183.85    172.24.43.77   80:30389/TCP,443:31903/TCP   2m47s
1
2

验证阶段

##删除刚刚的测试pod
# kubectl delete -f hello.yaml 
deployment.apps "hellok8s-dep" deleted
service "hellok8s-svc" deleted
ingress.networking.k8s.io "hellok8s-ingress" deleted
1
2
3
4
5

修改ingress.class配置绑定内网Nginx Controller

##只修改ingress.annotations片段
......
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hellok8s-ingress
  annotations:
    ##用来绑定特定的ingress-nginx
    kubernetes.io/ingress.class: ack-nginx-vpc    
    ##默认class
    #kubernetes.io/ingress.class: nginx
    ##配置了ssl证书即打开此配置
    #nginx.ingress.kubernetes.io/ssl-redirect: 'true'
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/enable-cors: 'true'
    nginx.ingress.kubernetes.io/cors-allow-headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization,Access-Control-Allow-Origin
    nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS, DELETE
    nginx.ingress.kubernetes.io/cors-allow-origin: '*'
    nginx.ingress.kubernetes.io/cors-allow-credentials: 'true'

spec:
......

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

部署查看效果

# kubectl get ing
NAME               CLASS    HOSTS   ADDRESS        PORTS   AGE
hellok8s-ingress   <none>   *       172.24.43.77   80      55s
[root@iZ2vc69mi81oajsfk8bjefZ .kube]# kubectl  get svc -A |grep LoadBalancer
kube-system   nginx-ingress-lb                                    LoadBalancer   192.168.223.150   47.108.153.86   80:30299/TCP,443:31348/TCP   162m
kube-system   vpc-lb                                              LoadBalancer   192.168.37.202    172.24.43.76    80:32532/TCP,443:31330/TCP   65m
vpc-nginx     ack-ingress-nginx-v1-default-controller-internal    LoadBalancer   192.168.183.85    172.24.43.77    80:30389/TCP,443:31903/TCP   7m48s
1
2
3
4
5
6
7

发现ingress 已经成功绑定到内网的LB上

测试访问

## 公网私网LB失败，说明ingress未添加
[root@iZ2vc69mi81oajsfk8bjefZ .kube]# curl 47.108.153.86 
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@iZ2vc69mi81oajsfk8bjefZ .kube]# curl 172.24.43.76
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
## 私网LB访问成功 说明ingress添加成功
[root@iZ2vc69mi81oajsfk8bjefZ .kube]# curl 172.24.43.77
<!DOCTYPE html>
<html>
<head>
    <title>Hello Kubernetes!</title>
    <link rel="stylesheet" type="text/css" href="/css/main.css">
    <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Ubuntu:300" >
</head>
<body>

  <div class="main">
    <img src="/images/kubernetes.png"/>
    <div class="content">
      <div id="message">
  Hello world!
</div>
<div id="info">
  <table>
    <tr>
      <th>pod:</th>
      <td>hellok8s-dep-6588f6bd76-xwj7q</td>
    </tr>
    <tr>
      <th>node:</th>
      <td>Linux (4.19.91-26.al7.x86_64)</td>
    </tr>
  </table>

</div>

    </div>
  </div>

</body>
</html>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

总结：部署阶段稍麻烦，但是部署完成后，仅通过修改ingress配置绑定对应的Nginx Controller即可实现业务的对外控制

相关阅读:
HTML按钮通过JS实现选中和取消
 【微客云】外卖霸王餐-城市分站加盟-区域代理-服务商模式
 顺序表与链表（下）
使用Python进行名片OCR（识别姓名，职务，电话，Email邮箱）
超全面的SpringCloud Alibaba电子版教程，我司已用三年（新人老人万能通用版）
Git构建分布式版本控制系统
 【C++笔试强训】第九天
 代码随想录算法训练营第二十四天丨回溯算法part02
深度学习day01
开发Chrome插件，实现网站自动登录
原文地址：https://blog.csdn.net/ht9999i/article/details/125486151

K8s集群 实现集群业务是否对外暴露的控制 （多LB实施方案）