• k8s 安装 ingress-nginx

    k8s 安装 ingress-nginx

    参考官方文档：https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/

    Kubernetes 暴露服务的方式目前只有三种：LoadBlancer Service、NodePort Service、Ingress

    image-20220830095008297

    这个负载均衡是基于nginx七层反向代理来实现的，ingress工作原理如下图：

    image-20220830095029973

    image-20220830095042228

    外部客户端通过访问负载均衡器，然后调度到service上，然后在调度到IngressController，IngressController通过Ingress规则（域名或虚拟主机）访问到后端pod，而在Ingress规则当中对应的主机是由service分组来设定的，可以看到，这幅图有2种service，最上面的service是用来对外提供服务的，而下面2个service仅仅是用来分pod组的。

    • 1
    Kubernetes 并没有自带 Ingress Controller，实际上ingress-controller只是一个统称，具体实现有多种，需要自己单独安装，目前，由k8s维护的ingress-controller只有google云的GCE与ingress-nginx两个，常用的是 Ingress-nginx Controller。

    • 1
    Ingress 一般由三个组件组成： 1. Nginx 反向代理负载均衡器 2. Ingress Controller 可以理解为控制器，它通过不断的跟 Kubernetes API 交互，实时获取后端 Service、Pod 等的变化，比如新增、删除等，然后结合 Ingress 定义的规则生成配置，然后动态更新上边的 Nginx 负载均衡器，并刷新使配置生效，来达到服务自动发现的作用。 3. Ingress 则是定义规则，通过它定义某个域名的请求过来之后转发到集群中指定的 Service。它可以通过 Yaml 文件定义，可以给一个或多个 Service 定义一个或多个 Ingress 规则

    • 1

    如何创建 Ingress 资源

    • Ingress 中的spec字段是Ingress资源的核心组成部分，主要包含以下3个字段：

      • rules：用于定义当前Ingress资源的转发规则列表；由rules定义规则，或没有匹配到规则时，所有的流量会转发到由backend定义的默认后端。
      • backend：默认的后端，用于服务那些没有匹配到任何规则的请求；定义Ingress资源时，必须要定义backend或rules两者之一，该字段用于让负载均衡器指定一个全局默认的后端。
      • tls：TLS配置，目前仅支持通过默认端口443提供服务，如果要配置指定的列表成员指向不同的主机，则需要通过SNI TLS扩展机制来支持该功能。

    部署 Ingress 控制器（Nginx）

    • 下载 ingress controller
    • wget https://github.com/kubernetes/ingress-nginx/archive/nginx-0.30.0.tar.gz
    所需安装包
    链接：https://pan.baidu.com/s/1wM3IVmGCud9QNXYH7QiXNw?pwd=mq57 
提取码：mq57 
--来自百度网盘超级会员V1的分享

    • 1
    • 2
    • 3
    ingress-nginx文件位于deploy/static目录下，各文件的作用：

configmap.yaml:
提供configmap可以在线更新nginx的配置

default-backend.yaml:
提供一个缺省的后台错误页面 404

namespace.yaml:
创建一个独立的命名空间 ingress-nginx

rbac.yaml：
创建对应的role rolebinding 用于rbac

tcp-services-configmap.yaml:
修改L4负载均衡配置的configmap

udp-services-configmap.yaml:
修改L4负载均衡配置的configmap

with-rbac.yaml:
有应用rbac的nginx-ingress-controller组件

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22

    image-20220830095217376

    [root@k8s-master ~]# cd /mnt/
[root@k8s-master mnt]# wget https://github.com/kubernetes/ingress-nginx/archive/nginx-0.30.0.tar.gz
[root@k8s-master mnt]# tar xzvf nginx-0.30.0.tar.gz
[root@k8s-master mnt]# cd ingress-nginx-nginx-0.30.0/deploy/static/
[root@k8s-master static]# ls
configmap.yaml  mandatory.yaml  namespace.yaml  provider  rbac.yaml  with-rbac.yaml
[root@k8s-master static]# cp mandatory.yaml /root/
[root@k8s-master static]# cd

# 创建命名空间ingress-nginx
[root@k8s-master static]# cat namespace.yaml 
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    
# 创建几个configmap。可以更新pod中的配置
[root@k8s-master static]# cat configmap.yaml 
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

[root@k8s-master ~]# vim mandatory.yaml #修改配置文件
找到已下apiserver的版本：
apiVersion: apps/v1
kind: DaemonSet  #将原来的Deployment修改为DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  #replicas: 1 #将这里注释
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      hostNetwork: true  #添加此配置，共享宿主机网络
#      nodeSelector:
#        custom/ingress-controller-ready: "true"  #添加此配置
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0 #提前将镜像下载下来
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94
    • 95
    • 96
    • 97
    • 98
    • 99
    • 100
    • 101
    • 102
    • 103
    • 104
    • 105
    • 106
    • 107
    • 108
    • 109
    • 110
    • 111
    • 112
    • 113
    • 114
    • 115
    • 116
    • 117
    • 118
    • 119
    • 120
    • 121
    • 122
    • 123
    • 124
    • 125
    • 126
    • 127
    • 128
    • 129
    • 130
    • 131
    • 132
    • 133
    • 134
    • 135
    • 136
    • 137
    • 138
    • 139
    • 140

    需要修改的地方：

    kind: DaemonSet：
    官方原始文件使用的是deployment，replicate 为 1，这样将会在某一台节点上启动对应的nginx-ingress-controller pod。外部流量访问至该节点，由该节点负载分担至内部的service。测试环境考虑防止单点故障，改为DaemonSet然后删掉replicate ，配合亲和性部署在制定节点上启动nginx-ingress-controller pod，确保有多个节点启动nginx-ingress-controller pod，后续将这些节点加入到外部硬件负载均衡组实现高可用性。

    • 1
    hostNetwork: true：
    添加该字段，暴露nginx-ingress-controller pod的服务端口（80）

    • 1
    nodeSelector:
    增加亲和性部署，有custom/ingress-controller-ready 标签的节点才会部署该DaemonSet

为需要部署nginx-ingress-controller的节点设置lable

    • 1
    • 2
    • 3
    给两个node节点设置lable，我这里用的DeamonSet，不设置也可以。
    [root@k8s-master ~]# kubectl label nodes k8s-node1 custom/ingress-controller-ready=true

[root@k8s-master ~]# kubectl label nodes k8s-node2 custom/ingress-controller-ready=true

同时在两个node节点下载镜像
[root@k8s-node1 ~]# docker pull quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    我这边是设置了master节点污点

    对 master 设置污点，其他工作节点都部署

    kubectl taint nodes k8s-master01 node-shellrole.kubernetes.io/master=true:NoSchedule

    • 1

    image-20220830095538756

    参数解释：
    terminationGracePeriodSeconds: 300

1、Pod 被删除，状态置为 Terminating。
2、kube-proxy 更新转发规则，将 Pod 从 service 的 endpoint 列表中摘除掉，新的流量不再转发到该 Pod。
3、如果 Pod 配置了 preStop Hook ，将会执行。
4、kubelet 对 Pod 中各个 container 发送 SIGTERM 信号以通知容器进程开始优雅停止。
5、等待容器进程完全停止，如果在 terminationGracePeriodSeconds 内 (默认 30s) 还未完全停止，就发送 SIGKILL 信号强制杀死进程。
6、所有容器进程终止，清理 Pod 资源。

优雅退出
要实现优雅终止，务必在业务代码里面处理下 SIGTERM 信号

进程优雅退出的方法

1、preStop-webhook
    lifecycle:
      preStop:
        exec:
          command:
          - sleep
          - 5s
          
2、调整优雅终止时间，terminationGracePeriodSeconds 默认是30s。自己视情况而定（terminationGracePeriodSeconds 一定大于sleep的时间）
特别说明： preStop Hook并不会影响SIGTERM的处理，因此有可能preStopHook还没有执行完就收到SIGKILL导致容器强制退出。因此如果preStop Hook设置了n秒，需要设置terminationGracePeriodSeconds为terminationGracePeriodSeconds+n秒。

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    创建ingress-controller
    [root@k8s-master ~]# kubectl apply -f mandatory.yaml
这里需要注意：rbac和rbaccluster的apiserver的版本需要修改为：
rbac.authorization.k8s.io/v1

查看ingress-controller资源
[root@k8s-master ~]# kubectl get pod -n ingress-nginx

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    image-20220830100046435

    测试ingress

    创建两个应用和service
    [root@k8s-master ~]# vim my-apache.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-apache
spec:
 selector:
   matchLabels:
     run: my-apache
 replicas: 2
 template:
  metadata:
   labels:
    run: my-apache
  spec:
   containers:
   - name: my-apache
     image: daocloud.io/library/httpd:2.4
     ports:
     - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
 name: my-apache
 labels:
  run: my-apache
spec:
 #type: NodePort
 ports:
 - port: 80
   targetPort: 80
   #nodePort: 30002
 selector:
  run: my-apache

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35

    image-20220830100153426

    [root@k8s-master ~]# cat my-nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-nginx
spec:
 selector:
   matchLabels:
     run: my-nginx
 replicas: 2
 template:
  metadata:
   labels:
    run: my-nginx
  spec:
   containers:
   - name: my-nginx
     image: daocloud.io/library/nginx:1.7.9
     ports:
     - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
 name: my-nginx
 labels:
  run: my-nginx
spec:
 #type: NodePort
 ports:
 - port: 80
   targetPort: 80
   #nodePort: 30001
 selector:
  run: my-nginx

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35

    image-20220830100219867

    创建pod和service
    创建pod和service
[root@k8s-master ~]# kubectl apply -f my-apache.yaml
[root@k8s-master ~]# kubectl apply -f my-nginx.yaml

    • 1
    • 2
    • 3

    image-20220830100345095

    配置ingress转发文件

    宿主机网络模式 hostNetwork: true
    [root@k8s-master ~]# cat ingress-test.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: test-ingress
 namespace: default
 annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
 rules:  #定义转发规则
 - host: www.ingress.com  #指定域名方式
   http:
    paths:
    - path: /  #指定访问的路径
      pathType: Prefix  #定义路径的类型
      backend:   #定义转发后端的服务
       service:  #定义转发的service
         name: my-apache
         port:
          number: 80 #由于Ingress控制器开启了hostNetwork: true。这里必须设置80
 - host: www.youngfit.com
   http:
    paths:
    - path: /
      pathType: Prefix
      backend:
       service:
         name: my-nginx
         port:
          number: 80

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30

    image-20220830100430166

    [root@k8s-master mingyu-ingress]# kubectl apply -f ingress-test.yaml

    • 1
    root@k8s-master1:~/ingress_test/pod# kubectl get ing -o wide

    • 1

    image-20220830100536674

    nginx-ingress-controller运行在node1,node2两个节点上。

    如果网络中有dns服务器，在dns中把这两个域名映射到nginx-ingress-controller运行的任意一个节点上，如果没有dns服务器只能修改host文件了。

    任意一个节点上操作：(客户端解析)

    image-20220830100618435

    我这里有两个节点部署了控制器，ip分别为192.168.2.213，192.168.2.215 ，如果有多个，可以随便选。

    访问测试

    在windows电脑设置本地解析

    image-20220830100705566

    image-20220830100931494

    访问测试

    image-20220830101007359

    image-20220830101015677

    配置nodePort方式

    [root@k8s-master nodeport]# ls
ingress-test.yaml  mandatory.yaml  my-apache.yml  my-nginx.yml

    • 1
    • 2
    配置my-nginx.yml
    apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-nginx
spec:
 selector:
   matchLabels:
     run: my-nginx
 replicas: 2
 template:
  metadata:
   labels:
    run: my-nginx
  spec:
   containers:
   - name: my-nginx
     image: daocloud.io/library/nginx:1.7.9
     ports:
     - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
 name: my-nginx
 labels:
  run: my-nginx
spec:
 #type: NodePort
 ports:
 - port: 80
   targetPort: 80
   #nodePort: 30001
 selector:
  run: my-nginx


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35

    image-20220830104625791

    配置my-apache.yml
    apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-apache
spec:
 selector:
   matchLabels:
     run: my-apache
 replicas: 2
 template:
  metadata:
   labels:
    run: my-apache
  spec:
   containers:
   - name: my-apache
     image: daocloud.io/library/httpd:2.4
     ports:
     - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
 name: my-apache
 labels:
  run: my-apache
spec:
 type: NodePort
 ports:
 - port: 80
   targetPort: 80
#   nodePort: 30002
 selector:
  run: my-apache


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35

    image-20220830103211767

    配置mandatory.yaml
    apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "-"
      # Here: "-"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
#kind: DaemonSet
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
#      hostNetwork: true   # 关闭此网络模式
      serviceAccountName: nginx-ingress-serviceaccount
#      nodeSelector:
#        custom/ingress-controller-ready: "true"
#        kubernetes.io/os: linux
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
#          image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94
    • 95
    • 96
    • 97
    • 98
    • 99
    • 100
    • 101
    • 102
    • 103
    • 104
    • 105
    • 106
    • 107
    • 108
    • 109
    • 110
    • 111
    • 112
    • 113
    • 114
    • 115
    • 116
    • 117
    • 118
    • 119
    • 120
    • 121
    • 122
    • 123
    • 124
    • 125
    • 126
    • 127
    • 128
    • 129
    • 130
    • 131
    • 132
    • 133
    • 134
    • 135
    • 136
    • 137
    • 138
    • 139
    • 140
    • 141
    • 142
    • 143
    • 144
    • 145
    • 146
    • 147
    • 148
    • 149
    • 150
    • 151
    • 152
    • 153
    • 154
    • 155
    • 156
    • 157
    • 158
    • 159
    • 160
    • 161
    • 162
    • 163
    • 164
    • 165
    • 166
    • 167
    • 168
    • 169
    • 170
    • 171
    • 172
    • 173
    • 174
    • 175
    • 176
    • 177
    • 178
    • 179
    • 180
    • 181
    • 182
    • 183
    • 184
    • 185
    • 186
    • 187
    • 188
    • 189
    • 190
    • 191
    • 192
    • 193
    • 194
    • 195
    • 196
    • 197
    • 198
    • 199
    • 200
    • 201
    • 202
    • 203
    • 204
    • 205
    • 206
    • 207
    • 208
    • 209
    • 210
    • 211
    • 212
    • 213
    • 214
    • 215
    • 216
    • 217
    • 218
    • 219
    • 220
    • 221
    • 222
    • 223
    • 224
    • 225
    • 226
    • 227
    • 228
    • 229
    • 230
    • 231
    • 232
    • 233
    • 234
    • 235
    • 236
    • 237
    • 238
    • 239
    • 240
    • 241
    • 242
    • 243
    • 244
    • 245
    • 246
    • 247
    • 248
    • 249
    • 250
    • 251
    • 252
    • 253
    • 254
    • 255
    • 256
    • 257
    • 258
    • 259
    • 260
    • 261
    • 262
    • 263
    • 264
    • 265
    • 266
    • 267
    • 268
    • 269
    • 270
    • 271
    • 272
    • 273
    • 274
    • 275
    • 276
    • 277
    • 278
    • 279
    • 280
    • 281
    • 282
    • 283
    • 284
    • 285
    • 286
    • 287
    • 288
    • 289
    • 290
    • 291
    • 292
    • 293
    • 294
    • 295
    • 296
    • 297
    配置 ingress-test.yaml
    [root@k8s-master nodeport]# cat ingress-test.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: test-ingress
 namespace: default
 annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
 rules:  #定义转发规则
 - host: www.ingress.com
   http:
    paths:
    - path: /  #指定访问的路径
      pathType: Prefix  #定义路径的类型
      backend:   #定义转发后端的服务
       service:  #定义转发的service
         name: my-apache
         port:
          number: 80  #这个跟clusetrIP端口保持一致；这里有点模糊。
 - host: www.youngfit.com
   http:
    paths:
    - path: /
      pathType: Prefix
      backend:
       service:
         name: my-nginx
         port:
          number: 81

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30

    image-20220830103406046

    创建服务
    [root@k8s-master nodeport]# kubectl apply -f mandatory.yaml
[root@k8s-master nodeport]# kubectl apply -f my-apache.yml  
[root@k8s-master nodeport]# kubectl apply -f my-nginx.yml
[root@k8s-master nodeport]# kubectl apply -f ingress-test.yaml

    • 1
    • 2
    • 3
    • 4

    image-20220830104721866

    root@k8s-master1:~/ingress_test/nodeport# 
root@k8s-master1:~/ingress_test/nodeport# kubectl get pod -o wide
NAME                         READY   STATUS    RESTARTS   AGE    IP               NODE          NOMINATED NODE   READINESS GATES
my-apache-7c88db656c-msndj   1/1     Running   0          4m5s   10.244.169.143   k8s-node2     <none>           <none>
my-apache-7c88db656c-nfnfs   1/1     Running   0          4m5s   10.244.107.209   k8s-node3     <none>           <none>
my-nginx-67f6c85b8-d4q2x     1/1     Running   0          26s    10.244.107.211   k8s-node3     <none>           <none>
my-nginx-67f6c85b8-xbptf     1/1     Running   0          26s    10.244.169.145   k8s-node2     <none>           <none>
nginx-dep-58d6596f96-jmwb4   1/1     Running   0          4d     10.244.224.1     k8s-master2   <none>           <none>
nginx-dep-58d6596f96-xhlhd   1/1     Running   0          4d     10.244.36.67     k8s-node1     <none>           <none>
root@k8s-master1:~/ingress_test/nodeport# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        7d19h
my-apache    NodePort    10.0.0.123   <none>        80:31796/TCP   4m17s
my-nginx     ClusterIP   10.0.0.249   <none>        80/TCP         38s
root@k8s-master1:~/ingress_test/nodeport# kubectl get ingress
NAME           CLASS    HOSTS                              ADDRESS   PORTS   AGE
test-ingress   <none>   www.ingress.com,www.youngfit.com             80      4m16s


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18

    image-20220830105228876

    配置解析

    image-20220830105401453

    image-20220830105459081

    image-20220830105510502

    遇到报错

    配置文件ingress-test.yaml出现报错
    报错问题：
root@k8s-master1:~/ingress_test/pod# kubectl apply -f ingress-test.yaml 
error: error validating "ingress-test.yaml": error validating data: [ValidationError(Ingress.spec.rules[0].http.paths[0].backend): unknown field "service" in io.k8s.api.networking.v1beta1.IngressBackend, ValidationError(Ingress.spec.rules[1].http.paths[0].backend): unknown field "service" in io.k8s.api.networking.v1beta1.IngressBackend]; if you choose to ignore these errors, turn validation off with --validate=false


    • 1
    • 2
    • 3
    • 4

    image-20220830101407668

    解决方法：
    no matches for kind “Ingress” in version “networking.k8s.io/v1beta1”
apiVersion: networking.k8s.io/v1beta1 换成
apiVersion: networking.k8s.io/v1
APIVersion 改成 v1 过后，pathType 又不对了
是因为版本问题，新版本和旧版本的配置是不一样的
直接上代码

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    报错问题：
root@k8s-master1:~/ingress_test/pod# kubectl apply -f ingress-test.yaml 
Error from server (InternalError): error when creating "ingress-test.yaml": Internal error occurred: failed calling webhook "validate.tomcat.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission-tomcat.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": service "ingress-nginx-controller-admission-tomcat" not found

    • 1
    • 2
    • 3

    image-20220830101700516

     ingress 内部访问错误
如果不百度和Google，永远都不知道怎么解决

先kubectl get ValidatingWebhookConfiguration
再kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
把那个该死的 admission 删掉
然后就正常了

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7

    se

    
[外链图片转存中...(img-gi6XrAnf-1661910373654)]

###### 解决方法：

```shell
no matches for kind “Ingress” in version “networking.k8s.io/v1beta1”
apiVersion: networking.k8s.io/v1beta1 换成
apiVersion: networking.k8s.io/v1
APIVersion 改成 v1 过后，pathType 又不对了
是因为版本问题，新版本和旧版本的配置是不一样的
直接上代码

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    报错问题：
root@k8s-master1:~/ingress_test/pod# kubectl apply -f ingress-test.yaml 
Error from server (InternalError): error when creating "ingress-test.yaml": Internal error occurred: failed calling webhook "validate.tomcat.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission-tomcat.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": service "ingress-nginx-controller-admission-tomcat" not found

    • 1
    • 2
    • 3

    [外链图片转存中…(img-GP0DEiff-1661910373654)]

     ingress 内部访问错误
如果不百度和Google，永远都不知道怎么解决

先kubectl get ValidatingWebhookConfiguration
再kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
把那个该死的 admission 删掉
然后就正常了

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7

    image-20220830101753980

  • 相关阅读:
    Create Web, Desktop, Mobile Apps for .NET 8 with Telerik
    Makefile入门（二）
    win7 资源管理器打开ftp显示乱码
    JavaScript/jQuery功能函数集
    任正非：天空足够大，世界会越来越兴盛
    Vue组件＞一个简单人物介绍轮播组件
    drf-过滤、排序、异常处理、自封装Response
    telnet 连接闪退
    Mysql 千万级别查询优化
    Windows10安装Jenkins
  • 原文地址：https://blog.csdn.net/tianmingqing0806/article/details/126617589