• 猿创征文｜云原生|minikube的部署安装完全手册

    前言：

    学习一个新平台首先当然是能够有这么一个平台了，而kubernetes的部署安装无疑是提高了这一学习的门槛，不管是二进制安装还是kubeadm安装都还是需要比较多的运维技巧的，并且在搭建学习的时候，需要的硬件资源也是比较多的，至少都需要三台或者三台以上的服务器才能够完成部署安装。

    那么，kind或者minikube这样的工具就是一个能够快速的搭建出学习平台的工具，特点是简单，易用，一台服务器的资源就可以搞定了（只是单机需要的内存大一些，建议至少8G内存吧），自动化程度高，基本什么都给你设置好了，并且支持多种虚拟化引擎，比如，docker，container，kvm这些常用的虚拟化引擎都支持。缺点是基本没有定制化。

    minikube支持的虚拟化引擎：

     好了，本教程大部分资料都是从官网的docs里扒的，docs的网址是：Welcome! | minikube

    相关安装部署文件（conntrack.tar.gz解压后，rpm -ivh * 安装就可以了，是相关依赖，minikube-images.tar.gz是镜像包，解压后倒入docker，三个可执行文件放入/root/.minikube/cache/linux/amd64/v1.18.8/目录下即可。）：

    链接：https://pan.baidu.com/s/14-r59VfpZRpfiVGj4IadxA?pwd=k8ss 
    提取码：k8ss 
     

    一，

    get started minikube（开始部署minkube）

    安装部署前的先决条件：

    至少两个CPU，2G内存，20G空余磁盘空间，可访问互联网，有一个虚拟化引擎， Docker, Hyperkit, Hyper-V, KVM, Parallels, Podman, VirtualBox, or VMware Fusion/Workstation其中的一个，那么，docker是比较容易安装的，就不说了，docker吧，操作系统是centos。

    1. What you’ll need
    2. 2 CPUs or more
    3. 2GB of free memory
    4. 20GB of free disk space
    5. Internet connection
    6. Container or virtual machine manager, such as: Docker, Hyperkit, Hyper-V, KVM, Parallels, Podman, VirtualBox, or VMware Fusion/Workstation

    docker的离线安装以及本地化配置_zsk_john的博客-CSDN博客 离线安装docker环境的博文，照此做就可以了，请确保docker环境是安装好的。

    docker版本至少是18.09到20.10

    二，

    开始安装

    下载minikube的执行程序

    1. curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
    2. sudo install minikube-linux-amd64 /usr/local/bin/minikube

    三，

    导入镜像

    由于minikube在安装kubernetes的时候使用的镜像是从外网拉取的，国内由于被墙是无法拉取的，因此，制作了这个离线镜像包。

    1. [root@slave3 ~]# tar zxf minikube-images.tar.gz 
    2. [root@slave3 ~]# cd minikube-images
    3. [root@slave3 minikube-images]# for i in `ls ./*`;do docker load <$i;done
    4. dfccba63d0cc: Loading layer [==================================================>]  80.82MB/80.82MB
    5. Loaded image: gcr.io/k8s-minikube/storage-provisioner:v1.8.1
    6. 225df95e717c: Loading layer [==================================================>]  336.4kB/336.4kB
    7. c965b38a6629: Loading layer [==================================================>]  43.58MB/43.58MB
    8. 。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。略略略

    四，

    初始化kubernetes集群的命令：

    这里大概介绍一下，image-repostory是使用阿里云下载镜像，cni是指定网络插件就用flannel，如果不想用这个删掉这行就可以了，其它没什么要注意的。

    1. minikube config set driver none
    2. minikube start pod-network-cidr='10.244.0.0/16'\
    3.     --extra-config=kubelet.pod-cidr=10.244.0.0/16 \
    4.     --network-plugin=cni \
    5.     --image-repository='registry.aliyuncs.com/google_containers' \
    6.     --cni=flannel \
    7.     --apiserver-ips=192.168.217.23 \
    8.     --kubernetes-version=1.18.8 \
    9.     --vm-driver=none

    启动集群的日志

    1. [root@slave3 conntrack]# minikube start --driver=none --kubernetes-version=1.18.8
    2. * minikube v1.26.1 on Centos 7.4.1708
    3. * Using the none driver based on user configuration
    4. * Starting control plane node minikube in cluster minikube
    5. * Running on localhost (CPUs=4, Memory=7983MB, Disk=51175MB) ...
    6. * OS release is CentOS Linux 7 (Core)
    7. E0911 11:23:25.121495   14039 docker.go:148] "Failed to enable" err=<
    8. 	sudo systemctl enable docker.socket: exit status 1
    9. 	stdout:
    10. 	
    11. 	stderr:
    12. 	Failed to execute operation: No such file or directory
    13.  > service="docker.socket"
    14. ! This bare metal machine is having trouble accessing https://k8s.gcr.io
    15. * To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
    16.     > kubectl.sha256:  65 B / 65 B [-------------------------] 100.00% ? p/s 0s
    17.     > kubelet:  108.05 MiB / 108.05 MiB [--------] 100.00% 639.49 KiB p/s 2m53s                                                                                                                             
    18.   - Generating certificates and keys ...
    19.   - Booting up control plane ...
    20. ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.8:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap": exit status 1
    21. stdout:
    22. [init] Using Kubernetes version: v1.18.8
    23. [preflight] Running pre-flight checks
    24. [preflight] Pulling images required for setting up a Kubernetes cluster
    25. [preflight] This might take a minute or two, depending on the speed of your internet connection
    26. [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
    27. [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
    28. [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
    29. [kubelet-start] Starting the kubelet
    30. [certs] Using certificateDir folder "/var/lib/minikube/certs"
    31. [certs] Using existing ca certificate authority
    32. [certs] Using existing apiserver certificate and key on disk
    33. [certs] Generating "apiserver-kubelet-client" certificate and key
    34. [certs] Generating "front-proxy-ca" certificate and key
    35. [certs] Generating "front-proxy-client" certificate and key
    36. [certs] Generating "etcd/ca" certificate and key
    37. [certs] Generating "etcd/server" certificate and key
    38. [certs] etcd/server serving cert is signed for DNS names [slave3 localhost] and IPs [192.168.217.136 127.0.0.1 ::1]
    39. [certs] Generating "etcd/peer" certificate and key
    40. [certs] etcd/peer serving cert is signed for DNS names [slave3 localhost] and IPs [192.168.217.136 127.0.0.1 ::1]
    41. [certs] Generating "etcd/healthcheck-client" certificate and key
    42. [certs] Generating "apiserver-etcd-client" certificate and key
    43. [certs] Generating "sa" key and public key
    44. [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
    45. [kubeconfig] Writing "admin.conf" kubeconfig file
    46. [kubeconfig] Writing "kubelet.conf" kubeconfig file
    47. [kubeconfig] Writing "controller-manager.conf" kubeconfig file
    48. [kubeconfig] Writing "scheduler.conf" kubeconfig file
    49. [control-plane] Using manifest folder "/etc/kubernetes/manifests"
    50. [control-plane] Creating static Pod manifest for "kube-apiserver"
    51. [control-plane] Creating static Pod manifest for "kube-controller-manager"
    52. [control-plane] Creating static Pod manifest for "kube-scheduler"
    53. [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
    54. [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
    55. [kubelet-check] Initial timeout of 40s passed.
    56. [kubelet-check] It seems like the kubelet isn't running or healthy.
    57. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
    58. [kubelet-check] It seems like the kubelet isn't running or healthy.
    59. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
    60. [kubelet-check] It seems like the kubelet isn't running or healthy.
    61. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
    62. [kubelet-check] It seems like the kubelet isn't running or healthy.
    63. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
    64. [kubelet-check] It seems like the kubelet isn't running or healthy.
    65. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
    67. 	Unfortunately, an error has occurred:
    68. 		timed out waiting for the condition
    70. 	This error is likely caused by:
    71. 		- The kubelet is not running
    72. 		- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
    74. 	If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
    75. 		- 'systemctl status kubelet'
    76. 		- 'journalctl -xeu kubelet'
    78. 	Additionally, a control plane component may have crashed or exited when started by the container runtime.
    79. 	To troubleshoot, list all containers using your preferred container runtimes CLI.
    81. 	Here is one example how you may list all Kubernetes containers running in docker:
    82. 		- 'docker ps -a | grep kube | grep -v pause'
    83. 		Once you have found the failing container, you can inspect its logs with:
    84. 		- 'docker logs CONTAINERID'
    87. stderr:
    88. W0911 11:26:38.783101   14450 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
    89. 	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    90. 	[WARNING Swap]: running with swap on is not supported. Please disable swap
    91. 	[WARNING FileExisting-socat]: socat not found in system path
    92. 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
    93. W0911 11:26:48.464749   14450 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
    94. W0911 11:26:48.466754   14450 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
    95. error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
    96. To see the stack trace of this error execute with --v=5 or higher
    97.  
    98.   - Generating certificates and keys ...
    99.   - Booting up control plane ...
    100.   - Configuring RBAC rules ...
    101. * Configuring local host environment ...
    102. * 
    103. ! The 'none' driver is designed for experts who need to integrate with an existing VM
    104. * Most users should use the newer 'docker' driver instead, which does not require root!
    105. * For more information, see: https://minikube.sigs.k8s.io/docs/reference/drivers/none/
    106. * 
    107. ! kubectl and minikube configuration will be stored in /root
    108. ! To use kubectl or minikube commands as your own user, you may need to relocate them. For example, to overwrite your own settings, run:
    109. * 
    110.   - sudo mv /root/.kube /root/.minikube $HOME
    111.   - sudo chown -R $USER $HOME/.kube $HOME/.minikube
    112. * 
    113. * This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true
    114. * Verifying Kubernetes components...
    115.   - Using image gcr.io/k8s-minikube/storage-provisioner:v5
    116. * Enabled addons: storage-provisioner, default-storageclass
    117. * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

    minikube的停止和删除： 

    如果这个集群想停止的话，那么命令就非常简单了：

    1. minkube stop
    2. 输出如下；
    3. * Stopping "minikube" in none ...
    4. * Node "minikube" stopped.

    如果重启了服务器，那么，只需要参数换成start就可以再次启动minikube了。删除minkube也非常简单，参数换成 delete即可，这个删除会将配置文件什么的都给删除掉，前提这些文件是minikube自己建立的，否则不会删除。

    start的输出：

    1. [root@node3 manifests]# minikube start
    2. * minikube v1.12.0 on Centos 7.4.1708
    3. * Using the none driver based on existing profile
    4. * Starting control plane node minikube in cluster minikube
    5. * Restarting existing none bare metal machine for "minikube" ...
    6. * OS release is CentOS Linux 7 (Core)
    7. * Preparing Kubernetes v1.18.8 on Docker 19.03.9 ...
    8. * Configuring local host environment ...
    9. * 
    10. ! The 'none' driver is designed for experts who need to integrate with an existing VM
    11. * Most users should use the newer 'docker' driver instead, which does not require root!
    12. * For more information, see: https://minikube.sigs.k8s.io/docs/reference/drivers/none/
    13. * 
    14. ! kubectl and minikube configuration will be stored in /root
    15. ! To use kubectl or minikube commands as your own user, you may need to relocate them. For example, to overwrite your own settings, run:
    16. * 
    17.   - sudo mv /root/.kube /root/.minikube $HOME
    18.   - sudo chown -R $USER $HOME/.kube $HOME/.minikube
    19. * 
    20. * This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true
    21. * Verifying Kubernetes components...
    22. * Enabled addons: default-storageclass, storage-provisioner
    23. * Done! kubectl is now configured to use "minikube"

     

    以上的输出表明kubernetes单节点集群已经安装成功了，但有一些警告需要处理：

    （1）

    关于kubeadmin，kubelet，kubectl这三个命令的缓存：

    1.     > kubectl.sha256:  65 B / 65 B [-------------------------] 100.00% ? p/s 0s
    2.     > kubelet:  108.05 MiB / 108.05 MiB [--------] 100.00% 639.49 KiB p/s 2m53s

    这几个命令是下载到/root/.minikube/cache/linux/amd64/v1.18.8/这个目录下的，因此，想要提高速度，离线化部署就需要这么做：

    建立以上的目录：

    mkdir -p /root/.minikube/cache/linux/amd64/v1.18.8/

    给文件赋予权限并拷贝文件到这个目录下：

    1. chmod a+x kube*  #赋予权限
    2.  
    3.  
    4. [root@node3 v1.18.8]# pwd
    5. /root/.minikube/cache/linux/amd64/v1.18.8
    6.  
    7.  
    8. [root@slave3 v1.18.8]# ll
    9. total 192544
    10. -rwxr-xr-x 1 root root  39821312 Sep 11 11:24 kubeadm
    11. -rwxr-xr-x 1 root root  44040192 Sep 11 11:24 kubectl
    12. -rwxr-xr-x 1 root root 113300248 Sep 11 11:26 kubelet

    （2）

    集群健康检查报错的解决方案：

    1. [root@slave3 ~]# kubectl get cs
    2. NAME                 STATUS      MESSAGE                                                                                     ERROR
    3. controller-manager   Unhealthy   Get http://127.0.0.1:10252/healthz: dial tcp 127.0.0.1:10252: connect: connection refused   
    4. scheduler            Unhealthy   Get http://127.0.0.1:10251/healthz: dial tcp 127.0.0.1:10251: connect: connection refused   
    5. etcd-0               Healthy     {"health":"true"}

    解决方案：

    删除/etc/kubernetes/manifests/kube-scheduler.yaml和/etc/kubernetes/manifests/kube-controller-manager.yaml两个文件内的--port=0 这个字段

    稍等片刻，再次查询就正常了：

    1. [root@slave3 ~]# kubectl get cs
    2. NAME                 STATUS    MESSAGE             ERROR
    3. controller-manager   Healthy   ok                  
    4. scheduler            Healthy   ok                  
    5. etcd-0               Healthy   {"health":"true"}

    三，

    dashboard的安装

    1. [root@slave3 ~]# minikube dashboard
    2. * Enabling dashboard ...
    3.   - Using image kubernetesui/metrics-scraper:v1.0.8
    4.   - Using image kubernetesui/dashboard:v2.6.0
    5. * Verifying dashboard health ...
    6. * Launching proxy ...
    7. * Verifying proxy health ...
    8. http://127.0.0.1:32844/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/

    设置代理：

    1. [root@slave3 v1.18.8]# kubectl proxy --port=45396 --address='0.0.0.0' --disable-filter=true --accept-hosts='^.*' 
    2. W0911 12:49:38.664081    8709 proxy.go:167] Request filter disabled, your proxy is vulnerable to XSRF attacks, please be cautious
    3. Starting to serve on [::]:45396

    浏览器登录网址：

    本机IP是192.168.217.11，和上面的http://127.0.0.1:32844/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/
    拼接就好了

    http://192.168.217.11:45396/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/

    至此，minikube就安装完了。

    附录：

    关于addons

    可以看到有安装StorageClass，但很多addons还没有安装

    1. [root@slave3 v1.18.8]# minikube addons list
    2. |-----------------------------|----------|--------------|--------------------------------|
    3. |         ADDON NAME          | PROFILE  |    STATUS    |           MAINTAINER           |
    4. |-----------------------------|----------|--------------|--------------------------------|
    5. | ambassador                  | minikube | disabled     | 3rd party (Ambassador)         |
    6. | auto-pause                  | minikube | disabled     | Google                         |
    7. | csi-hostpath-driver         | minikube | disabled     | Kubernetes                     |
    8. | dashboard                   | minikube | enabled ✅   | Kubernetes                     |
    9. | default-storageclass        | minikube | enabled ✅   | Kubernetes                     |
    10. | efk                         | minikube | disabled     | 3rd party (Elastic)            |
    11. | freshpod                    | minikube | disabled     | Google                         |
    12. | gcp-auth                    | minikube | disabled     | Google                         |
    13. | gvisor                      | minikube | disabled     | Google                         |
    14. | headlamp                    | minikube | disabled     | 3rd party (kinvolk.io)         |
    15. | helm-tiller                 | minikube | disabled     | 3rd party (Helm)               |
    16. | inaccel                     | minikube | disabled     | 3rd party (InAccel             |
    17. |                             |          |              | [info@inaccel.com])            |
    18. | ingress                     | minikube | disabled     | Kubernetes                     |
    19. | ingress-dns                 | minikube | disabled     | Google                         |
    20. | istio                       | minikube | disabled     | 3rd party (Istio)              |
    21. | istio-provisioner           | minikube | disabled     | 3rd party (Istio)              |
    22. | kong                        | minikube | disabled     | 3rd party (Kong HQ)            |
    23. | kubevirt                    | minikube | disabled     | 3rd party (KubeVirt)           |
    24. | logviewer                   | minikube | disabled     | 3rd party (unknown)            |
    25. | metallb                     | minikube | disabled     | 3rd party (MetalLB)            |
    26. | metrics-server              | minikube | disabled     | Kubernetes                     |
    27. | nvidia-driver-installer     | minikube | disabled     | Google                         |
    28. | nvidia-gpu-device-plugin    | minikube | disabled     | 3rd party (Nvidia)             |
    29. | olm                         | minikube | disabled     | 3rd party (Operator Framework) |
    30. | pod-security-policy         | minikube | disabled     | 3rd party (unknown)            |
    31. | portainer                   | minikube | disabled     | 3rd party (Portainer.io)       |
    32. | registry                    | minikube | disabled     | Google                         |
    33. | registry-aliases            | minikube | disabled     | 3rd party (unknown)            |
    34. | registry-creds              | minikube | disabled     | 3rd party (UPMC Enterprises)   |
    35. | storage-provisioner         | minikube | enabled ✅   | Google                         |
    36. | storage-provisioner-gluster | minikube | disabled     | 3rd party (Gluster)            |
    37. | volumesnapshots             | minikube | disabled     | Kubernetes                     |
    38. |-----------------------------|----------|--------------|--------------------------------|

     以安装ingress为例（安装的同时，输出安装的错误日志）：

    1. [root@slave3 v1.18.8]# minikube addons enable ingress --alsologtostderr
    2. I0911 13:09:08.559523   14428 out.go:296] Setting OutFile to fd 1 ...
    3. I0911 13:09:08.572541   14428 out.go:343] TERM=xterm,COLORTERM=, which probably does not support color
    4. I0911 13:09:08.572593   14428 out.go:309] Setting ErrFile to fd 2...
    5. I0911 13:09:08.572609   14428 out.go:343] TERM=xterm,COLORTERM=, which probably does not support color
    6. I0911 13:09:08.572908   14428 root.go:333] Updating PATH: /root/.minikube/bin
    7. I0911 13:09:08.577988   14428 out.go:177] * ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
    8. You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
    9. * ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
    10. You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
    11. I0911 13:09:08.580137   14428 config.go:180] Loaded profile config "minikube": Driver=none, ContainerRuntime=docker, KubernetesVersion=v1.18.8
    12. I0911 13:09:08.580198   14428 addons.go:65] Setting ingress=true in profile "minikube"
    13. I0911 13:09:08.580243   14428 addons.go:153] Setting addon ingress=true in "minikube"
    14. I0911 13:09:08.580572   14428 host.go:66] Checking if "minikube" exists ...
    15. I0911 13:09:08.581080   14428 exec_runner.go:51] Run: systemctl --version
    16. I0911 13:09:08.584877   14428 kubeconfig.go:92] found "minikube" server: "https://192.168.217.136:8443"
    17. I0911 13:09:08.584942   14428 api_server.go:165] Checking apiserver status ...
    18. I0911 13:09:08.584982   14428 exec_runner.go:51] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
    19. I0911 13:09:08.611630   14428 exec_runner.go:51] Run: sudo egrep ^[0-9]+:freezer: /proc/15576/cgroup
    20. I0911 13:09:08.626851   14428 api_server.go:181] apiserver freezer: "9:freezer:/kubepods/burstable/pod1a4a24f29bac3cef528a8b328b9798b5/c8a589a612154591de984664d86a3ad96a449f3d0b1145527ceea9c5ed267124"
    21. I0911 13:09:08.626952   14428 exec_runner.go:51] Run: sudo cat /sys/fs/cgroup/freezer/kubepods/burstable/pod1a4a24f29bac3cef528a8b328b9798b5/c8a589a612154591de984664d86a3ad96a449f3d0b1145527ceea9c5ed267124/freezer.state
    22. I0911 13:09:08.638188   14428 api_server.go:203] freezer state: "THAWED"
    23. I0911 13:09:08.638329   14428 api_server.go:240] Checking apiserver healthz at https://192.168.217.136:8443/healthz ...
    24. I0911 13:09:08.649018   14428 api_server.go:266] https://192.168.217.136:8443/healthz returned 200:
    25. ok
    26. I0911 13:09:08.650082   14428 out.go:177]   - Using image k8s.gcr.io/ingress-nginx/controller:v0.49.3
    27.   - Using image k8s.gcr.io/ingress-nginx/controller:v0.49.3
    28. I0911 13:09:08.652268   14428 out.go:177]   - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1
    29.   - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1
    30. I0911 13:09:08.653129   14428 out.go:177]   - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1
    31.   - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1
    32. I0911 13:09:08.654440   14428 addons.go:345] installing /etc/kubernetes/addons/ingress-deploy.yaml
    33. I0911 13:09:08.654528   14428 exec_runner.go:151] cp: memory --> /etc/kubernetes/addons/ingress-deploy.yaml (15118 bytes)
    34. I0911 13:09:08.654720   14428 exec_runner.go:51] Run: sudo cp -a /tmp/minikube4099945938 /etc/kubernetes/addons/ingress-deploy.yaml
    35. I0911 13:09:08.668351   14428 exec_runner.go:51] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.8/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml
    36. I0911 13:09:09.748481   14428 exec_runner.go:84] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.8/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml: (1.080019138s)
    37. I0911 13:09:09.748552   14428 addons.go:383] Verifying addon ingress=true in "minikube"
    38. I0911 13:09:09.751805   14428 out.go:177] * Verifying ingress addon...

    可以看到，安装的时候使用的资源清单文件是这个：

    sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.8/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml

    该文件内容非常多，但，由于是使用的国外的镜像网址，因此，一般是不会安装成功的。

    解决方案为查找里面涉及的images，替换为国内可下载的镜像即可。 

    还有一个权限问题，可能会报错：

    F0911 05:24:52.171825       6 ssl.go:389] unexpected error storing fake SSL Cert: could not create PEM certificate file /etc/ingress-controller/ssl/default-fake-certificate.pem: open /etc/ingress-controller/ssl/default-fake-certificate.pem: permission denied

    解决方案是：

    还是编辑下面这个文件， runAsUser 的值修改为33 

    重新apply 此文件：

    kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml

    1. [root@slave3 v1.18.8]# cat /etc/kubernetes/addons/ingress-deploy.yaml
    2. # Copyright 2021 The Kubernetes Authors All rights reserved.
    3. #
    4. # Licensed under the Apache License, Version 2.0 (the "License");
    5. # you may not use this file except in compliance with the License.
    6. # You may obtain a copy of the License at
    7. #
    8. #     http://www.apache.org/licenses/LICENSE-2.0
    9. #
    10. # Unless required by applicable law or agreed to in writing, software
    11. # distributed under the License is distributed on an "AS IS" BASIS,
    12. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    13. # See the License for the specific language governing permissions and
    14. # limitations under the License.
    15.  
    16. # ref: https://github.com/kubernetes/ingress-nginx/blob/main/deploy/static/provider/kind/deploy.yaml
    17.  
    18. apiVersion: v1
    19. kind: Namespace
    20. metadata:
    21.   labels:
    22.     app.kubernetes.io/instance: ingress-nginx
    23.     app.kubernetes.io/name: ingress-nginx
    24.   name: ingress-nginx
    25. ---
    26. apiVersion: v1
    27. automountServiceAccountToken: true
    28. kind: ServiceAccount
    29. metadata:
    30.   labels:
    31.     app.kubernetes.io/component: controller
    32.     app.kubernetes.io/instance: ingress-nginx
    33.     app.kubernetes.io/name: ingress-nginx
    34.   name: ingress-nginx
    35.   namespace: ingress-nginx
    36. ---
    37. apiVersion: v1
    38. kind: ServiceAccount
    39. metadata:
    40.   labels:
    41.     app.kubernetes.io/component: admission-webhook
    42.     app.kubernetes.io/instance: ingress-nginx
    43.     app.kubernetes.io/name: ingress-nginx
    44.   name: ingress-nginx-admission
    45.   namespace: ingress-nginx
    46. ---
    47. apiVersion: rbac.authorization.k8s.io/v1
    48. kind: Role
    49. metadata:
    50.   labels:
    51.     app.kubernetes.io/component: controller
    52.     app.kubernetes.io/instance: ingress-nginx
    53.     app.kubernetes.io/name: ingress-nginx
    54.   name: ingress-nginx
    55.   namespace: ingress-nginx
    56. rules:
    57. - apiGroups:
    58.   - ""
    59.   resources:
    60.   - namespaces
    61.   verbs:
    62.   - get
    63. - apiGroups:
    64.   - ""
    65.   resources:
    66.   - configmaps
    67.   - pods
    68.   - secrets
    69.   - endpoints
    70.   verbs:
    71.   - get
    72.   - list
    73.   - watch
    74. - apiGroups:
    75.   - ""
    76.   resources:
    77.   - services
    78.   verbs:
    79.   - get
    80.   - list
    81.   - watch
    82. - apiGroups:
    83.   - extensions
    84.   - networking.k8s.io
    85.   resources:
    86.   - ingresses
    87.   verbs:
    88.   - get
    89.   - list
    90.   - watch
    91. - apiGroups:
    92.   - extensions
    93.   - networking.k8s.io
    94.   resources:
    95.   - ingresses/status
    96.   verbs:
    97.   - update
    98. - apiGroups:
    99.   - networking.k8s.io
    100.   resources:
    101.   - ingressclasses
    102.   verbs:
    103.   - get
    104.   - list
    105.   - watch
    106. - apiGroups:
    107.   - ""
    108.   resourceNames:
    109.   - ingress-controller-leader
    110.   resources:
    111.   - configmaps
    112.   verbs:
    113.   - get
    114.   - update
    115. - apiGroups:
    116.   - ""
    117.   resources:
    118.   - configmaps
    119.   verbs:
    120.   - create
    121. - apiGroups:
    122.   - ""
    123.   resources:
    124.   - events
    125.   verbs:
    126.   - create
    127.   - patch
    128. ---
    129. apiVersion: rbac.authorization.k8s.io/v1
    130. kind: Role
    131. metadata:
    132.   labels:
    133.     app.kubernetes.io/component: admission-webhook
    134.     app.kubernetes.io/instance: ingress-nginx
    135.     app.kubernetes.io/name: ingress-nginx
    136.   name: ingress-nginx-admission
    137.   namespace: ingress-nginx
    138. rules:
    139. - apiGroups:
    140.   - ""
    141.   resources:
    142.   - secrets
    143.   verbs:
    144.   - get
    145.   - create
    146. ---
    147. apiVersion: rbac.authorization.k8s.io/v1
    148. kind: ClusterRole
    149. metadata:
    150.   labels:
    151.     app.kubernetes.io/instance: ingress-nginx
    152.     app.kubernetes.io/name: ingress-nginx
    153.   name: ingress-nginx
    154. rules:
    155. - apiGroups:
    156.   - ""
    157.   resources:
    158.   - configmaps
    159.   - endpoints
    160.   - nodes
    161.   - pods
    162.   - secrets
    163.   - namespaces
    164.   verbs:
    165.   - list
    166.   - watch
    167. - apiGroups:
    168.   - ""
    169.   resources:
    170.   - nodes
    171.   verbs:
    172.   - get
    173. - apiGroups:
    174.   - ""
    175.   resources:
    176.   - services
    177.   verbs:
    178.   - get
    179.   - list
    180.   - watch
    181. - apiGroups:
    182.   - extensions
    183.   - networking.k8s.io
    184.   resources:
    185.   - ingresses
    186.   verbs:
    187.   - get
    188.   - list
    189.   - watch
    190. - apiGroups:
    191.   - ""
    192.   resources:
    193.   - events
    194.   verbs:
    195.   - create
    196.   - patch
    197. - apiGroups:
    198.   - extensions
    199.   - networking.k8s.io
    200.   resources:
    201.   - ingresses/status
    202.   verbs:
    203.   - update
    204. - apiGroups:
    205.   - networking.k8s.io
    206.   resources:
    207.   - ingressclasses
    208.   verbs:
    209.   - get
    210.   - list
    211.   - watch
    212. ---
    213. apiVersion: rbac.authorization.k8s.io/v1
    214. kind: ClusterRole
    215. metadata:
    216.   labels:
    217.     app.kubernetes.io/component: admission-webhook
    218.     app.kubernetes.io/instance: ingress-nginx
    219.     app.kubernetes.io/name: ingress-nginx
    220.   name: ingress-nginx-admission
    221. rules:
    222. - apiGroups:
    223.   - admissionregistration.k8s.io
    224.   resources:
    225.   - validatingwebhookconfigurations
    226.   verbs:
    227.   - get
    228.   - update
    229. ---
    230. apiVersion: rbac.authorization.k8s.io/v1
    231. kind: RoleBinding
    232. metadata:
    233.   labels:
    234.     app.kubernetes.io/component: controller
    235.     app.kubernetes.io/instance: ingress-nginx
    236.     app.kubernetes.io/name: ingress-nginx
    237.   name: ingress-nginx
    238.   namespace: ingress-nginx
    239. roleRef:
    240.   apiGroup: rbac.authorization.k8s.io
    241.   kind: Role
    242.   name: ingress-nginx
    243. subjects:
    244. - kind: ServiceAccount
    245.   name: ingress-nginx
    246.   namespace: ingress-nginx
    247. ---
    248. apiVersion: rbac.authorization.k8s.io/v1
    249. kind: RoleBinding
    250. metadata:
    251.   labels:
    252.     app.kubernetes.io/component: admission-webhook
    253.     app.kubernetes.io/instance: ingress-nginx
    254.     app.kubernetes.io/name: ingress-nginx
    255.   name: ingress-nginx-admission
    256.   namespace: ingress-nginx
    257. roleRef:
    258.   apiGroup: rbac.authorization.k8s.io
    259.   kind: Role
    260.   name: ingress-nginx-admission
    261. subjects:
    262. - kind: ServiceAccount
    263.   name: ingress-nginx-admission
    264.   namespace: ingress-nginx
    265. ---
    266. apiVersion: rbac.authorization.k8s.io/v1
    267. kind: ClusterRoleBinding
    268. metadata:
    269.   labels:
    270.     app.kubernetes.io/instance: ingress-nginx
    271.     app.kubernetes.io/name: ingress-nginx
    272.   name: ingress-nginx
    273. roleRef:
    274.   apiGroup: rbac.authorization.k8s.io
    275.   kind: ClusterRole
    276.   name: ingress-nginx
    277. subjects:
    278. - kind: ServiceAccount
    279.   name: ingress-nginx
    280.   namespace: ingress-nginx
    281. ---
    282. apiVersion: rbac.authorization.k8s.io/v1
    283. kind: ClusterRoleBinding
    284. metadata:
    285.   labels:
    286.     app.kubernetes.io/component: admission-webhook
    287.     app.kubernetes.io/instance: ingress-nginx
    288.     app.kubernetes.io/name: ingress-nginx
    289.   name: ingress-nginx-admission
    290. roleRef:
    291.   apiGroup: rbac.authorization.k8s.io
    292.   kind: ClusterRole
    293.   name: ingress-nginx-admission
    294. subjects:
    295. - kind: ServiceAccount
    296.   name: ingress-nginx-admission
    297.   namespace: ingress-nginx
    298. ---
    299. apiVersion: v1
    300. data:
    301.   # see https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/configmap.md for all possible options and their description
    302.   hsts: "false"
    303.   # see https://github.com/kubernetes/minikube/pull/12702#discussion_r727519180: 'allow-snippet-annotations' should be used only if strictly required by another part of the deployment
    304. #  allow-snippet-annotations: "true"
    305. kind: ConfigMap
    306. metadata:
    307.   labels:
    308.     app.kubernetes.io/component: controller
    309.     app.kubernetes.io/instance: ingress-nginx
    310.     app.kubernetes.io/name: ingress-nginx
    311.   name: ingress-nginx-controller
    312.   namespace: ingress-nginx
    313. ---
    314. apiVersion: v1
    315. kind: ConfigMap
    316. metadata:
    317.   name: tcp-services
    318.   namespace: ingress-nginx
    319.   labels:
    320.     app.kubernetes.io/name: ingress-nginx
    321.     app.kubernetes.io/instance: ingress-nginx
    322.     app.kubernetes.io/component: controller
    323. ---
    324. apiVersion: v1
    325. kind: ConfigMap
    326. metadata:
    327.   name: udp-services
    328.   namespace: ingress-nginx
    329.   labels:
    330.     app.kubernetes.io/name: ingress-nginx
    331.     app.kubernetes.io/instance: ingress-nginx
    332.     app.kubernetes.io/component: controller
    333. ---
    334. apiVersion: v1
    335. kind: Service
    336. metadata:
    337.   labels:
    338.     app.kubernetes.io/component: controller
    339.     app.kubernetes.io/instance: ingress-nginx
    340.     app.kubernetes.io/name: ingress-nginx
    341.   name: ingress-nginx-controller
    342.   namespace: ingress-nginx
    343. spec:
    344.   ports:
    345.   - name: http
    346.     port: 80
    347.     protocol: TCP
    348.     targetPort: http
    349.   - name: https
    350.     port: 443
    351.     protocol: TCP
    352.     targetPort: https
    353.   selector:
    354.     app.kubernetes.io/component: controller
    355.     app.kubernetes.io/instance: ingress-nginx
    356.     app.kubernetes.io/name: ingress-nginx
    357.   type: NodePort
    358. ---
    359. apiVersion: v1
    360. kind: Service
    361. metadata:
    362.   labels:
    363.     app.kubernetes.io/component: controller
    364.     app.kubernetes.io/instance: ingress-nginx
    365.     app.kubernetes.io/name: ingress-nginx
    366.   name: ingress-nginx-controller-admission
    367.   namespace: ingress-nginx
    368. spec:
    369.   ports:
    370.   - name: https-webhook
    371.     port: 443
    372.     targetPort: webhook
    373.   selector:
    374.     app.kubernetes.io/component: controller
    375.     app.kubernetes.io/instance: ingress-nginx
    376.     app.kubernetes.io/name: ingress-nginx
    377.   type: ClusterIP
    378. ---
    379. apiVersion: apps/v1
    380. kind: Deployment
    381. metadata:
    382.   labels:
    383.     app.kubernetes.io/component: controller
    384.     app.kubernetes.io/instance: ingress-nginx
    385.     app.kubernetes.io/name: ingress-nginx
    386.   name: ingress-nginx-controller
    387.   namespace: ingress-nginx
    388. spec:
    389.   minReadySeconds: 0
    390.   revisionHistoryLimit: 10
    391.   selector:
    392.     matchLabels:
    393.       app.kubernetes.io/component: controller
    394.       app.kubernetes.io/instance: ingress-nginx
    395.       app.kubernetes.io/name: ingress-nginx
    396.   strategy:
    397.     rollingUpdate:
    398.       maxUnavailable: 1
    399.     type: RollingUpdate
    400.   template:
    401.     metadata:
    402.       labels:
    403.         app.kubernetes.io/component: controller
    404.         app.kubernetes.io/instance: ingress-nginx
    405.         app.kubernetes.io/name: ingress-nginx
    406.         gcp-auth-skip-secret: "true"
    407.     spec:
    408.       containers:
    409.       - args:
    410.         - /nginx-ingress-controller
    411.         - --election-id=ingress-controller-leader
    412.         - --ingress-class=nginx
    413.         - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
    414.         - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
    415.         - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
    416.         - --validating-webhook=:8443
    417.         - --validating-webhook-certificate=/usr/local/certificates/cert
    418.         - --validating-webhook-key=/usr/local/certificates/key
    419.         env:
    420.         - name: POD_NAME
    421.           valueFrom:
    422.             fieldRef:
    423.               fieldPath: metadata.name
    424.         - name: POD_NAMESPACE
    425.           valueFrom:
    426.             fieldRef:
    427.               fieldPath: metadata.namespace
    428.         - name: LD_PRELOAD
    429.           value: /usr/local/lib/libmimalloc.so
    430.         image: k8s.gcr.io/ingress-nginx/controller:v0.49.3@sha256:35fe394c82164efa8f47f3ed0be981b3f23da77175bbb8268a9ae438851c8324
    431.         imagePullPolicy: IfNotPresent
    432.         lifecycle:
    433.           preStop:
    434.             exec:
    435.               command:
    436.               - /wait-shutdown
    437.         livenessProbe:
    438.           failureThreshold: 5
    439.           httpGet:
    440.             path: /healthz
    441.             port: 10254
    442.             scheme: HTTP
    443.           initialDelaySeconds: 10
    444.           periodSeconds: 10
    445.           successThreshold: 1
    446.           timeoutSeconds: 1
    447.         name: controller
    448.         ports:
    449.         - containerPort: 80
    450.           hostPort: 80
    451.           name: http
    452.           protocol: TCP
    453.         - containerPort: 443
    454.           hostPort: 443
    455.           name: https
    456.           protocol: TCP
    457.         - containerPort: 8443
    458.           name: webhook
    459.           protocol: TCP
    460.         readinessProbe:
    461.           failureThreshold: 3
    462.           httpGet:
    463.             path: /healthz
    464.             port: 10254
    465.             scheme: HTTP
    466.           initialDelaySeconds: 10
    467.           periodSeconds: 10
    468.           successThreshold: 1
    469.           timeoutSeconds: 1
    470.         resources:
    471.           requests:
    472.             cpu: 100m
    473.             memory: 90Mi
    474.         securityContext:
    475.           allowPrivilegeEscalation: true
    476.           capabilities:
    477.             add:
    478.             - NET_BIND_SERVICE
    479.             drop:
    480.             - ALL
    481.           runAsUser: 101
    482.         volumeMounts:
    483.         - mountPath: /usr/local/certificates/
    484.           name: webhook-cert
    485.           readOnly: true
    486.       dnsPolicy: ClusterFirst
    487.       nodeSelector:
    488.         minikube.k8s.io/primary: "true"
    489.         kubernetes.io/os: linux
    490.       serviceAccountName: ingress-nginx
    491.       terminationGracePeriodSeconds: 0
    492.       tolerations:
    493.       - effect: NoSchedule
    494.         key: node-role.kubernetes.io/master
    495.         operator: Equal
    496.       volumes:
    497.       - name: webhook-cert
    498.         secret:
    499.           secretName: ingress-nginx-admission
    500. ---
    501. apiVersion: batch/v1
    502. kind: Job
    503. metadata:
    504.   labels:
    505.     app.kubernetes.io/component: admission-webhook
    506.     app.kubernetes.io/instance: ingress-nginx
    507.     app.kubernetes.io/name: ingress-nginx
    508.   name: ingress-nginx-admission-create
    509.   namespace: ingress-nginx
    510. spec:
    511.   template:
    512.     metadata:
    513.       labels:
    514.         app.kubernetes.io/component: admission-webhook
    515.         app.kubernetes.io/instance: ingress-nginx
    516.         app.kubernetes.io/name: ingress-nginx
    517.       name: ingress-nginx-admission-create
    518.     spec:
    519.       containers:
    520.       - args:
    521.         - create
    522.         - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
    523.         - --namespace=$(POD_NAMESPACE)
    524.         - --secret-name=ingress-nginx-admission
    525.         env:
    526.         - name: POD_NAMESPACE
    527.           valueFrom:
    528.             fieldRef:
    529.               fieldPath: metadata.namespace
    530.         image: docker.io/jettech/kube-webhook-certgen:v1.5.1@sha256:950833e19ade18cd389d647efb88992a7cc077abedef343fa59e012d376d79b7
    531.         imagePullPolicy: IfNotPresent
    532.         name: create
    533.         securityContext:
    534.           allowPrivilegeEscalation: false
    535.       nodeSelector:
    536.         minikube.k8s.io/primary: "true"
    537.         kubernetes.io/os: linux
    538.       restartPolicy: OnFailure
    539.       securityContext:
    540.         runAsNonRoot: true
    541.         runAsUser: 2000
    542.       serviceAccountName: ingress-nginx-admission
    543. ---
    544. apiVersion: batch/v1
    545. kind: Job
    546. metadata:
    547.   labels:
    548.     app.kubernetes.io/component: admission-webhook
    549.     app.kubernetes.io/instance: ingress-nginx
    550.     app.kubernetes.io/name: ingress-nginx
    551.   name: ingress-nginx-admission-patch
    552.   namespace: ingress-nginx
    553. spec:
    554.   template:
    555.     metadata:
    556.       labels:
    557.         app.kubernetes.io/component: admission-webhook
    558.         app.kubernetes.io/instance: ingress-nginx
    559.         app.kubernetes.io/name: ingress-nginx
    560.       name: ingress-nginx-admission-patch
    561.     spec:
    562.       containers:
    563.       - args:
    564.         - patch
    565.         - --webhook-name=ingress-nginx-admission
    566.         - --namespace=$(POD_NAMESPACE)
    567.         - --patch-mutating=false
    568.         - --secret-name=ingress-nginx-admission
    569.         - --patch-failure-policy=Fail
    570.         env:
    571.         - name: POD_NAMESPACE
    572.           valueFrom:
    573.             fieldRef:
    574.               fieldPath: metadata.namespace
    575.         image: docker.io/jettech/kube-webhook-certgen:v1.5.1@sha256:950833e19ade18cd389d647efb88992a7cc077abedef343fa59e012d376d79b7
    576.         imagePullPolicy: IfNotPresent
    577.         name: patch
    578.         securityContext:
    579.           allowPrivilegeEscalation: false
    580.       nodeSelector:
    581.         minikube.k8s.io/primary: "true"
    582.         kubernetes.io/os: linux
    583.       restartPolicy: OnFailure
    584.       securityContext:
    585.         runAsNonRoot: true
    586.         runAsUser: 2000
    587.       serviceAccountName: ingress-nginx-admission
    588. ---
    589. apiVersion: admissionregistration.k8s.io/v1
    590. kind: ValidatingWebhookConfiguration
    591. metadata:
    592.   labels:
    593.     app.kubernetes.io/component: admission-webhook
    594.     app.kubernetes.io/instance: ingress-nginx
    595.     app.kubernetes.io/name: ingress-nginx
    596.   name: ingress-nginx-admission
    597. webhooks:
    598. - admissionReviewVersions:
    599.   - v1
    600.   - v1beta1
    601.   clientConfig:
    602.     service:
    603.       name: ingress-nginx-controller-admission
    604.       namespace: ingress-nginx
    605.       path: /networking/v1beta1/ingresses
    606.   failurePolicy: Fail
    607.   matchPolicy: Equivalent
    608.   name: validate.nginx.ingress.kubernetes.io
    609.   rules:
    610.   - apiGroups:
    611.     - networking.k8s.io
    612.     apiVersions:
    613.     - v1beta1
    614.     operations:
    615.     - CREATE
    616.     - UPDATE
    617.     resources:
    618.     - ingresses
    619.   sideEffects: None

    安装完毕后可以看到：

    1. [root@slave3 v1.18.8]# kubectl get all -n ingress-nginx
    2. NAME                                            READY   STATUS      RESTARTS   AGE
    3. pod/ingress-nginx-admission-create-n5hc5        0/1     Completed   0          28m
    4. pod/ingress-nginx-admission-patch-cgzl9         0/1     Completed   0          28m
    5. pod/ingress-nginx-controller-54b856d6d7-7fr7q   1/1     Running     0          9m54s
    6.  
    7. NAME                                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
    8. service/ingress-nginx-controller             NodePort    10.107.186.74           80:31411/TCP,443:32683/TCP   28m
    9. service/ingress-nginx-controller-admission   ClusterIP   10.106.184.40           443/TCP                      28m
    10.  
    11. NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
    12. deployment.apps/ingress-nginx-controller   1/1     1            1           28m
    13.  
    14. NAME                                                  DESIRED   CURRENT   READY   AGE
    15. replicaset.apps/ingress-nginx-controller-54b856d6d7   1         1         1       9m54s
    16. replicaset.apps/ingress-nginx-controller-7689b8b4f9   0         0         0       17m
    17. replicaset.apps/ingress-nginx-controller-77cc874b76   0         0         0       28m
    18.  
    19. NAME                                       COMPLETIONS   DURATION   AGE
    20. job.batch/ingress-nginx-admission-create   1/1           21s        28m
    21. job.batch/ingress-nginx-admission-patch    1/1           22s        28m
    22. [root@slave3 v1.18.8]#

    addons里的ingress就安装好啦。

  • 相关阅读:
    2022腾讯云年终双十一配置表汇总-云服务器CVM+轻量应用服务器
    代码随想录——钥匙和房间（图论）
    AidLux OS搭建web站点
    mysql主从复制与读写分离
    代理，反射，AOP
    数据库：Centos7安装解压版mysql5.7图文教程，亲测成功
    【微前端】single-spa 到底是个什么鬼
    Obsidian配置
    P5661 [CSP-J2019] 公交换乘
    Python学习基础笔记二十一——迭代器
  • 原文地址：https://blog.csdn.net/alwaysbefine/article/details/126802883