• 通过cri-o部署k8s集群环境

    目录

    一.基础环境配置（每个节点都做）

    1.hosts解析

    2.防火墙和selinux

    3.安装基本软件并配置时间同步

    4.禁用swap分区

    5.更改内核参数

    6.配置ipvs

    7.k8s下载

    （1）配置镜像下载相关软件

    （2）配置kubelet上的cgroup

    二.下载cri-o包（所有节点做）

    1.在线下载

    2.本地下载

    三.配置文件（所有节点做）

    1.修改crio配置文件

    2.修改kubelet参数（所有节点做）

    3.重启crio和kubelet

    四.master节点初始化（master节点做）

    五.node节点加入（node节点做）

    六.添加网络插件（master节点做）

    1.应用calico.yaml

    2.查看验证

    七.配置kubectl命令补全

     

    192.168.2.190master
    192.168.2.191node2-191.com
    192.168.2.193node4-193.com

    一.基础环境配置（每个节点都做）

    1.hosts解析

    1. [root@master ~]# tail -3 /etc/hosts
    2. 192.168.2.190 master
    3. 192.168.2.191 node2-191.com
    4. 192.168.2.193 node4-193.com

    2.防火墙和selinux

    1. [root@master ~]# systemctl status firewalld.service;getenforce 
    2. ● firewalld.service - firewalld - dynamic firewall daemon
    3.    Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
    4.    Active: inactive (dead)
    5.      Docs: man:firewalld(1)
    6. Disabled
    7. #临时
    8. systemctl stop firewalld
    9. setenforce 0
    10. #禁用
    11. systemctl disable firewalld
    12. sed -i '/^SELINUX=/ c SELINUX=disabled' /etc/selinux/config

    3.安装基本软件并配置时间同步

    1. [root@master ~]# yum install -y wget tree bash-completion lrzsz psmisc net-tools vim chrony
    2. [root@master ~]# vim /etc/chrony.conf
    3. :3,6 s/^/#      #注释掉原有行
    4. server ntp1.aliyun.com iburst
    5. [root@node1-190 ~]# systemctl restart chronyd
    6. [root@node1-190 ~]# chronyc sources
    7. 210 Number of sources = 1
    8. MS Name/IP address         Stratum Poll Reach LastRx Last sample               
    9. ===============================================================================
    10. ^* 120.25.115.20                 2   8   341   431   -357us[ -771us] +/-   20ms

    4.禁用swap分区

    1. [root@master ~]# swapoff -a && sed -i 's/.*swap.*/#&/' /etc/fstab && free -m
    2.               total        used        free      shared  buff/cache   available
    3. Mem:          10376         943        8875          11         557        9178
    4. Swap:             0           0           0

    5.更改内核参数

    1. [root@node1-190 ~]# cat >> /etc/sysctl.d/k8s.conf << EOF
    2. vm.swappiness=0
    3. net.bridge.bridge-nf-call-ip6tables = 1
    4. net.bridge.bridge-nf-call-iptables = 1
    5. net.ipv4.ip_forward = 1
    6. EOF
    7. [root@node1-190 ~]# modprobe br_netfilter &&  modprobe overlay && sysctl -p /etc/sysctl.d/k8s.conf
    8. vm.swappiness = 0
    9. net.bridge.bridge-nf-call-ip6tables = 1
    10. net.bridge.bridge-nf-call-iptables = 1
    11. net.ipv4.ip_forward = 1

    6.配置ipvs

    1. [root@node1-190 ~]# yum install ipset ipvsadm -y
    2. [root@node1-190 ~]# cat <<EOF >  /etc/sysconfig/modules/ipvs.modules
    3. #!/bin/bash
    4. modprobe -- ip_vs
    5. modprobe -- ip_vs_rr
    6. modprobe -- ip_vs_wrr
    7. modprobe -- ip_vs_sh
    8. modprobe -- nf_conntrack
    9. EOF
    10. # 为脚本文件添加执行权限并运行，验证是否加载成功
    11. [root@node1-190 ~]# chmod +x /etc/sysconfig/modules/ipvs.modules && /bin/bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
    12. nf_conntrack_ipv4      15053  2 
    13. nf_defrag_ipv4         12729  1 nf_conntrack_ipv4
    14. ip_vs_sh               12688  0 
    15. ip_vs_wrr              12697  0 
    16. ip_vs_rr               12600  0 
    17. ip_vs                 145458  6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
    18. nf_conntrack          139264  7 ip_vs,nf_nat,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_ipv4
    19. libcrc32c              12644  4 xfs,ip_vs,nf_nat,nf_conntrack

    7.k8s下载

    （1）配置镜像下载相关软件

    1. [root@master ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
    2. [kubernetes]
    3. name=Kubernetes
    4. baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
    5. enabled=1
    6. gpgcheck=0
    7. repo_gpgcheck=0
    8. gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
    9. http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    10. EOF
    11. ​
    12. [root@master ~]# yum install -y kubeadm kubelet kubectl 
    13. [root@master ~]# kubeadm version
    14. kubeadm version: &version.Info{Major:"1", Minor:"28", GitVersion:"v1.28.2", GitCommit:"89a4ea3e1e4ddd7f7572286090359983e0387b2f", GitTreeState:"clean", BuildDate:"2023-09-13T09:34:32Z", GoVersion:"go1.20.8", Compiler:"gc", Platform:"linux/amd64"}

    （2）配置kubelet上的cgroup

    1. [root@master ~]# cat <<EOF > /etc/sysconfig/kubelet
    2. KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
    3. KUBE_PROXY_MODE="ipvs"
    4. EOF
    5. [root@master ~]# systemctl start kubelet
    6. [root@master ~]# systemctl enable kubelet
    7. Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.

    二.下载cri-o包（所有节点做）

    1.在线下载

    1. [root@master ~]# VERSION=1.25
    2. [root@master ~]# curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_7/devel:kubic:libcontainers:stable.repo
    3. ners:stable:cri-o:${VERSION}.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:${VERSION}/CentOS_7/devel:kubic:libcontainers:stable:cri-o:${VERSION}.repo  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
    4.                                  Dload  Upload   Total   Spent    Left  Speed
    5. 100   359  100   359    0     0    293      0  0:00:01  0:00:01 --:--:--   293
    6. [root@master ~]# curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:${VERSION}.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:${VERSION}/CentOS_7/devel:kubic:libcontainers:stable:cri-o:${VERSION}.repo
    7.   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
    8.                                  Dload  Upload   Total   Spent    Left  Speed
    9. 100   425  100   425    0     0    353      0  0:00:01  0:00:01 --:--:--   353
    10. 100   426  100   426    0     0    185      0  0:00:02  0:00:02 --:--:--  4840
    11. 100   427  100   427    0     0    166      0  0:00:02  0:00:02 --:--:--   166
    12. 100   428  100   428    0     0    151      0  0:00:02  0:00:02 --:--:--   151
    13. 100   429  100   429    0     0    138      0  0:00:03  0:00:03 --:--:--     0
    14. 100   381  100   381    0     0    112      0  0:00:03  0:00:03 --:--:--   112
    15. [root@master ~]# yum install -y cri-o

    2.本地下载

    链接：百度网盘 请输入提取码 提取码：i7ih

    三.配置文件（所有节点做）

    1.修改crio配置文件

    找到下面两处位置进行修改

    1. [root@master ~]# vim /etc/crio/crio.conf
    2. pause_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9"
    3. insecure_registries = ["docker.mirrors.ustc.edu.cn","dockerhub.azk8s.cn","hub-mirror.c.163.com"]

     f20ccbe48af54293b91535f78870701a.png

    2.修改kubelet参数（所有节点做）

    将原有的KUBELET_EXTRA_ARGS修改

    1. [root@master ~]# vim /etc/sysconfig/kubelet
    2. KUBELET_EXTRA_ARGS="--cgroup-driver=systemd --container-runtime-endpoint='unix:///var/run/crio/crio.sock' --runtime-request-timeout=5m"

     0d31222ffa8341cf97008f29ab873237.png

    3.重启crio和kubelet

    1. [root@master ~]# systemctl start crio
    2. [root@master ~]# systemctl enable crio
    3. [root@master ~]# systemctl restart crio kubelet

    四.master节点初始化（master节点做）

    1. #若初始化失败，使用这个kubeadm reset命令进行重置
    2. [root@master ~]# kubeadm init --kubernetes-version=v1.28.2 --pod-network-cidr=10.224.0.0/16 \
    3. > --apiserver-advertise-address=192.168.2.190 \       #修改为自己master节点的IP
    4. > --service-cidr=10.96.0.0/12 \
    5. > --cri-socket unix:///var/run/crio/crio.sock \
    6. > --image-repository registry.aliyuncs.com/google_containers
    7. ......
    8. Your Kubernetes control-plane has initialized successfully!
    9. ​
    10. To start using your cluster, you need to run the following as a regular user:
    11. ​
    12.   mkdir -p $HOME/.kube         #master上是普通用户操作
    13.   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    14.   sudo chown $(id -u):$(id -g) $HOME/.kube/config
    15. ​
    16. Alternatively, if you are the root user, you can run:
    17. ​
    18.   export KUBECONFIG=/etc/kubernetes/admin.conf     #master上是普root用户操作
    19. ​
    20. You should now deploy a pod network to the cluster.
    21. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
    22.   https://kubernetes.io/docs/concepts/cluster-administration/addons/
    23. ​
    24. Then you can join any number of worker nodes by running the following on each as root:
    25. ​
    26. kubeadm join 192.168.2.190:6443 --token jxl4ka.4umrziv1a7xf1dpn \
    27.     --discovery-token-ca-cert-hash sha256:6903b8d024842b24fd9c25848df5980d93c42d0fcd7838fdc24ef123c5eba794 
    28.     #node节点使用这行命令加入master
    29.  
    30. [root@master ~]# export KUBECONFIG=/etc/kubernetes/admin.conf

    五.node节点加入（node节点做）

    1. #若加入失败，使用这个kubeadm reset命令进行重置
    2. [root@node2-191 ~]# kubeadm join 192.168.2.190:6443 --token jxl4ka.4umrziv1a7xf1dpn \
    3. > --discovery-token-ca-cert-hash sha256:6903b8d024842b24fd9c25848df5980d93c42d0fcd7838fdc24ef123c5eba794
    4. [preflight] Running pre-flight checks
    5. [preflight] Reading configuration from the cluster...
    6. [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
    7. [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
    8. [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
    9. [kubelet-start] Starting the kubelet
    10. [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
    11. ​
    12. This node has joined the cluster:
    13. * Certificate signing request was sent to apiserver and a response was received.
    14. * The Kubelet was informed of the new secure connection details.
    15. ​
    16. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
    17. ​
    18. [root@master ~]# kubectl get nodes
    19. NAME            STATUS   ROLES           AGE   VERSION
    20. master          Ready    control-plane   36m   v1.28.2
    21. node2-191.com   Ready    <none>          70s   v1.28.2
    22. node4-193.com   Ready    <none>          16s   v1.28.2

    六.添加网络插件（master节点做）

    1.应用calico.yaml

    链接：百度网盘 请输入提取码 提取码：tswi

    [root@master ~]# wget --no-check-certificate https://projectcalico.docs.tigera.io/archive/v3.25/manifests/calico.yaml

    （1）找到这行增加以下内容

    1. [root@master ~]# vim calico.yaml
    2. - name: IP_AUTODETECTION_METHOD
    3.               value: "interface=ens33"      #ens33处为你的网卡名称
    4. [root@master ~]# kubectl apply -f calico.yaml

    129bc4eba64b4cf38aec47151ed1e977.png

    （2） 取消注释这部分并修改地址

    1. - name: CALICO_IPV4POOL_CIDR
    2.               value: 10.244.0.0/16"

    51654d7073f44eed9b46a3f239b18f9c.png

    2.查看验证

    1. #等待初始完成即可
    2. [root@master ~]# kubectl get pod -A
    3. NAMESPACE     NAME                                       READY   STATUS     RESTARTS   AGE
    4. kube-system   calico-kube-controllers-658d97c59c-8ww2m   1/1     Running    0          3m43s
    5. kube-system   calico-node-86787                          1/1     Running    0          3m43s
    6. kube-system   calico-node-jgz7g                          1/1     Running    0          3m43s
    7. kube-system   calico-node-lwxg5                          1/1     Running    0          3m43s
    8. kube-system   coredns-66f779496c-9tvjh                   1/1     Running    0          89m
    9. kube-system   coredns-66f779496c-qzqv9                   1/1     Running    0          89m
    10. kube-system   etcd-master                                1/1     Running    0          89m
    11. kube-system   kube-apiserver-master                      1/1     Running    0          89m
    12. kube-system   kube-controller-manager-master             1/1     Running    0          89m
    13. kube-system   kube-proxy-8nf9q                           1/1     Running    0          54m
    14. kube-system   kube-proxy-sczts                           1/1     Running    0          89m
    15. kube-system   kube-proxy-vrms2                           1/1     Running    0          53m
    16. kube-system   kube-scheduler-master                      1/1     Running    0          89m

    七.配置kubectl命令补全

    1. [root@master ~]#  yum install -y bash-completion
    2. [root@master ~]# source /usr/share/bash-completion/bash_completion
    3. [root@master ~]# source <(kubectl completion bash)
    4. [root@master ~]# echo "source <(kubectl completion bash)" >> ~/.bashrc

     

     

     

  • 相关阅读:
    fork, branch的异同及其cherry-pick和pull request操作
    Intellij各个功能小件的样子
    数据可视化训练第四天（模拟投掷筛子并且统计频次）
    Linux下实现程序开机自启（rc.local 和 systemctl）
    汽车屏类产品（二）：360全景环视(SVC)、多分割显示、行车记录
    SAP 通过 SAT 查找增强 （实例 ：AS01/AS02/AS03屏幕增强新增页签或字段）＜转载＞
    详解闲鱼推荐系统（长文收藏）
    你准备好了吗，9月19日Java21要来了
    单调队列 → 常用于动态规划问题的优化
    揭秘梦幻般的Glam风格是什么？
  • 原文地址：https://blog.csdn.net/weixin_64334766/article/details/133687425