• 在virtualbox上搭建kubenetes集群入门

    一、启动虚拟机，准备环境

    使用Dockerfile文件自动创建virtualbox

    1. Vagrant.configure("2") do |config|
    2.     (1..3).each do |i|
    3.         config.vm.define "k8s-node#{i}" do |node|
    4.             #设置虚拟机的box
    5.             node.vm.box="centos/7"
    6.             
    7.             #设置虚拟机的主机名
    8.             node.vm.hostname="k8s-node#{i}"
    9.             
    10.             #设置虚拟机的ip
    11.             node.vm.network "private_network",ip:"192.168.56.#{99+i}",netmask:"255.255.255.0"
    12.             
    13.             #VirtualBox相关配置
    14.             node.vm.provider "virtualbox" do |v|
    15.                 #设置虚拟机的名称
    16.                 v.name="k8s-node#{i}"
    17.                 #设置虚拟机的内存大小
    18.                 v.memory=4096
    19.                 #设置虚拟机的cpu个数
    20.                 v.cpus=4
    21.             end
    22.         end
    23.     end
    24. end

    进入此window 文件目录

    鼠标放入上方文件目录处 输入 cmd

    如图：第一步

     第二步

     第三步，然后回车，
     在window命令窗口中输入 vagrant up，


     virtualbox就会自动创建三台虚拟机
     
     然后打开virtualbox的 管理--> 全局设定 --> 网络 -->创建NatNetwork


     
     管理 --> 主机网络管理器，保留一个 host-only 启用状态  192.168.56.1  255.255.255.0
     

    （以下操作，三台机器都要操作）

    默认用户名：root

    默认密码是：vagrant

    使用virtualbox的正常启动

    在命令终端输入

    vi  etc/ssh/sshd_config

     找到 passwordAuthentication no 将no修改为yes

    在使用xshell登录就是使用密码，而不是public key了


     二、服务器环境初始化
     
     

    关闭防火墙（清空防火墙规则）
       

    1. systemctl stop firewalld
    2. systemctl disable firewalld


    关闭selinux（关闭安全机制）
       

    1. setenforce 0                   #临时关闭
    2. sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config  #永久


    关闭swap（必须关闭）
     

    1.   swapoff -a                  #临时关闭
    2.   sed -ri 's/.*swap.*/#&/' /etc/fstab            #永久


    使用 hostname 查看每个机器的机器名
       

    1.  192.168.56.100  k8s-node1
    2.  192.168.56.101  k8s-node2
    3.  192.168.56.102  k8s-node3


    添加host：（在host文件末尾添加）
        使用 vi /etc/hosts 打开文件，加入如下内容:

    1.  192.168.56.100  k8s-node1
    2.  192.168.56.101  k8s-node2
    3.  192.168.56.102  k8s-node3


        
    将桥接的ipv4流量传递到iptables的链：

    1. cat > /etc/sysctl.d/k8s.conf << EOF
    2. net.bridge.bridge-nf-call-ip6tables = 1
    3. net.bridge.bridge-nf-call-iptables = 1
    4. EOF
    sysctl --system   #生效


        
       （以上操作，三台机器都要操作）

     


    三、安装docker

    删除之前安装的docker

    1. sudo yum remove docker \
    2. docker-client \
    3. docker-client-latest \
    4. docker-common \
    5. docker-latest \
    6. docker-latest-logrotate \
    7. docker-logrotate \
    8. docker-engine

    安装必须的环境

    1. yum install -y yum-utils \
    2. device-mapper-persistent-data \
    3. lvm2

    设置docker repo的yum位置

    1. sudo yum-config-manager \
    2. --add-repo \
    3. https://download.docker.com/linux/centos/docker-ce.repo


    安装docker 以及docker-cli

    sudo yum install -y docker-ce docker-ce-di containerd.io

    配置docker加速

    1. sudo mkdir -p /etc/docker
    2.  
    3.  
    4. sudo tee /etc/docker/daemon.json <<-'EOF'
    5. {
    6.     "registry-mirrors":["https://82m9ar63.mirror.aliyuns.com"]
    7. }
    8. EOF
    9.  
    10.  
    11. sudo systemctl daemon-reload
    12. sudo systemctl restart docker

    启动docker 和设置docker开机自启动

    systemctl enable docker


    加速阿里云yum源

    1. cat > /etc/yum.repos.d/kubernetes.repo << EOF
    2. [kubernetes]
    3. name=Kubernetes
    4. baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
    5. enabled=1
    6. gpgcheck=0
    7. repo_gpgcheck=0
    8. gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg 
    9. https://mirrors.aliyun.com/kubemetes/yum/doc/rpm-package-key.gpg
    10. EOF


    四、安装kuberadm ,kubelet 和kubectl

    1. yum list|grep kube
    2. yum install -y kubelet-1.17.3 kubeadm-1.17.3 kubectl-1.17.3


    执行上面两个yum命令都 报错如下：

    https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno 14] curl#6 - "Could not resolve host: mirrors.aliyun.com; Unknown error"
    Trying other mirror.


     One of the configured repositories failed (Kubernetes Repo),
     and yum doesn't have enough cached data to continue. At this point the only
     safe thing yum can do is fail. There are a few ways to work "fix" this:

         1. Contact the upstream for the repository and get them to fix the problem.

         2. Reconfigure the baseurl/etc. for the repository, to point to a working
            upstream. This is most often useful if you are using a newer
            distribution release than is supported by the repository (and the
            packages for the previous distribution release still work).

         3. Run the command with the repository temporarily disabled
                yum --disablerepo=kubernetes ...

         4. Disable the repository permanently, so yum won't use it by default. Yum
            will then just ignore the repository until you permanently enable it
            again or use --enablerepo for temporary usage:

                yum-config-manager --disable kubernetes
            or
                subscription-manager repos --disable=kubernetes

         5. Configure the failing repository to be skipped, if it is unavailable.
            Note that yum will try to contact the repo. when it runs most commands,
            so will have to try and fail each time (and thus. yum will be be much
            slower). If it is a very temporary problem though, this is often a nice
            compromise:

                yum-config-manager --save --setopt=kubernetes.skip_if_unavailable=true

    failure: repodata/repomd.xml from kubernetes: [Errno 256] No more mirrors to try.
    https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno 14] curl#6 - "Could not resolve host: mirrors.aliyun.com; Unknown error"

    这里是因为上面配置的docker加速中registry-mirrors对应的地址错了
    正确的：https://82m9ar63.mirror.aliyuns.com
    错误的：https://82m9ar63.mirror.aliyuncs.com

    1. sudo tee /etc/docker/daemon.json <<-'EOF'
    2. {
    3.     "registry-mirrors":["https://82m9ar63.mirror.aliyuns.com"]
    4. }
    5. EOF
    6.  
    7. systemctl enable kubelet
    8.  
    9. systemctl start kubelet
    10.  
    11. systemctl status kubelet


    五、部署k8s-master

    1、master节点初始化

    主节点镜像脚本
    master_images.sh 

    1. #!/bin/bash
    2.  
    3. images=(
    4.     kube-apiserver:v1.17.3
    5.     kube-proxy:v1.17.3
    6.     kube-controller-manager:v1.17.3
    7.     kube-scheduler:v1.17.3
    8.     coredns:1.6.5
    9.     etcd:3.4.3-0
    10.     pause:3.1
    11. )
    12.  
    13. for imageName in ${images[@]}; do
    14.     docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
    15. #   docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
    16. done


    执行 sh master_images.sh


    docker images 查看


    使用 ip addr 
    查看eth0:的ip是： 10.0.2.15

    kubeadm 初始化

    1.  kubeadm init \
    2. --apiserver-advertise-address=10.0.2.15 \
    3. --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers \
    4. --kubernetes-version v1.17.3 \
    5. --service-cidr=10.96.0.0/16 \
    6. --pod-network-cidr=10.244.0.0/16


    报错 ：
    W0819 06:30:25.408387    7990 validation.go:28] Cannot validate kubelet config - no validator is available
    W0819 06:30:25.408444    7990 validation.go:28] Cannot validate kube-proxy config - no validator is available
    [init] Using Kubernetes version: v1.17.3
    [preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 19.03
    error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
    [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
    To see the stack trace of this error execute with --v=5 or higher


    以上错误的原因重点看这条：
    [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1

    使用下面的命令将ip_forward设置为1即可

    sysctl -w net.ipv4.ip_forward=1

    然后再次执行

    1. kubeadm init \
    2. --apiserver-advertise-address=10.0.2.15 \
    3. --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers \
    4. --kubernetes-version v1.17.3 \
    5. --service-cidr=10.96.0.0/16 \
    6. --pod-network-cidr=10.244.0.0/16

    成功，下面这个日志信息要先保存下来，后面使用

    1. Your Kubernetes control-plane has initialized successfully!
    2.  
    3. To start using your cluster, you need to run the following as a regular user:
    4.  
    5.   mkdir -p $HOME/.kube
    6.   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    7.   sudo chown $(id -u):$(id -g) $HOME/.kube/config
    8.  
    9. You should now deploy a pod network to the cluster.
    10. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
    11.   https://kubernetes.io/docs/concepts/cluster-administration/addons/
    12.  
    13. Then you can join any number of worker nodes by running the following on each as root:
    14.  
    15. kubeadm join 10.0.2.15:6443 --token vt4rgu.57cgqbkj1hm7l79b \
    16.     --discovery-token-ca-cert-hash sha256:1668b87c5aaa89bf1c60e4ff3cf59ed10ad54633e3c1df686d13fc8947cda96e


    执行 

    1. mkdir -p $HOME/.kube
    2.   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    3.   sudo chown $(id -u):$(id -g) $HOME/.kube/config


    安装pod网络插件（cni）

    $ kubectl apply -f \
    https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

    1. kubectl apply -f \
    2. https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

    这里我们在上面这个kube-flannel.yum文件中增加了一些信息，详见：

    https://download.csdn.net/download/u011159417/86514665

    执行

    kubectl apply -f kube-flannel.yml


    执行  kubectl get pods
    No resources found in default namespace.

    [root@k8s-node1 k8s]# kubectl get ns
    NAME              STATUS   AGE
    default           Active   154m
    kube-node-lease   Active   154m
    kube-public       Active   154m
    kube-system       Active   154m

    查看pod是否是running状态
    [root@k8s-node1 k8s]# kubectl get pods --all-namespaces
    NAMESPACE     NAME                                READY   STATUS    RESTARTS   AGE
    kube-system   coredns-7f9c544f75-pxcdk            1/1     Running   0          155m
    kube-system   coredns-7f9c544f75-qlpc5            1/1     Running   0          155m
    kube-system   etcd-k8s-node1                      1/1     Running   0          155m
    kube-system   kube-apiserver-k8s-node1            1/1     Running   0          155m
    kube-system   kube-controller-manager-k8s-node1   1/1     Running   0          155m
    kube-system   kube-flannel-ds-amd64-46kpj         1/1     Running   0          4m13s
    kube-system   kube-proxy-jg9pf                    1/1     Running   0          155m
    kube-system   kube-scheduler-k8s-node1            1/1     Running   0          155m


    查看节点信息
    [root@k8s-node1 k8s]# kubectl  get nodes
    NAME        STATUS   ROLES    AGE    VERSION
    k8s-node1   Ready    master   157m   v1.17.3


    2、从节点加入主节点


    在k8s-node2 和 k8s-node3上执行
    上面执行

    1. kubeadm join 10.0.2.15:6443 --token vt4rgu.57cgqbkj1hm7l79b \
    2.     --discovery-token-ca-cert-hash sha256:1668b87c5aaa89bf1c60e4ff3cf59ed10ad54633e3c1df686d13fc8947cda96e

    [root@k8s-node2 k8s]# kubeadm join 10.0.2.15:6443 --token 8mgmlh.cgtgsp3samkvpksn \
    > --discovery-token-ca-cert-hash sha256.3cf99aa2e6bfc114c5490a7c6dffcf200b670af21c5a662c299b6de606023f85
    W0819 09:25:22.037037   15679 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
    [preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 19.03
    error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
    [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
    To see the stack trace of this error execute with --v=5 or higher

    上面的报错
    使用下面的命令将ip_forward设置为1即可

    sysctl -w net.ipv4.ip_forward=1

    [root@k8s-node2 k8s]# kubeadm join 10.0.2.15:6443 --token 8mgmlh.cgtgsp3samkvpksn --discovery-token-ca-cert-hash sha256.3cf99aa2e6bfc114c5490a7c6dffcf200b670af21c5a662c299b6de606023f85
    W0819 09:26:20.274076   15869 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
    [preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 19.03
    error execution phase preflight: couldn't validate the identity of the API Server: invalid public key hash, expected "format:value"
    To see the stack trace of this error execute with --v=5 or higher

    报这个错误说明之前执行
    kubeadm init
    命令后面的日志你没有保存下来，或者复制错误了，导致的

    如果发生了这种情况，重新从第五步执行一下


    查看kube-system命名空间中pod状态

    kubectl get pod -n kube-system -o wide

    这一张的内容有点多，下一章记录：入门操作kubernetes集群


     

  • 相关阅读:
    LeetCode 1624. 两个相同字符之间的最长子字符串
    Redis通过防火墙配置开放局域网连接，禁用外网连接
    记录 | docker权限原因导致service ssh start失败
    OpenGL 波浪特效
    62.WEB渗透测试-信息收集- WAF、框架组件识别（2）
    Logback 日志格式参数说明
    clickhouse运维常用语句
    C#，图论与图算法，图着色问题（Graph Coloring）的威尔士-鲍威尔（Welch Powell Algorithm）算法与源代码
    【毕业设计】深度学习+python+opencv实现动物识别 - 图像识别
    常见聚类算法及使用--层次聚类（Agglomerative clustering）
  • 原文地址：https://blog.csdn.net/u011159417/article/details/126485288