• k8s资源对象(一)

    　　什么是资源对象？

    　　所谓资源对象是指在k8s上创建的资源实例；即通过apiserver提供的各资源api接口（可以理解为各种资源模板），使用yaml文件或者命令行的方式向对应资源api接口传递参数赋值实例化的结果；比如我们在k8s上创建一个pod，那么我们就需要通过给apiserver交互，传递创建pod的相关参数，让apiserver拿着这些参数去实例化一个pod的相关信息存放在etcd中，然后再由调度器进行调度，由node节点的kubelet执行创建pod；简单讲资源对象就是把k8s之上的api接口进行实例化的结果；

    　　k8s逻辑运行环境

    　　提示：k8s运行环境如上，k8s能够将多个node节点的底层提供的资源（如内存，cpu，存储，网络等）逻辑的整合成一个大的资源池，统一由k8s进行调度编排；用户只管在k8s上创建各种资源即可，创建完成的资源是通过k8s统一编排调度，用户无需关注具体资源在那个node上运行，也无需关注node节点资源情况；

    　　k8s的设计理念——分层架构

    　　k8s的设计理念——API设计原则

    　　1、所有API应该是声明式的；

    　　2、API对象是彼此互补而且可组合的，即“高内聚，松耦合”；

    　　3、高层API以操作意图为基础设计；

    　　4、低层API根据高层API的控制需要设计；

    　　5、尽量避免简单封装，不要有在外部API无法显式知道的内部隐藏的机制；

    　　6、API操作复杂度与对象数量成正比；

    　　7、API对象状态不能依赖于网络连接状态；

    　　8、尽量避免让操作机制依赖于全局状态，因为在分布式系统中要保证全局状态的同步是非常困难的；

    　　kubernetes API简介

    　　提示：在k8s上api分内置api和自定义api；所谓内置api是指部署好k8s集群后自带的api接口；自定义api也称自定义资源（CRD，Custom Resource Definition），部署好k8s之后，通过安装其他组件等方式扩展出来的api；

    　　apiserver资源组织逻辑

    　　提示：apiserver对于不同资源是通过分类，分组，分版本的方式逻辑组织的，如上图所示；

    　　k8s内置资源对象简介

    　　k8s资源对象操作命令

    　　资源配置清单必需字段

    　　1、apiVersion - 创建该对象所使用的Kubernetes API的版本；

    　　2、kind - 想要创建的对象的类型；

    　　3、metadata - 定义识别对象唯一性的数据，包括一个name名称 、可选的namespace，默认不写就是default名称空间；

    　　4、spec：定义资源对象的详细规范信息(统一的label标签、容器名称、镜像、端口映射等)，即用户期望对应资源处于什么状态；

    　　5、status（Pod创建完成后k8s自动生成status状态），该字段信息由k8s自动维护，用户无需定义，即对应资源的实际状态；

    　　Pod资源对象

    　　提示：pod是k8s中最小控制单元，一个pod中可以运行一个或多个容器；一个pod的中的容器是一起调度，即调度的最小单位是pod；pod的生命周期是短暂的，不会自愈，是用完就销毁的实体；一般我们通过Controller来创建和管理pod；使用控制器创建的pod具有自动恢复功能，即pod状态不满足用户期望状态，对应控制器会通过重启或重建的方式，让对应pod状态和数量始终和用户定义的期望状态一致；

    　　示例：自主式pod配置清单

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    apiVersion: v1
    kind: Pod
    metadata:
      name: "pod-demo"
      namespace: default
      labels:
        app: "pod-demo"
    spec:
      containers:
      - name: pod-demo
        image: "harbor.ik8s.cc/baseimages/nginx:v1"
        ports:
        - containerPort:  80
          name:  http
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
      volumes:
        - name: localtime
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai

    　　应用配置清单

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    root@k8s-deploy:/yaml# kubectl get pods
    NAME        READY   STATUS    RESTARTS        AGE
    net-test1   1/1     Running   2 (4m35s ago)   7d7h
    test        1/1     Running   4 (4m34s ago)   13d
    test1       1/1     Running   4 (4m35s ago)   13d
    test2       1/1     Running   4 (4m35s ago)   13d
    root@k8s-deploy:/yaml# kubectl apply -f pod-demo.yaml
    pod/pod-demo created
    root@k8s-deploy:/yaml# kubectl get pods
    NAME        READY   STATUS              RESTARTS        AGE
    net-test1   1/1     Running             2 (4m47s ago)   7d7h
    pod-demo    0/1     ContainerCreating   0               4s
    test        1/1     Running             4 (4m46s ago)   13d
    test1       1/1     Running             4 (4m47s ago)   13d
    test2       1/1     Running             4 (4m47s ago)   13d
    root@k8s-deploy:/yaml# kubectl get pods
    NAME        READY   STATUS    RESTARTS        AGE
    net-test1   1/1     Running   2 (4m57s ago)   7d7h
    pod-demo    1/1     Running   0               14s
    test        1/1     Running   4 (4m56s ago)   13d
    test1       1/1     Running   4 (4m57s ago)   13d
    test2       1/1     Running   4 (4m57s ago)   13d
    root@k8s-deploy:/yaml#

    　　提示：此pod只是在k8s上运行起来，它没有控制器的监视，对应pod删除，故障都不会自动恢复；

    　　Job控制器，详细说明请参考https://www.cnblogs.com/qiuhom-1874/p/14157306.html；

    　　job控制器配置清单示例

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: job-demo
      namespace: default
      labels:
        app: job-demo
    spec:
      template:
        metadata:
          name: job-demo
          labels:
            app: job-demo
        spec:   
          containers:
          - name: job-demo-container
            image: harbor.ik8s.cc/baseimages/centos7:2023
            command: ["/bin/sh"]
            args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
            volumeMounts:
            - mountPath: /cache
              name: cache-volume
            - name: localtime
              mountPath: /etc/localtime
          volumes:
          - name: cache-volume
            hostPath:
              path: /tmp/jobdata
          - name: localtime
            hostPath:
              path: /usr/share/zoneinfo/Asia/Shanghai
          restartPolicy: Never

    　　提示：定义job资源必须定义restartPolicy；

    　　应用清单

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    root@k8s-deploy:/yaml# kubectl get pods
    NAME        READY   STATUS    RESTARTS      AGE
    net-test1   1/1     Running   3 (48m ago)   7d10h
    pod-demo    1/1     Running   1 (48m ago)   3h32m
    test        1/1     Running   5 (48m ago)   14d
    test1       1/1     Running   5 (48m ago)   14d
    test2       1/1     Running   5 (48m ago)   14d
    root@k8s-deploy:/yaml# kubectl apply -f job-demo.yaml
    job.batch/job-demo created
    root@k8s-deploy:/yaml# kubectl get pods -o wide
    NAME             READY   STATUS      RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
    job-demo-z8gmb   0/1     Completed   0             26s     10.200.211.130   192.168.0.34             
    net-test1        1/1     Running     3 (49m ago)   7d10h   10.200.211.191   192.168.0.34             
    pod-demo         1/1     Running     1 (49m ago)   3h32m   10.200.155.138   192.168.0.36             
    test             1/1     Running     5 (49m ago)   14d     10.200.209.6     192.168.0.35             
    test1            1/1     Running     5 (49m ago)   14d     10.200.209.8     192.168.0.35             
    test2            1/1     Running     5 (49m ago)   14d     10.200.211.177   192.168.0.34             
    root@k8s-deploy:/yaml#

    　　验证：查看192.168.0.34的/tmp/jobdata目录下是否有job执行的任务数据？

    ?
    1
    2
    3
    4
    5
    root@k8s-deploy:/yaml# ssh 192.168.0.34 "ls /tmp/jobdata"
    data.log
    root@k8s-deploy:/yaml# ssh 192.168.0.34 "cat /tmp/jobdata/data.log"
    data init job at 2023-05-06_23-31-32
    root@k8s-deploy:/yaml#

    　　提示：可以看到对应job所在宿主机的/tmp/jobdata/目录下有job执行过后的数据，这说明我们定义的job任务顺利完成；

    　　定义并行job

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: job-multi-demo
      namespace: default
      labels:
        app: job-multi-demo
    spec:
      completions: 5  
      template:
        metadata:
          name: job-multi-demo
          labels:
            app: job-multi-demo
        spec:
          containers:
          - name: job-multi-demo-container
            image: harbor.ik8s.cc/baseimages/centos7:2023
            command: ["/bin/sh"]
            args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
            volumeMounts:
            - mountPath: /cache
              name: cache-volume
            - name: localtime
              mountPath: /etc/localtime
          volumes:
          - name: cache-volume
            hostPath:
              path: /tmp/jobdata
          - name: localtime
            hostPath:
              path: /usr/share/zoneinfo/Asia/Shanghai
          restartPolicy: Never

    　　提示：spec字段下使用completions来指定执行任务需要的对应pod的数量；

    　　应用清单

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    root@k8s-deploy:/yaml# kubectl get pods
    NAME             READY   STATUS      RESTARTS      AGE
    job-demo-z8gmb   0/1     Completed   0             24m
    net-test1        1/1     Running     3 (73m ago)   7d11h
    pod-demo         1/1     Running     1 (73m ago)   3h56m
    test             1/1     Running     5 (73m ago)   14d
    test1            1/1     Running     5 (73m ago)   14d
    test2            1/1     Running     5 (73m ago)   14d
    root@k8s-deploy:/yaml# kubectl apply -f job-multi-demo.yaml
    job.batch/job-multi-demo created
    root@k8s-deploy:/yaml# kubectl get job
    NAME             COMPLETIONS   DURATION   AGE
    job-demo         1/1           5s         24m
    job-multi-demo   1/5           10s        10s
    root@k8s-deploy:/yaml# kubectl get pods -o wide
    NAME                   READY   STATUS              RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
    job-demo-z8gmb         0/1     Completed           0             24m     10.200.211.130   192.168.0.34             
    job-multi-demo-5vp9w   0/1     Completed           0             12s     10.200.211.144   192.168.0.34             
    job-multi-demo-frstg   0/1     Completed           0             22s     10.200.211.186   192.168.0.34             
    job-multi-demo-gd44s   0/1     Completed           0             17s     10.200.211.184   192.168.0.34             
    job-multi-demo-kfm79   0/1     ContainerCreating   0             2s                 192.168.0.34             
    job-multi-demo-nsmpg   0/1     Completed           0             7s      10.200.211.135   192.168.0.34             
    net-test1              1/1     Running             3 (73m ago)   7d11h   10.200.211.191   192.168.0.34             
    pod-demo               1/1     Running             1 (73m ago)   3h56m   10.200.155.138   192.168.0.36             
    test                   1/1     Running             5 (73m ago)   14d     10.200.209.6     192.168.0.35             
    test1                  1/1     Running             5 (73m ago)   14d     10.200.209.8     192.168.0.35             
    test2                  1/1     Running             5 (73m ago)   14d     10.200.211.177   192.168.0.34             
    root@k8s-deploy:/yaml# kubectl get pods -o wide
    NAME                   READY   STATUS      RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
    job-demo-z8gmb         0/1     Completed   0             24m     10.200.211.130   192.168.0.34             
    job-multi-demo-5vp9w   0/1     Completed   0             33s     10.200.211.144   192.168.0.34             
    job-multi-demo-frstg   0/1     Completed   0             43s     10.200.211.186   192.168.0.34             
    job-multi-demo-gd44s   0/1     Completed   0             38s     10.200.211.184   192.168.0.34             
    job-multi-demo-kfm79   0/1     Completed   0             23s     10.200.211.140   192.168.0.34             
    job-multi-demo-nsmpg   0/1     Completed   0             28s     10.200.211.135   192.168.0.34             
    net-test1              1/1     Running     3 (73m ago)   7d11h   10.200.211.191   192.168.0.34             
    pod-demo               1/1     Running     1 (73m ago)   3h57m   10.200.155.138   192.168.0.36             
    test                   1/1     Running     5 (73m ago)   14d     10.200.209.6     192.168.0.35             
    test1                  1/1     Running     5 (73m ago)   14d     10.200.209.8     192.168.0.35             
    test2                  1/1     Running     5 (73m ago)   14d     10.200.211.177   192.168.0.34             
    root@k8s-deploy:/yaml#

    　　验证：查看192.168.0.34的/tmp/jobdata/目录下是否有job数据产生？

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    root@k8s-deploy:/yaml# ssh 192.168.0.34 "ls /tmp/jobdata"
    data.log
    root@k8s-deploy:/yaml# ssh 192.168.0.34 "cat /tmp/jobdata/data.log"
    data init job at 2023-05-06_23-31-32
    data init job at 2023-05-06_23-55-44
    data init job at 2023-05-06_23-55-49
    data init job at 2023-05-06_23-55-54
    data init job at 2023-05-06_23-55-59
    data init job at 2023-05-06_23-56-04
    root@k8s-deploy:/yaml#

    　　定义并行度

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: job-multi-demo2
      namespace: default
      labels:
        app: job-multi-demo2
    spec:
      completions: 6
      parallelism: 2  
      template:
        metadata:
          name: job-multi-demo2
          labels:
            app: job-multi-demo2
        spec:
          containers:
          - name: job-multi-demo2-container
            image: harbor.ik8s.cc/baseimages/centos7:2023
            command: ["/bin/sh"]
            args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
            volumeMounts:
            - mountPath: /cache
              name: cache-volume
            - name: localtime
              mountPath: /etc/localtime
          volumes:
          - name: cache-volume
            hostPath:
              path: /tmp/jobdata
          - name: localtime
            hostPath:
              path: /usr/share/zoneinfo/Asia/Shanghai
          restartPolicy: Never

    　　提示：在spec字段下使用parallelism字段来指定并行度，即一次几个pod同时运行；上述清单表示，一次2个pod同时运行，总共需要6个pod；

    　　应用清单

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    root@k8s-deploy:/yaml# kubectl get jobs
    NAME             COMPLETIONS   DURATION   AGE
    job-demo         1/1           5s         34m
    job-multi-demo   5/5           25s        9m56s
    root@k8s-deploy:/yaml# kubectl apply -f job-multi-demo2.yaml
    job.batch/job-multi-demo2 created
    root@k8s-deploy:/yaml# kubectl get jobs
    NAME              COMPLETIONS   DURATION   AGE
    job-demo          1/1           5s         34m
    job-multi-demo    5/5           25s        10m
    job-multi-demo2   0/6           2s         3s
    root@k8s-deploy:/yaml# kubectl get pods
    NAME                    READY   STATUS      RESTARTS      AGE
    job-demo-z8gmb          0/1     Completed   0             34m
    job-multi-demo-5vp9w    0/1     Completed   0             10m
    job-multi-demo-frstg    0/1     Completed   0             10m
    job-multi-demo-gd44s    0/1     Completed   0             10m
    job-multi-demo-kfm79    0/1     Completed   0             9m59s
    job-multi-demo-nsmpg    0/1     Completed   0             10m
    job-multi-demo2-7ppxc   0/1     Completed   0             10s
    job-multi-demo2-mxbtq   0/1     Completed   0             5s
    job-multi-demo2-rhgh7   0/1     Completed   0             4s
    job-multi-demo2-th6ff   0/1     Completed   0             11s
    net-test1               1/1     Running     3 (83m ago)   7d11h
    pod-demo                1/1     Running     1 (83m ago)   4h6m
    test                    1/1     Running     5 (83m ago)   14d
    test1                   1/1     Running     5 (83m ago)   14d
    test2                   1/1     Running     5 (83m ago)   14d
    root@k8s-deploy:/yaml# kubectl get pods
    NAME                    READY   STATUS      RESTARTS      AGE
    job-demo-z8gmb          0/1     Completed   0             34m
    job-multi-demo-5vp9w    0/1     Completed   0             10m
    job-multi-demo-frstg    0/1     Completed   0             10m
    job-multi-demo-gd44s    0/1     Completed   0             10m
    job-multi-demo-kfm79    0/1     Completed   0             10m
    job-multi-demo-nsmpg    0/1     Completed   0             10m
    job-multi-demo2-7ppxc   0/1     Completed   0             16s
    job-multi-demo2-8bh22   0/1     Completed   0             6s
    job-multi-demo2-dbjqw   0/1     Completed   0             6s
    job-multi-demo2-mxbtq   0/1     Completed   0             11s
    job-multi-demo2-rhgh7   0/1     Completed   0             10s
    job-multi-demo2-th6ff   0/1     Completed   0             17s
    net-test1               1/1     Running     3 (83m ago)   7d11h
    pod-demo                1/1     Running     1 (83m ago)   4h6m
    test                    1/1     Running     5 (83m ago)   14d
    test1                   1/1     Running     5 (83m ago)   14d
    test2                   1/1     Running     5 (83m ago)   14d
    root@k8s-deploy:/yaml# kubectl get pods -o wide
    NAME                    READY   STATUS      RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
    job-demo-z8gmb          0/1     Completed   0             35m     10.200.211.130   192.168.0.34             
    job-multi-demo-5vp9w    0/1     Completed   0             10m     10.200.211.144   192.168.0.34             
    job-multi-demo-frstg    0/1     Completed   0             11m     10.200.211.186   192.168.0.34             
    job-multi-demo-gd44s    0/1     Completed   0             11m     10.200.211.184   192.168.0.34             
    job-multi-demo-kfm79    0/1     Completed   0             10m     10.200.211.140   192.168.0.34             
    job-multi-demo-nsmpg    0/1     Completed   0             10m     10.200.211.135   192.168.0.34             
    job-multi-demo2-7ppxc   0/1     Completed   0             57s     10.200.211.145   192.168.0.34             
    job-multi-demo2-8bh22   0/1     Completed   0             47s     10.200.211.148   192.168.0.34             
    job-multi-demo2-dbjqw   0/1     Completed   0             47s     10.200.211.141   192.168.0.34             
    job-multi-demo2-mxbtq   0/1     Completed   0             52s     10.200.211.152   192.168.0.34             
    job-multi-demo2-rhgh7   0/1     Completed   0             51s     10.200.211.143   192.168.0.34             
    job-multi-demo2-th6ff   0/1     Completed   0             58s     10.200.211.136   192.168.0.34             
    net-test1               1/1     Running     3 (84m ago)   7d11h   10.200.211.191   192.168.0.34             
    pod-demo                1/1     Running     1 (84m ago)   4h7m    10.200.155.138   192.168.0.36             
    test                    1/1     Running     5 (84m ago)   14d     10.200.209.6     192.168.0.35             
    test1                   1/1     Running     5 (84m ago)   14d     10.200.209.8     192.168.0.35             
    test2                   1/1     Running     5 (84m ago)   14d     10.200.211.177   192.168.0.34             
    root@k8s-deploy:/yaml#

    　　验证job数据

    　　提示：可以看到后面job追加的时间几乎都是两个重复的，这说明两个pod同时执行了job里的任务；

    　　Cronjob控制器，详细说明请参考https://www.cnblogs.com/qiuhom-1874/p/14157306.html；

    　　示例：定义cronjob

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    apiVersion: batch/v1
    kind: CronJob
    metadata:
      name: job-cronjob
      namespace: default
    spec:
      schedule: "*/1 * * * *"
      jobTemplate:
        spec:
          parallelism: 2
          template:
            spec:
              containers:
              - name: job-cronjob-container
                image: harbor.ik8s.cc/baseimages/centos7:2023
                command: ["/bin/sh"]
                args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/cronjob-data.log"]
                volumeMounts:
                - mountPath: /cache
                  name: cache-volume
                - name: localtime
                  mountPath: /etc/localtime
              volumes:
              - name: cache-volume
                hostPath:
                  path: /tmp/jobdata
              - name: localtime
                hostPath:
                  path: /usr/share/zoneinfo/Asia/Shanghai
              restartPolicy: OnFailure

    　　应用清单

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    root@k8s-deploy:/yaml# kubectl apply -f cronjob-demo.yaml
    cronjob.batch/job-cronjob created
    root@k8s-deploy:/yaml# kubectl get cronjob
    NAME          SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
    job-cronjob   */1 * * * *   False     0                  6s
    root@k8s-deploy:/yaml# kubectl get pods
    NAME                         READY   STATUS      RESTARTS       AGE
    job-cronjob-28056516-njddz   0/1     Completed   0              12s
    job-cronjob-28056516-wgbns   0/1     Completed   0              12s
    job-demo-z8gmb               0/1     Completed   0              64m
    job-multi-demo-5vp9w         0/1     Completed   0              40m
    job-multi-demo-frstg         0/1     Completed   0              40m
    job-multi-demo-gd44s         0/1     Completed   0              40m
    job-multi-demo-kfm79         0/1     Completed   0              40m
    job-multi-demo-nsmpg         0/1     Completed   0              40m
    job-multi-demo2-7ppxc        0/1     Completed   0              30m
    job-multi-demo2-8bh22        0/1     Completed   0              30m
    job-multi-demo2-dbjqw        0/1     Completed   0              30m
    job-multi-demo2-mxbtq        0/1     Completed   0              30m
    job-multi-demo2-rhgh7        0/1     Completed   0              30m
    job-multi-demo2-th6ff        0/1     Completed   0              30m
    net-test1                    1/1     Running     3 (113m ago)   7d11h
    pod-demo                     1/1     Running     1 (113m ago)   4h36m
    test                         1/1     Running     5 (113m ago)   14d
    test1                        1/1     Running     5 (113m ago)   14d
    test2                        1/1     Running     5 (113m ago)   14d
    root@k8s-deploy:/yaml# kubectl get cronjob
    NAME          SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
    job-cronjob   */1 * * * *   False     0        12s             108s
    root@k8s-deploy:/yaml# kubectl get pods  
    NAME                         READY   STATUS      RESTARTS       AGE
    job-cronjob-28056516-njddz   0/1     Completed   0              77s
    job-cronjob-28056516-wgbns   0/1     Completed   0              77s
    job-cronjob-28056517-d6n9h   0/1     Completed   0              17s
    job-cronjob-28056517-krsvb   0/1     Completed   0              17s
    job-demo-z8gmb               0/1     Completed   0              65m
    job-multi-demo-5vp9w         0/1     Completed   0              41m
    job-multi-demo-frstg         0/1     Completed   0              41m
    job-multi-demo-gd44s         0/1     Completed   0              41m
    job-multi-demo-kfm79         0/1     Completed   0              41m
    job-multi-demo-nsmpg         0/1     Completed   0              41m
    job-multi-demo2-7ppxc        0/1     Completed   0              31m
    job-multi-demo2-8bh22        0/1     Completed   0              31m
    job-multi-demo2-dbjqw        0/1     Completed   0              31m
    job-multi-demo2-mxbtq        0/1     Completed   0              31m
    job-multi-demo2-rhgh7        0/1     Completed   0              31m
    job-multi-demo2-th6ff        0/1     Completed   0              31m
    net-test1                    1/1     Running     3 (114m ago)   7d11h
    pod-demo                     1/1     Running     1 (114m ago)   4h38m
    test                         1/1     Running     5 (114m ago)   14d
    test1                        1/1     Running     5 (114m ago)   14d
    test2                        1/1     Running     5 (114m ago)   14d
    root@k8s-deploy:/yaml#

    　　提示：cronjob 默认保留最近3个历史记录；

    　　验证：查看周期执行任务的数据

     　　提示：从上面的时间就可以看到每过一分钟就有两个pod执行一次任务；

    　　RC/RS 副本控制器

    　　RC（Replication Controller），副本控制器，该控制器主要负责控制pod副本数量始终满足用户期望的副本数量，该副本控制器是第一代pod副本控制器，仅支持selector = !=；

    　　rc控制器示例

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    apiVersion: v1 
    kind: ReplicationController 
    metadata: 
      name: ng-rc
    spec: 
      replicas: 2
      selector: 
        app: ng-rc-80
      template:  
        metadata: 
          labels: 
            app: ng-rc-80
        spec: 
          containers:
          - name: pod-demo
            image: "harbor.ik8s.cc/baseimages/nginx:v1"
            ports:
            - containerPort:  80
              name:  http
            volumeMounts:
            - name: localtime
              mountPath: /etc/localtime
          volumes:
            - name: localtime
              hostPath:
                path: /usr/share/zoneinfo/Asia/Shanghai

    　　应用配置清单

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    root@k8s-deploy:/yaml# kubectl get pods
    NAME    READY   STATUS    RESTARTS      AGE
    test    1/1     Running   6 (11m ago)   16d
    test1   1/1     Running   6 (11m ago)   16d
    test2   1/1     Running   6 (11m ago)   16d
    root@k8s-deploy:/yaml# kubectl apply -f rc-demo.yaml
    replicationcontroller/ng-rc created
    root@k8s-deploy:/yaml# kubectl get pods -o wide
    NAME          READY   STATUS    RESTARTS      AGE   IP               NODE           NOMINATED NODE   READINESS GATES
    ng-rc-l7xmp   1/1     Running   0             10s   10.200.211.136   192.168.0.34             
    ng-rc-wl5d6   1/1     Running   0             9s    10.200.155.185   192.168.0.36             
    test          1/1     Running   6 (11m ago)   16d   10.200.209.24    192.168.0.35             
    test1         1/1     Running   6 (11m ago)   16d   10.200.209.31    192.168.0.35             
    test2         1/1     Running   6 (11m ago)   16d   10.200.211.186   192.168.0.34             
    root@k8s-deploy:/yaml# kubectl get rc
    NAME    DESIRED   CURRENT   READY   AGE
    ng-rc   2         2         2       25s
    root@k8s-deploy:/yaml#

    　　验证：修改pod标签，看看对应pod是否会重新创建？

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    root@k8s-deploy:/yaml# kubectl get pods --show-labels
    NAME          READY   STATUS    RESTARTS      AGE     LABELS
    ng-rc-l7xmp   1/1     Running   0             2m32s   app=ng-rc-80
    ng-rc-wl5d6   1/1     Running   0             2m31s   app=ng-rc-80
    test          1/1     Running   6 (13m ago)   16d     run=test
    test1         1/1     Running   6 (13m ago)   16d     run=test1
    test2         1/1     Running   6 (13m ago)   16d     run=test2
    root@k8s-deploy:/yaml# kubectl label pod/ng-rc-l7xmp app=nginx-demo --overwrite
    pod/ng-rc-l7xmp labeled
    root@k8s-deploy:/yaml# kubectl get pods --show-labels                         
    NAME          READY   STATUS              RESTARTS      AGE     LABELS
    ng-rc-l7xmp   1/1     Running             0             4m42s   app=nginx-demo
    ng-rc-rxvd4   0/1     ContainerCreating   0             3s      app=ng-rc-80
    ng-rc-wl5d6   1/1     Running             0             4m41s   app=ng-rc-80
    test          1/1     Running             6 (15m ago)   16d     run=test
    test1         1/1     Running             6 (15m ago)   16d     run=test1
    test2         1/1     Running             6 (15m ago)   16d     run=test2
    root@k8s-deploy:/yaml# kubectl get pods --show-labels
    NAME          READY   STATUS    RESTARTS      AGE     LABELS
    ng-rc-l7xmp   1/1     Running   0             4m52s   app=nginx-demo
    ng-rc-rxvd4   1/1     Running   0             13s     app=ng-rc-80
    ng-rc-wl5d6   1/1     Running   0             4m51s   app=ng-rc-80
    test          1/1     Running   6 (16m ago)   16d     run=test
    test1         1/1     Running   6 (16m ago)   16d     run=test1
    test2         1/1     Running   6 (16m ago)   16d     run=test2
    root@k8s-deploy:/yaml# kubectl label pod/ng-rc-l7xmp app=ng-rc-80 --overwrite 
    pod/ng-rc-l7xmp labeled
    root@k8s-deploy:/yaml# kubectl get pods --show-labels                       
    NAME          READY   STATUS    RESTARTS      AGE     LABELS
    ng-rc-l7xmp   1/1     Running   0             5m27s   app=ng-rc-80
    ng-rc-wl5d6   1/1     Running   0             5m26s   app=ng-rc-80
    test          1/1     Running   6 (16m ago)   16d     run=test
    test1         1/1     Running   6 (16m ago)   16d     run=test1
    test2         1/1     Running   6 (16m ago)   16d     run=test2
    root@k8s-deploy:/yaml#

    　　提示：rc控制器是通过标签选择器来识别对应pod是否归属对应rc控制器管控，如果发现对应pod的标签发生改变，那么rc控制器会通过新建或删除的方法将对应pod数量始终和用户定义的数量保持一致；

    　　RS（ReplicaSet），副本控制器，该副本控制器和rc类似，都是通过标签选择器来匹配归属自己管控的pod数量，如果标签或对应pod数量少于或多余用户期望的数量，该控制器会通过新建或删除pod的方式将对应pod数量始终和用户期望的pod数量保持一致；rs控制器和rc控制器唯一区别就是rs控制器支持selector = !=精确匹配外，还支持模糊匹配in notin；是k8s之上的第二代pod副本控制器；

    　　rs控制器示例

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    apiVersion: apps/v1
    kind: ReplicaSet
    metadata:
      name: rs-demo
      labels:
        app: rs-demo
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: rs-demo
      template:
        metadata:
          labels:
            app: rs-demo
        spec:
          containers:
          - name: rs-demo
            image: "harbor.ik8s.cc/baseimages/nginx:v1"
            ports:
            - name: web
              containerPort:  80
              protocol: TCP
            env:
            - name: NGX_VERSION
              value: 1.16.1
            volumeMounts:
            - name: localtime
              mountPath: /etc/localtime
          volumes:
            - name: localtime
              hostPath:
                path: /usr/share/zoneinfo/Asia/Shanghai

    　　应用配置清单

    　　验证：修改pod标签，看看对应pod是否会发生变化？

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    root@k8s-deploy:/yaml# kubectl get pods --show-labels
    NAME            READY   STATUS    RESTARTS      AGE   LABELS
    ng-rc-l7xmp     1/1     Running   0             18m   app=ng-rc-80
    ng-rc-wl5d6     1/1     Running   0             18m   app=ng-rc-80
    rs-demo-nzmqs   1/1     Running   0             71s   app=rs-demo
    rs-demo-v2vb6   1/1     Running   0             71s   app=rs-demo
    rs-demo-x27fv   1/1     Running   0             71s   app=rs-demo
    test            1/1     Running   6 (29m ago)   16d   run=test
    test1           1/1     Running   6 (29m ago)   16d   run=test1
    test2           1/1     Running   6 (29m ago)   16d   run=test2
    root@k8s-deploy:/yaml# kubectl label pod/rs-demo-nzmqs app=nginx --overwrite
    pod/rs-demo-nzmqs labeled
    root@k8s-deploy:/yaml# kubectl get pods --show-labels                      
    NAME            READY   STATUS    RESTARTS      AGE    LABELS
    ng-rc-l7xmp     1/1     Running   0             19m    app=ng-rc-80
    ng-rc-wl5d6     1/1     Running   0             19m    app=ng-rc-80
    rs-demo-bdfdd   1/1     Running   0             4s     app=rs-demo
    rs-demo-nzmqs   1/1     Running   0             103s   app=nginx
    rs-demo-v2vb6   1/1     Running   0             103s   app=rs-demo
    rs-demo-x27fv   1/1     Running   0             103s   app=rs-demo
    test            1/1     Running   6 (30m ago)   16d    run=test
    test1           1/1     Running   6 (30m ago)   16d    run=test1
    test2           1/1     Running   6 (30m ago)   16d    run=test2
    root@k8s-deploy:/yaml# kubectl label pod/rs-demo-nzmqs app=rs-demo --overwrite
    pod/rs-demo-nzmqs labeled
    root@k8s-deploy:/yaml# kubectl get pods --show-labels                        
    NAME            READY   STATUS    RESTARTS      AGE    LABELS
    ng-rc-l7xmp     1/1     Running   0             19m    app=ng-rc-80
    ng-rc-wl5d6     1/1     Running   0             19m    app=ng-rc-80
    rs-demo-nzmqs   1/1     Running   0             119s   app=rs-demo
    rs-demo-v2vb6   1/1     Running   0             119s   app=rs-demo
    rs-demo-x27fv   1/1     Running   0             119s   app=rs-demo
    test            1/1     Running   6 (30m ago)   16d    run=test
    test1           1/1     Running   6 (30m ago)   16d    run=test1
    test2           1/1     Running   6 (30m ago)   16d    run=test2
    root@k8s-deploy:/yaml#

    　　提示：可以看到当我们修改pod标签为其他标签以后，对应rs控制器会新建一个pod，其标签为app=rs-demo，这是因为当我们修改pod标签以后，rs控制器发现标签选择器匹配的pod数量少于用户定义的数量，所以rs控制器会新建一个标签为app=rs-demo的pod；当我们把pod标签修改为rs-demo时，rs控制器发现对应标签选择器匹配pod数量多余用户期望的pod数量，此时rs控制器会通过删除pod方法，让app=rs-demo标签的pod和用户期望的pod数量保持一致；

    　　Deployment 副本控制器，详细说明请参考https://www.cnblogs.com/qiuhom-1874/p/14149042.html；

    　　Deployment副本控制器时k8s第三代pod副本控制器，该控制器比rs控制器更高级，除了有rs的功能之外，还有很多高级功能,，比如说最重要的滚动升级、回滚等；

    　　deploy控制器示例

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name:  deploy-demo
      namespace: default
      labels:
        app:  deploy-demo
    spec:
      selector:
        matchLabels:
          app: deploy-demo
      replicas: 2
      template:
        metadata:
          labels:
            app:  deploy-demo
        spec:
          containers:
          - name:  deploy-demo
            image:  "harbor.ik8s.cc/baseimages/nginx:v1"
            ports:
            - containerPort:  80
              name:  http
            volumeMounts:
            - name: localtime
              mountPath: /etc/localtime
          volumes:
            - name: localtime
              hostPath:
                path: /usr/share/zoneinfo/Asia/Shanghai

    　　应用配置清单

    　　提示：deploy控制器是通过创建rs控制器来实现管控对应pod数量；

     　　通过修改镜像版本来更新pod版本

    　　应用配置清单

    　　使用命令更新pod版本

     　　查看rs更新历史版本

     　　查看更新历史记录

    　　提示：这里历史记录中没有记录版本信息，原因是默认不记录，需要记录历史版本，可以手动使用--record选项来记录版本信息；如下所示

    　　查看某个历史版本的详细信息

    　　提示：查看某个历史版本的详细信息，加上--revision=对应历史版本的编号即可；

    　　回滚到上一个版本

    　　提示：使用kubectl rollout undo 命令可以将对应deploy回滚到上一个版本；

    　　回滚指定编号的历史版本

    　　提示：使用--to-revision选项来指定对应历史版本编号，即可回滚到对应编号的历史版本；

    　　Service资源，详细说明请参考https://www.cnblogs.com/qiuhom-1874/p/14161950.html；

    　　nodeport类型的service访问流程

    　　nodeport类型service主要解决了k8s集群外部客户端访问pod，其流程是外部客户端访问k8s集群任意node节点的对应暴露的端口，被访问的node或通过本机的iptables或ipvs规则将外部客户端流量转发给对应pod之上，从而实现外部客户端访问k8s集群pod的目的；通常使用nodeport类型service为了方便外部客户端访问，都会在集群外部部署一个负载均衡器，即外部客户端访问对应负载均衡器的对应端口，通过负载均衡器将外部客户端流量引入k8s集群，从而完成对pod的访问；

    　　ClusterIP类型svc示例

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    apiVersion: v1
    kind: Service
    metadata:
      name: ngx-svc
      namespace: default
    spec:
      selector:
        app: deploy-demo
      type: ClusterIP
      ports:
      - name: http
        protocol: TCP
        port: 80
        targetPort: 80

    　　应用配置清单

    　　提示：可以看到创建clusterip类型service以后，对应svc会有一个clusterip，后端endpoints会通过标签选择器去关联对应pod，即我们访问对应svc的clusterip，对应流量会被转发至后端endpoint pod之上进行响应；不过这种clusterip类型svc只能在k8s集群内部客户端访问，集群外部客户端是访问不到的，原因是这个clusterip是k8s内部网络IP地址；

    　　验证，访问10.100.100.23的80端口，看看对应后端nginxpod是否可以正常被访问呢？

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    root@k8s-node01:~# curl 10.100.100.23
    <head>
    Welcome to nginx!<</code><code class="bash plain">/title</code><code class="bash plain">></code></div><div class="line number6 index5 alt1"><code class="bash plain"><style></code></div><div class="line number7 index6 alt2"><code class="bash plain">html { color-scheme: light dark; }</code></div><div class="line number8 index7 alt1"><code class="bash plain">body { width: 35em; margin: 0 auto;</code></div><div class="line number9 index8 alt2"><code class="bash plain">font-family: Tahoma, Verdana, Arial, sans-serif; }</code></div><div class="line number10 index9 alt1"><code class="bash plain"><</code><code class="bash plain">/style</code><code class="bash plain">></code></div><div class="line number11 index10 alt2"><code class="bash plain"><</code><code class="bash plain">/head</code><code class="bash plain">></code></div><div class="line number12 index11 alt1"><code class="bash plain"><body></code></div><div class="line number13 index12 alt2"><code class="bash plain"><h1>Welcome to nginx!<</code><code class="bash plain">/h1</code><code class="bash plain">></code></div><div class="line number14 index13 alt1"><code class="bash plain"><p>If you see this page, the nginx web server is successfully installed and</code></div><div class="line number15 index14 alt2"><code class="bash plain">working. Further configuration is required.<</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number16 index15 alt1"> </div><div class="line number17 index16 alt2"><code class="bash plain"><p>For online documentation and support please refer to</code></div><div class="line number18 index17 alt1"><code class="bash plain"><a href=</code><code class="bash string">"http://nginx.org/"</code><code class="bash plain">>nginx.org<</code><code class="bash plain">/a</code><code class="bash plain">>.<br/></code></div><div class="line number19 index18 alt2"><code class="bash plain">Commercial support is available at</code></div><div class="line number20 index19 alt1"><code class="bash plain"><a href=</code><code class="bash string">"http://nginx.com/"</code><code class="bash plain">>nginx.com<</code><code class="bash plain">/a</code><code class="bash plain">>.<</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number21 index20 alt2"> </div><div class="line number22 index21 alt1"><code class="bash plain"><p><em>Thank you </code><code class="bash keyword">for</code> <code class="bash plain">using nginx.<</code><code class="bash plain">/em</code><code class="bash plain">><</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number23 index22 alt2"><code class="bash plain"><</code><code class="bash plain">/body</code><code class="bash plain">></code></div><div class="line number24 index23 alt1"><code class="bash plain"><</code><code class="bash plain">/html</code><code class="bash plain">></code></div><div class="line number25 index24 alt2"><code class="bash plain">root@k8s-node01:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　nodeport类型service示例</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_594712" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: Service</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name: ngx-nodeport-svc</code></div><div class="line number5 index4 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: default</code></div><div class="line number6 index5 alt1"><code class="bash plain">spec:</code></div><div class="line number7 index6 alt2"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number8 index7 alt1"><code class="bash spaces">    </code><code class="bash plain">app: deploy-demo</code></div><div class="line number9 index8 alt2"><code class="bash spaces">  </code><code class="bash functions">type</code><code class="bash plain">: NodePort</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash plain">ports:</code></div><div class="line number11 index10 alt2"><code class="bash spaces">  </code><code class="bash plain">- name: http</code></div><div class="line number12 index11 alt1"><code class="bash spaces">    </code><code class="bash plain">protocol: TCP</code></div><div class="line number13 index12 alt2"><code class="bash spaces">    </code><code class="bash plain">port: 80</code></div><div class="line number14 index13 alt1"><code class="bash spaces">    </code><code class="bash plain">targetPort: 80</code></div><div class="line number15 index14 alt2"><code class="bash spaces">    </code><code class="bash plain">nodePort: 30012</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　提示：nodeport类型service只需要在clusterip类型的svc之上修改type为NodePort，然后再ports字段下用nodePort指定对应node端口即可；</p> <p>　　应用配置清单</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_205992" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl apply -f nodeport-svc-demo.yaml</code></div><div class="line number2 index1 alt1"><code class="bash plain">service</code><code class="bash plain">/ngx-nodeport-svc</code> <code class="bash plain">created</code></div><div class="line number3 index2 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl get svc</code></div><div class="line number4 index3 alt1"><code class="bash plain">NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE</code></div><div class="line number5 index4 alt2"><code class="bash plain">kubernetes         ClusterIP   10.100.0.1       <none>        443</code><code class="bash plain">/TCP</code>        <code class="bash plain">16d</code></div><div class="line number6 index5 alt1"><code class="bash plain">ngx-nodeport-svc   NodePort    10.100.209.225   <none>        80:30012</code><code class="bash plain">/TCP</code>   <code class="bash plain">11s</code></div><div class="line number7 index6 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl describe svc ngx-nodeport-svc</code></div><div class="line number8 index7 alt1"><code class="bash plain">Name:                     ngx-nodeport-svc</code></div><div class="line number9 index8 alt2"><code class="bash plain">Namespace:                default</code></div><div class="line number10 index9 alt1"><code class="bash plain">Labels:                   <none></code></div><div class="line number11 index10 alt2"><code class="bash plain">Annotations:              <none></code></div><div class="line number12 index11 alt1"><code class="bash plain">Selector:                 app=deploy-demo</code></div><div class="line number13 index12 alt2"><code class="bash plain">Type:                     NodePort</code></div><div class="line number14 index13 alt1"><code class="bash plain">IP Family Policy:         SingleStack</code></div><div class="line number15 index14 alt2"><code class="bash plain">IP Families:              IPv4</code></div><div class="line number16 index15 alt1"><code class="bash plain">IP:                       10.100.209.225</code></div><div class="line number17 index16 alt2"><code class="bash plain">IPs:                      10.100.209.225</code></div><div class="line number18 index17 alt1"><code class="bash plain">Port:                     http  80</code><code class="bash plain">/TCP</code></div><div class="line number19 index18 alt2"><code class="bash plain">TargetPort:               80</code><code class="bash plain">/TCP</code></div><div class="line number20 index19 alt1"><code class="bash plain">NodePort:                 http  30012</code><code class="bash plain">/TCP</code></div><div class="line number21 index20 alt2"><code class="bash plain">Endpoints:                10.200.155.178:80,10.200.211.138:80</code></div><div class="line number22 index21 alt1"><code class="bash plain">Session Affinity:         None</code></div><div class="line number23 index22 alt2"><code class="bash plain">External Traffic Policy:  Cluster</code></div><div class="line number24 index23 alt1"><code class="bash plain">Events:                   <none></code></div><div class="line number25 index24 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　验证：访问k8s集群任意node的30012端口，看看对应nginxpod是否能够被访问到？</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_242426" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># curl 192.168.0.34:30012  </code></div><div class="line number2 index1 alt1"><code class="bash plain"><!DOCTYPE html></code></div><div class="line number3 index2 alt2"><code class="bash plain"><html></code></div><div class="line number4 index3 alt1"><code class="bash plain"><</code><code class="bash functions">head</code><code class="bash plain">></code></div><div class="line number5 index4 alt2"><code class="bash plain"><title>Welcome to nginx!<</code><code class="bash plain">/title</code><code class="bash plain">></code></div><div class="line number6 index5 alt1"><code class="bash plain"><style></code></div><div class="line number7 index6 alt2"><code class="bash plain">html { color-scheme: light dark; }</code></div><div class="line number8 index7 alt1"><code class="bash plain">body { width: 35em; margin: 0 auto;</code></div><div class="line number9 index8 alt2"><code class="bash plain">font-family: Tahoma, Verdana, Arial, sans-serif; }</code></div><div class="line number10 index9 alt1"><code class="bash plain"><</code><code class="bash plain">/style</code><code class="bash plain">></code></div><div class="line number11 index10 alt2"><code class="bash plain"><</code><code class="bash plain">/head</code><code class="bash plain">></code></div><div class="line number12 index11 alt1"><code class="bash plain"><body></code></div><div class="line number13 index12 alt2"><code class="bash plain"><h1>Welcome to nginx!<</code><code class="bash plain">/h1</code><code class="bash plain">></code></div><div class="line number14 index13 alt1"><code class="bash plain"><p>If you see this page, the nginx web server is successfully installed and</code></div><div class="line number15 index14 alt2"><code class="bash plain">working. Further configuration is required.<</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number16 index15 alt1"> </div><div class="line number17 index16 alt2"><code class="bash plain"><p>For online documentation and support please refer to</code></div><div class="line number18 index17 alt1"><code class="bash plain"><a href=</code><code class="bash string">"http://nginx.org/"</code><code class="bash plain">>nginx.org<</code><code class="bash plain">/a</code><code class="bash plain">>.<br/></code></div><div class="line number19 index18 alt2"><code class="bash plain">Commercial support is available at</code></div><div class="line number20 index19 alt1"><code class="bash plain"><a href=</code><code class="bash string">"http://nginx.com/"</code><code class="bash plain">>nginx.com<</code><code class="bash plain">/a</code><code class="bash plain">>.<</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number21 index20 alt2"> </div><div class="line number22 index21 alt1"><code class="bash plain"><p><em>Thank you </code><code class="bash keyword">for</code> <code class="bash plain">using nginx.<</code><code class="bash plain">/em</code><code class="bash plain">><</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number23 index22 alt2"><code class="bash plain"><</code><code class="bash plain">/body</code><code class="bash plain">></code></div><div class="line number24 index23 alt1"><code class="bash plain"><</code><code class="bash plain">/html</code><code class="bash plain">></code></div><div class="line number25 index24 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　提示：可以看到k8s外部客户端访问k8snode节点的30012端口是能够正常访问到nginxpod；当然集群内部的客户端是可以通过对应生成的clusterip进行访问的；</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_965146" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-node01:~</code><code class="bash comments"># curl 10.100.209.225:30012</code></div><div class="line number2 index1 alt1"><code class="bash plain">curl: (7) Failed to connect to 10.100.209.225 port 30012 after 0 ms: Connection refused</code></div><div class="line number3 index2 alt2"><code class="bash plain">root@k8s-node01:~</code><code class="bash comments"># curl 127.0.0.1:30012     </code></div><div class="line number4 index3 alt1"><code class="bash plain">curl: (7) Failed to connect to 127.0.0.1 port 30012 after 0 ms: Connection refused</code></div><div class="line number5 index4 alt2"><code class="bash plain">root@k8s-node01:~</code><code class="bash comments"># curl 192.168.0.34:30012</code></div><div class="line number6 index5 alt1"><code class="bash plain"><!DOCTYPE html></code></div><div class="line number7 index6 alt2"><code class="bash plain"><html></code></div><div class="line number8 index7 alt1"><code class="bash plain"><</code><code class="bash functions">head</code><code class="bash plain">></code></div><div class="line number9 index8 alt2"><code class="bash plain"><title>Welcome to nginx!<</code><code class="bash plain">/title</code><code class="bash plain">></code></div><div class="line number10 index9 alt1"><code class="bash plain"><style></code></div><div class="line number11 index10 alt2"><code class="bash plain">html { color-scheme: light dark; }</code></div><div class="line number12 index11 alt1"><code class="bash plain">body { width: 35em; margin: 0 auto;</code></div><div class="line number13 index12 alt2"><code class="bash plain">font-family: Tahoma, Verdana, Arial, sans-serif; }</code></div><div class="line number14 index13 alt1"><code class="bash plain"><</code><code class="bash plain">/style</code><code class="bash plain">></code></div><div class="line number15 index14 alt2"><code class="bash plain"><</code><code class="bash plain">/head</code><code class="bash plain">></code></div><div class="line number16 index15 alt1"><code class="bash plain"><body></code></div><div class="line number17 index16 alt2"><code class="bash plain"><h1>Welcome to nginx!<</code><code class="bash plain">/h1</code><code class="bash plain">></code></div><div class="line number18 index17 alt1"><code class="bash plain"><p>If you see this page, the nginx web server is successfully installed and</code></div><div class="line number19 index18 alt2"><code class="bash plain">working. Further configuration is required.<</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number20 index19 alt1"> </div><div class="line number21 index20 alt2"><code class="bash plain"><p>For online documentation and support please refer to</code></div><div class="line number22 index21 alt1"><code class="bash plain"><a href=</code><code class="bash string">"http://nginx.org/"</code><code class="bash plain">>nginx.org<</code><code class="bash plain">/a</code><code class="bash plain">>.<br/></code></div><div class="line number23 index22 alt2"><code class="bash plain">Commercial support is available at</code></div><div class="line number24 index23 alt1"><code class="bash plain"><a href=</code><code class="bash string">"http://nginx.com/"</code><code class="bash plain">>nginx.com<</code><code class="bash plain">/a</code><code class="bash plain">>.<</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number25 index24 alt2"> </div><div class="line number26 index25 alt1"><code class="bash plain"><p><em>Thank you </code><code class="bash keyword">for</code> <code class="bash plain">using nginx.<</code><code class="bash plain">/em</code><code class="bash plain">><</code><code class="bash plain">/p</code><code class="bash plain">></code></div><div class="line number27 index26 alt2"><code class="bash plain"><</code><code class="bash plain">/body</code><code class="bash plain">></code></div><div class="line number28 index27 alt1"><code class="bash plain"><</code><code class="bash plain">/html</code><code class="bash plain">></code></div><div class="line number29 index28 alt2"><code class="bash plain">root@k8s-node01:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　提示：集群内部客户端只能访问clusterip的80端口，或者访问node的对外IP的30012端口；</p> <p>　　Volume资源，详细说明请参考<a href="https://www.cnblogs.com/qiuhom-1874/p/14180752.html" target="_blank" rel="noopener">https://www.cnblogs.com/qiuhom-1874/p/14180752.html</a>；</p> <p>　　pod挂载nfs的使用</p> <p>　　在nfs服务器上准备数据目录</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_657931" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div><div class="line number30 index29 alt1">30</div><div class="line number31 index30 alt2">31</div><div class="line number32 index31 alt1">32</div><div class="line number33 index32 alt2">33</div><div class="line number34 index33 alt1">34</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># cat /etc/exports</code></div><div class="line number2 index1 alt1"><code class="bash comments"># /etc/exports: the access control list for filesystems which may be exported</code></div><div class="line number3 index2 alt2"><code class="bash comments">#               to NFS clients.  See exports(5).</code></div><div class="line number4 index3 alt1"><code class="bash comments">#</code></div><div class="line number5 index4 alt2"><code class="bash comments"># Example for NFSv2 and NFSv3:</code></div><div class="line number6 index5 alt1"><code class="bash comments"># /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)</code></div><div class="line number7 index6 alt2"><code class="bash comments">#</code></div><div class="line number8 index7 alt1"><code class="bash comments"># Example for NFSv4:</code></div><div class="line number9 index8 alt2"><code class="bash comments"># /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)</code></div><div class="line number10 index9 alt1"><code class="bash comments"># /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)</code></div><div class="line number11 index10 alt2"><code class="bash comments">#</code></div><div class="line number12 index11 alt1"><code class="bash plain">/data/k8sdata/kuboard</code> <code class="bash plain">*(rw,no_root_squash)</code></div><div class="line number13 index12 alt2"><code class="bash plain">/data/volumes</code> <code class="bash plain">*(rw,no_root_squash)</code></div><div class="line number14 index13 alt1"><code class="bash plain">/pod-vol</code> <code class="bash plain">*(rw,no_root_squash)</code></div><div class="line number15 index14 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># mkdir -p /pod-vol</code></div><div class="line number16 index15 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># ls /pod-vol -d</code></div><div class="line number17 index16 alt2"><code class="bash plain">/pod-vol</code></div><div class="line number18 index17 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># exportfs -av</code></div><div class="line number19 index18 alt2"><code class="bash plain">exportfs: </code><code class="bash plain">/etc/exports</code> <code class="bash plain">[1]: Neither </code><code class="bash string">'subtree_check'</code> <code class="bash plain">or </code><code class="bash string">'no_subtree_check'</code> <code class="bash plain">specified </code><code class="bash keyword">for</code> <code class="bash functions">export</code> <code class="bash string">"*:/data/k8sdata/kuboard"</code><code class="bash plain">.</code></div><div class="line number20 index19 alt1"><code class="bash spaces">  </code><code class="bash plain">Assuming default behaviour (</code><code class="bash string">'no_subtree_check'</code><code class="bash plain">).</code></div><div class="line number21 index20 alt2"><code class="bash spaces">  </code><code class="bash plain">NOTE: this default has changed since nfs-utils version 1.0.x</code></div><div class="line number22 index21 alt1"> </div><div class="line number23 index22 alt2"><code class="bash plain">exportfs: </code><code class="bash plain">/etc/exports</code> <code class="bash plain">[2]: Neither </code><code class="bash string">'subtree_check'</code> <code class="bash plain">or </code><code class="bash string">'no_subtree_check'</code> <code class="bash plain">specified </code><code class="bash keyword">for</code> <code class="bash functions">export</code> <code class="bash string">"*:/data/volumes"</code><code class="bash plain">.</code></div><div class="line number24 index23 alt1"><code class="bash spaces">  </code><code class="bash plain">Assuming default behaviour (</code><code class="bash string">'no_subtree_check'</code><code class="bash plain">).</code></div><div class="line number25 index24 alt2"><code class="bash spaces">  </code><code class="bash plain">NOTE: this default has changed since nfs-utils version 1.0.x</code></div><div class="line number26 index25 alt1"> </div><div class="line number27 index26 alt2"><code class="bash plain">exportfs: </code><code class="bash plain">/etc/exports</code> <code class="bash plain">[3]: Neither </code><code class="bash string">'subtree_check'</code> <code class="bash plain">or </code><code class="bash string">'no_subtree_check'</code> <code class="bash plain">specified </code><code class="bash keyword">for</code> <code class="bash functions">export</code> <code class="bash string">"*:/pod-vol"</code><code class="bash plain">.</code></div><div class="line number28 index27 alt1"><code class="bash spaces">  </code><code class="bash plain">Assuming default behaviour (</code><code class="bash string">'no_subtree_check'</code><code class="bash plain">).</code></div><div class="line number29 index28 alt2"><code class="bash spaces">  </code><code class="bash plain">NOTE: this default has changed since nfs-utils version 1.0.x</code></div><div class="line number30 index29 alt1"> </div><div class="line number31 index30 alt2"><code class="bash plain">exporting *:</code><code class="bash plain">/pod-vol</code></div><div class="line number32 index31 alt1"><code class="bash plain">exporting *:</code><code class="bash plain">/data/volumes</code></div><div class="line number33 index32 alt2"><code class="bash plain">exporting *:</code><code class="bash plain">/data/k8sdata/kuboard</code></div><div class="line number34 index33 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　在pod上挂载nfs目录</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_407880" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div><div class="line number30 index29 alt1">30</div><div class="line number31 index30 alt2">31</div><div class="line number32 index31 alt1">32</div><div class="line number33 index32 alt2">33</div><div class="line number34 index33 alt1">34</div><div class="line number35 index34 alt2">35</div><div class="line number36 index35 alt1">36</div><div class="line number37 index36 alt2">37</div><div class="line number38 index37 alt1">38</div><div class="line number39 index38 alt2">39</div><div class="line number40 index39 alt1">40</div><div class="line number41 index40 alt2">41</div><div class="line number42 index41 alt1">42</div><div class="line number43 index42 alt2">43</div><div class="line number44 index43 alt1">44</div><div class="line number45 index44 alt2">45</div><div class="line number46 index45 alt1">46</div><div class="line number47 index46 alt2">47</div><div class="line number48 index47 alt1">48</div><div class="line number49 index48 alt2">49</div><div class="line number50 index49 alt1">50</div><div class="line number51 index50 alt2">51</div><div class="line number52 index51 alt1">52</div><div class="line number53 index52 alt2">53</div><div class="line number54 index53 alt1">54</div><div class="line number55 index54 alt2">55</div><div class="line number56 index55 alt1">56</div><div class="line number57 index56 alt2">57</div><div class="line number58 index57 alt1">58</div><div class="line number59 index58 alt2">59</div><div class="line number60 index59 alt1">60</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: apps</code><code class="bash plain">/v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: Deployment</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name:  ngx-nfs-80</code></div><div class="line number5 index4 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: default</code></div><div class="line number6 index5 alt1"><code class="bash spaces">  </code><code class="bash plain">labels:</code></div><div class="line number7 index6 alt2"><code class="bash spaces">    </code><code class="bash plain">app:  ngx-nfs-80</code></div><div class="line number8 index7 alt1"><code class="bash plain">spec:</code></div><div class="line number9 index8 alt2"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number10 index9 alt1"><code class="bash spaces">    </code><code class="bash plain">matchLabels:</code></div><div class="line number11 index10 alt2"><code class="bash spaces">      </code><code class="bash plain">app: ngx-nfs-80</code></div><div class="line number12 index11 alt1"><code class="bash spaces">  </code><code class="bash plain">replicas: 1</code></div><div class="line number13 index12 alt2"><code class="bash spaces">  </code><code class="bash plain">template:</code></div><div class="line number14 index13 alt1"><code class="bash spaces">    </code><code class="bash plain">metadata:</code></div><div class="line number15 index14 alt2"><code class="bash spaces">      </code><code class="bash plain">labels:</code></div><div class="line number16 index15 alt1"><code class="bash spaces">        </code><code class="bash plain">app:  ngx-nfs-80</code></div><div class="line number17 index16 alt2"><code class="bash spaces">    </code><code class="bash plain">spec:</code></div><div class="line number18 index17 alt1"><code class="bash spaces">      </code><code class="bash plain">containers:</code></div><div class="line number19 index18 alt2"><code class="bash spaces">      </code><code class="bash plain">- name:  ngx-nfs-80</code></div><div class="line number20 index19 alt1"><code class="bash spaces">        </code><code class="bash plain">image:  </code><code class="bash string">"harbor.ik8s.cc/baseimages/nginx:v1"</code></div><div class="line number21 index20 alt2"><code class="bash spaces">        </code><code class="bash plain">resources:</code></div><div class="line number22 index21 alt1"><code class="bash spaces">          </code><code class="bash plain">requests:</code></div><div class="line number23 index22 alt2"><code class="bash spaces">            </code><code class="bash plain">cpu: 100m</code></div><div class="line number24 index23 alt1"><code class="bash spaces">            </code><code class="bash plain">memory: 100Mi</code></div><div class="line number25 index24 alt2"><code class="bash spaces">          </code><code class="bash plain">limits:</code></div><div class="line number26 index25 alt1"><code class="bash spaces">            </code><code class="bash plain">cpu: 100m</code></div><div class="line number27 index26 alt2"><code class="bash spaces">            </code><code class="bash plain">memory: 100Mi</code></div><div class="line number28 index27 alt1"><code class="bash spaces">        </code><code class="bash plain">ports:</code></div><div class="line number29 index28 alt2"><code class="bash spaces">        </code><code class="bash plain">- containerPort:  80</code></div><div class="line number30 index29 alt1"><code class="bash spaces">          </code><code class="bash plain">name:  ngx-nfs-80</code></div><div class="line number31 index30 alt2"><code class="bash spaces">        </code><code class="bash plain">volumeMounts:</code></div><div class="line number32 index31 alt1"><code class="bash spaces">        </code><code class="bash plain">- name: localtime</code></div><div class="line number33 index32 alt2"><code class="bash spaces">          </code><code class="bash plain">mountPath: </code><code class="bash plain">/etc/localtime</code></div><div class="line number34 index33 alt1"><code class="bash spaces">        </code><code class="bash plain">- name: nfs-vol</code></div><div class="line number35 index34 alt2"><code class="bash spaces">          </code><code class="bash plain">mountPath: </code><code class="bash plain">/usr/share/nginx/html/</code></div><div class="line number36 index35 alt1"><code class="bash spaces">      </code><code class="bash plain">volumes:</code></div><div class="line number37 index36 alt2"><code class="bash spaces">        </code><code class="bash plain">- name: localtime</code></div><div class="line number38 index37 alt1"><code class="bash spaces">          </code><code class="bash plain">hostPath:</code></div><div class="line number39 index38 alt2"><code class="bash spaces">            </code><code class="bash plain">path: </code><code class="bash plain">/usr/share/zoneinfo/Asia/Shanghai</code></div><div class="line number40 index39 alt1"><code class="bash spaces">        </code><code class="bash plain">- name: nfs-vol</code></div><div class="line number41 index40 alt2"><code class="bash spaces">          </code><code class="bash plain">nfs:</code></div><div class="line number42 index41 alt1"><code class="bash spaces">            </code><code class="bash plain">server: 192.168.0.42</code></div><div class="line number43 index42 alt2"><code class="bash spaces">            </code><code class="bash plain">path: </code><code class="bash plain">/pod-vol</code></div><div class="line number44 index43 alt1"><code class="bash spaces">      </code><code class="bash plain">restartPolicy: Always</code></div><div class="line number45 index44 alt2"><code class="bash plain">---</code></div><div class="line number46 index45 alt1"><code class="bash plain">apiVersion: v1</code></div><div class="line number47 index46 alt2"><code class="bash plain">kind: Service</code></div><div class="line number48 index47 alt1"><code class="bash plain">metadata:</code></div><div class="line number49 index48 alt2"><code class="bash spaces">  </code><code class="bash plain">name: ngx-nfs-svc</code></div><div class="line number50 index49 alt1"><code class="bash spaces">  </code><code class="bash plain">namespace: default</code></div><div class="line number51 index50 alt2"><code class="bash plain">spec:</code></div><div class="line number52 index51 alt1"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number53 index52 alt2"><code class="bash spaces">    </code><code class="bash plain">app: ngx-nfs-80</code></div><div class="line number54 index53 alt1"><code class="bash spaces">  </code><code class="bash functions">type</code><code class="bash plain">: NodePort</code></div><div class="line number55 index54 alt2"><code class="bash spaces">  </code><code class="bash plain">ports:</code></div><div class="line number56 index55 alt1"><code class="bash spaces">  </code><code class="bash plain">- name: ngx-nfs-svc</code></div><div class="line number57 index56 alt2"><code class="bash spaces">    </code><code class="bash plain">protocol: TCP</code></div><div class="line number58 index57 alt1"><code class="bash spaces">    </code><code class="bash plain">port: 80</code></div><div class="line number59 index58 alt2"><code class="bash spaces">    </code><code class="bash plain">targetPort: 80</code></div><div class="line number60 index59 alt1"><code class="bash spaces">    </code><code class="bash plain">nodePort: 30013</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　应用配置清单</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_99004" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl apply -f nfs-vol.yaml </code></div><div class="line number2 index1 alt1"><code class="bash plain">deployment.apps</code><code class="bash plain">/ngx-nfs-80</code> <code class="bash plain">created</code></div><div class="line number3 index2 alt2"><code class="bash plain">service</code><code class="bash plain">/ngx-nfs-svc</code> <code class="bash plain">created</code></div><div class="line number4 index3 alt1"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl get pods </code></div><div class="line number5 index4 alt2"><code class="bash plain">NAME                           READY   STATUS    RESTARTS      AGE</code></div><div class="line number6 index5 alt1"><code class="bash plain">deploy-demo-6849bdf444-pvsc9   1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (57m ago)   46h</code></div><div class="line number7 index6 alt2"><code class="bash plain">deploy-demo-6849bdf444-sg8fz   1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (57m ago)   46h</code></div><div class="line number8 index7 alt1"><code class="bash plain">ng-rc-l7xmp                    1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (57m ago)   47h</code></div><div class="line number9 index8 alt2"><code class="bash plain">ng-rc-wl5d6                    1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (57m ago)   47h</code></div><div class="line number10 index9 alt1"><code class="bash plain">ngx-nfs-80-66c9697cf4-8pm9k    1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   0             7s</code></div><div class="line number11 index10 alt2"><code class="bash plain">rs-demo-nzmqs                  1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (57m ago)   47h</code></div><div class="line number12 index11 alt1"><code class="bash plain">rs-demo-v2vb6                  1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (57m ago)   47h</code></div><div class="line number13 index12 alt2"><code class="bash plain">rs-demo-x27fv                  1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (57m ago)   47h</code></div><div class="line number14 index13 alt1"><code class="bash functions">test</code>                           <code class="bash plain">1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   7 (57m ago)   17d</code></div><div class="line number15 index14 alt2"><code class="bash plain">test1                          1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   7 (57m ago)   17d</code></div><div class="line number16 index15 alt1"><code class="bash plain">test2                          1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   7 (57m ago)   17d</code></div><div class="line number17 index16 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl get svc</code></div><div class="line number18 index17 alt1"><code class="bash plain">NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE</code></div><div class="line number19 index18 alt2"><code class="bash plain">kubernetes         ClusterIP   10.100.0.1       <none>        443</code><code class="bash plain">/TCP</code>        <code class="bash plain">18d</code></div><div class="line number20 index19 alt1"><code class="bash plain">ngx-nfs-svc        NodePort    10.100.16.14     <none>        80:30013</code><code class="bash plain">/TCP</code>   <code class="bash plain">15s</code></div><div class="line number21 index20 alt2"><code class="bash plain">ngx-nodeport-svc   NodePort    10.100.209.225   <none>        80:30012</code><code class="bash plain">/TCP</code>   <code class="bash plain">45h</code></div><div class="line number22 index21 alt1"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　在nfs服务器上/pod-vol目录下提供index.html文件</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_853438" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># echo "this page from nfs server.." >> /pod-vol/index.html</code></div><div class="line number2 index1 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># cat /pod-vol/index.html</code></div><div class="line number3 index2 alt2"><code class="bash plain">this page from nfs server..</code></div><div class="line number4 index3 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　访问pod，看看nfs服务器上的inde.html是否能够正常访问到？</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_148300" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># curl 192.168.0.35:30013</code></div><div class="line number2 index1 alt1"><code class="bash plain">this page from nfs server..</code></div><div class="line number3 index2 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　提示：能够看到访问pod对应返回的页面就是刚才在nfs服务器上创建的页面，说明pod正常挂载了nfs提供的目录；</p> <p>　　PV、PVC资源，详细说明请参考<a href="https://www.cnblogs.com/qiuhom-1874/p/14188621.html" target="_blank" rel="noopener">https://www.cnblogs.com/qiuhom-1874/p/14188621.html</a>；</p> <p>　　nfs实现静态pvc的使用</p> <p>　　在nfs服务器上准备目录</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_582526" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div><div class="line number30 index29 alt1">30</div><div class="line number31 index30 alt2">31</div><div class="line number32 index31 alt1">32</div><div class="line number33 index32 alt2">33</div><div class="line number34 index33 alt1">34</div><div class="line number35 index34 alt2">35</div><div class="line number36 index35 alt1">36</div><div class="line number37 index36 alt2">37</div><div class="line number38 index37 alt1">38</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># cat /etc/exports </code></div><div class="line number2 index1 alt1"><code class="bash comments"># /etc/exports: the access control list for filesystems which may be exported</code></div><div class="line number3 index2 alt2"><code class="bash comments">#               to NFS clients.  See exports(5).</code></div><div class="line number4 index3 alt1"><code class="bash comments">#</code></div><div class="line number5 index4 alt2"><code class="bash comments"># Example for NFSv2 and NFSv3:</code></div><div class="line number6 index5 alt1"><code class="bash comments"># /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)</code></div><div class="line number7 index6 alt2"><code class="bash comments">#</code></div><div class="line number8 index7 alt1"><code class="bash comments"># Example for NFSv4:</code></div><div class="line number9 index8 alt2"><code class="bash comments"># /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)</code></div><div class="line number10 index9 alt1"><code class="bash comments"># /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)</code></div><div class="line number11 index10 alt2"><code class="bash comments">#</code></div><div class="line number12 index11 alt1"><code class="bash plain">/data/k8sdata/kuboard</code> <code class="bash plain">*(rw,no_root_squash)</code></div><div class="line number13 index12 alt2"><code class="bash plain">/data/volumes</code> <code class="bash plain">*(rw,no_root_squash)</code></div><div class="line number14 index13 alt1"><code class="bash plain">/pod-vol</code> <code class="bash plain">*(rw,no_root_squash)</code></div><div class="line number15 index14 alt2"><code class="bash plain">/data/k8sdata/myserver/myappdata</code> <code class="bash plain">*(rw,no_root_squash)</code></div><div class="line number16 index15 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># mkdir -p /data/k8sdata/myserver/myappdata</code></div><div class="line number17 index16 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># exportfs -av</code></div><div class="line number18 index17 alt1"><code class="bash plain">exportfs: </code><code class="bash plain">/etc/exports</code> <code class="bash plain">[1]: Neither </code><code class="bash string">'subtree_check'</code> <code class="bash plain">or </code><code class="bash string">'no_subtree_check'</code> <code class="bash plain">specified </code><code class="bash keyword">for</code> <code class="bash functions">export</code> <code class="bash string">"*:/data/k8sdata/kuboard"</code><code class="bash plain">.</code></div><div class="line number19 index18 alt2"><code class="bash spaces">  </code><code class="bash plain">Assuming default behaviour (</code><code class="bash string">'no_subtree_check'</code><code class="bash plain">).</code></div><div class="line number20 index19 alt1"><code class="bash spaces">  </code><code class="bash plain">NOTE: this default has changed since nfs-utils version 1.0.x</code></div><div class="line number21 index20 alt2"> </div><div class="line number22 index21 alt1"><code class="bash plain">exportfs: </code><code class="bash plain">/etc/exports</code> <code class="bash plain">[2]: Neither </code><code class="bash string">'subtree_check'</code> <code class="bash plain">or </code><code class="bash string">'no_subtree_check'</code> <code class="bash plain">specified </code><code class="bash keyword">for</code> <code class="bash functions">export</code> <code class="bash string">"*:/data/volumes"</code><code class="bash plain">.</code></div><div class="line number23 index22 alt2"><code class="bash spaces">  </code><code class="bash plain">Assuming default behaviour (</code><code class="bash string">'no_subtree_check'</code><code class="bash plain">).</code></div><div class="line number24 index23 alt1"><code class="bash spaces">  </code><code class="bash plain">NOTE: this default has changed since nfs-utils version 1.0.x</code></div><div class="line number25 index24 alt2"> </div><div class="line number26 index25 alt1"><code class="bash plain">exportfs: </code><code class="bash plain">/etc/exports</code> <code class="bash plain">[3]: Neither </code><code class="bash string">'subtree_check'</code> <code class="bash plain">or </code><code class="bash string">'no_subtree_check'</code> <code class="bash plain">specified </code><code class="bash keyword">for</code> <code class="bash functions">export</code> <code class="bash string">"*:/pod-vol"</code><code class="bash plain">.</code></div><div class="line number27 index26 alt2"><code class="bash spaces">  </code><code class="bash plain">Assuming default behaviour (</code><code class="bash string">'no_subtree_check'</code><code class="bash plain">).</code></div><div class="line number28 index27 alt1"><code class="bash spaces">  </code><code class="bash plain">NOTE: this default has changed since nfs-utils version 1.0.x</code></div><div class="line number29 index28 alt2"> </div><div class="line number30 index29 alt1"><code class="bash plain">exportfs: </code><code class="bash plain">/etc/exports</code> <code class="bash plain">[4]: Neither </code><code class="bash string">'subtree_check'</code> <code class="bash plain">or </code><code class="bash string">'no_subtree_check'</code> <code class="bash plain">specified </code><code class="bash keyword">for</code> <code class="bash functions">export</code> <code class="bash string">"*:/data/k8sdata/myserver/myappdata"</code><code class="bash plain">.</code></div><div class="line number31 index30 alt2"><code class="bash spaces">  </code><code class="bash plain">Assuming default behaviour (</code><code class="bash string">'no_subtree_check'</code><code class="bash plain">).</code></div><div class="line number32 index31 alt1"><code class="bash spaces">  </code><code class="bash plain">NOTE: this default has changed since nfs-utils version 1.0.x</code></div><div class="line number33 index32 alt2"> </div><div class="line number34 index33 alt1"><code class="bash plain">exporting *:</code><code class="bash plain">/data/k8sdata/myserver/myappdata</code></div><div class="line number35 index34 alt2"><code class="bash plain">exporting *:</code><code class="bash plain">/pod-vol</code></div><div class="line number36 index35 alt1"><code class="bash plain">exporting *:</code><code class="bash plain">/data/volumes</code></div><div class="line number37 index36 alt2"><code class="bash plain">exporting *:</code><code class="bash plain">/data/k8sdata/kuboard</code></div><div class="line number38 index37 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　创建pv</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_958954" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: PersistentVolume</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name: myapp-static-pv</code></div><div class="line number5 index4 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: default<br>spec:</code></div><div class="line number6 index5 alt1"><code class="bash spaces">  </code><code class="bash plain">capacity:</code></div><div class="line number7 index6 alt2"><code class="bash spaces">    </code><code class="bash plain">storage: 2Gi</code></div><div class="line number8 index7 alt1"><code class="bash spaces">  </code><code class="bash plain">accessModes:</code></div><div class="line number9 index8 alt2"><code class="bash spaces">    </code><code class="bash plain">- ReadWriteOnce</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash plain">nfs:</code></div><div class="line number11 index10 alt2"><code class="bash spaces">    </code><code class="bash plain">path: </code><code class="bash plain">/data/k8sdata/myserver/myappdata</code></div><div class="line number12 index11 alt1"><code class="bash spaces">    </code><code class="bash plain">server: 192.168.0.42</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　创建pvc关联pv</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_560593" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: PersistentVolumeClaim</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name: myapp-static-pvc</code></div><div class="line number5 index4 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: default</code></div><div class="line number6 index5 alt1"><code class="bash plain">spec:</code></div><div class="line number7 index6 alt2"><code class="bash spaces">  </code><code class="bash plain">volumeName: myapp-static-pv</code></div><div class="line number8 index7 alt1"><code class="bash spaces">  </code><code class="bash plain">accessModes:</code></div><div class="line number9 index8 alt2"><code class="bash spaces">    </code><code class="bash plain">- ReadWriteOnce</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash plain">resources:</code></div><div class="line number11 index10 alt2"><code class="bash spaces">    </code><code class="bash plain">requests:</code></div><div class="line number12 index11 alt1"><code class="bash spaces">      </code><code class="bash plain">storage: 2Gi</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　创建pod使用pvc</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_212741" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div><div class="line number30 index29 alt1">30</div><div class="line number31 index30 alt2">31</div><div class="line number32 index31 alt1">32</div><div class="line number33 index32 alt2">33</div><div class="line number34 index33 alt1">34</div><div class="line number35 index34 alt2">35</div><div class="line number36 index35 alt1">36</div><div class="line number37 index36 alt2">37</div><div class="line number38 index37 alt1">38</div><div class="line number39 index38 alt2">39</div><div class="line number40 index39 alt1">40</div><div class="line number41 index40 alt2">41</div><div class="line number42 index41 alt1">42</div><div class="line number43 index42 alt2">43</div><div class="line number44 index43 alt1">44</div><div class="line number45 index44 alt2">45</div><div class="line number46 index45 alt1">46</div><div class="line number47 index46 alt2">47</div><div class="line number48 index47 alt1">48</div><div class="line number49 index48 alt2">49</div><div class="line number50 index49 alt1">50</div><div class="line number51 index50 alt2">51</div><div class="line number52 index51 alt1">52</div><div class="line number53 index52 alt2">53</div><div class="line number54 index53 alt1">54</div><div class="line number55 index54 alt2">55</div><div class="line number56 index55 alt1">56</div><div class="line number57 index56 alt2">57</div><div class="line number58 index57 alt1">58</div><div class="line number59 index58 alt2">59</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: apps</code><code class="bash plain">/v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: Deployment</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name:  ngx-nfs-pvc-80</code></div><div class="line number5 index4 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: default</code></div><div class="line number6 index5 alt1"><code class="bash spaces">  </code><code class="bash plain">labels:</code></div><div class="line number7 index6 alt2"><code class="bash spaces">    </code><code class="bash plain">app:  ngx-pvc-80</code></div><div class="line number8 index7 alt1"><code class="bash plain">spec:</code></div><div class="line number9 index8 alt2"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number10 index9 alt1"><code class="bash spaces">    </code><code class="bash plain">matchLabels:</code></div><div class="line number11 index10 alt2"><code class="bash spaces">      </code><code class="bash plain">app: ngx-pvc-80</code></div><div class="line number12 index11 alt1"><code class="bash spaces">  </code><code class="bash plain">replicas: 1</code></div><div class="line number13 index12 alt2"><code class="bash spaces">  </code><code class="bash plain">template:</code></div><div class="line number14 index13 alt1"><code class="bash spaces">    </code><code class="bash plain">metadata:</code></div><div class="line number15 index14 alt2"><code class="bash spaces">      </code><code class="bash plain">labels:</code></div><div class="line number16 index15 alt1"><code class="bash spaces">        </code><code class="bash plain">app:  ngx-pvc-80</code></div><div class="line number17 index16 alt2"><code class="bash spaces">    </code><code class="bash plain">spec:</code></div><div class="line number18 index17 alt1"><code class="bash spaces">      </code><code class="bash plain">containers:</code></div><div class="line number19 index18 alt2"><code class="bash spaces">      </code><code class="bash plain">- name:  ngx-pvc-80</code></div><div class="line number20 index19 alt1"><code class="bash spaces">        </code><code class="bash plain">image:  </code><code class="bash string">"harbor.ik8s.cc/baseimages/nginx:v1"</code></div><div class="line number21 index20 alt2"><code class="bash spaces">        </code><code class="bash plain">resources:</code></div><div class="line number22 index21 alt1"><code class="bash spaces">          </code><code class="bash plain">requests:</code></div><div class="line number23 index22 alt2"><code class="bash spaces">            </code><code class="bash plain">cpu: 100m</code></div><div class="line number24 index23 alt1"><code class="bash spaces">            </code><code class="bash plain">memory: 100Mi</code></div><div class="line number25 index24 alt2"><code class="bash spaces">          </code><code class="bash plain">limits:</code></div><div class="line number26 index25 alt1"><code class="bash spaces">            </code><code class="bash plain">cpu: 100m</code></div><div class="line number27 index26 alt2"><code class="bash spaces">            </code><code class="bash plain">memory: 100Mi</code></div><div class="line number28 index27 alt1"><code class="bash spaces">        </code><code class="bash plain">ports:</code></div><div class="line number29 index28 alt2"><code class="bash spaces">        </code><code class="bash plain">- containerPort:  80</code></div><div class="line number30 index29 alt1"><code class="bash spaces">          </code><code class="bash plain">name:  ngx-pvc-80</code></div><div class="line number31 index30 alt2"><code class="bash spaces">        </code><code class="bash plain">volumeMounts:</code></div><div class="line number32 index31 alt1"><code class="bash spaces">        </code><code class="bash plain">- name: localtime</code></div><div class="line number33 index32 alt2"><code class="bash spaces">          </code><code class="bash plain">mountPath: </code><code class="bash plain">/etc/localtime</code></div><div class="line number34 index33 alt1"><code class="bash spaces">        </code><code class="bash plain">- name: data-pvc</code></div><div class="line number35 index34 alt2"><code class="bash spaces">          </code><code class="bash plain">mountPath: </code><code class="bash plain">/usr/share/nginx/html/</code></div><div class="line number36 index35 alt1"><code class="bash spaces">      </code><code class="bash plain">volumes:</code></div><div class="line number37 index36 alt2"><code class="bash spaces">        </code><code class="bash plain">- name: localtime</code></div><div class="line number38 index37 alt1"><code class="bash spaces">          </code><code class="bash plain">hostPath:</code></div><div class="line number39 index38 alt2"><code class="bash spaces">            </code><code class="bash plain">path: </code><code class="bash plain">/usr/share/zoneinfo/Asia/Shanghai</code></div><div class="line number40 index39 alt1"><code class="bash spaces">        </code><code class="bash plain">- name: data-pvc</code></div><div class="line number41 index40 alt2"><code class="bash spaces">          </code><code class="bash plain">persistentVolumeClaim:</code></div><div class="line number42 index41 alt1"><code class="bash spaces">            </code><code class="bash plain">claimName: myapp-static-pvc </code></div><div class="line number43 index42 alt2"> </div><div class="line number44 index43 alt1"><code class="bash plain">---</code></div><div class="line number45 index44 alt2"><code class="bash plain">apiVersion: v1</code></div><div class="line number46 index45 alt1"><code class="bash plain">kind: Service</code></div><div class="line number47 index46 alt2"><code class="bash plain">metadata:</code></div><div class="line number48 index47 alt1"><code class="bash spaces">  </code><code class="bash plain">name: ngx-pvc-svc</code></div><div class="line number49 index48 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: default</code></div><div class="line number50 index49 alt1"><code class="bash plain">spec:</code></div><div class="line number51 index50 alt2"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number52 index51 alt1"><code class="bash spaces">    </code><code class="bash plain">app: ngx-pvc-80</code></div><div class="line number53 index52 alt2"><code class="bash spaces">  </code><code class="bash functions">type</code><code class="bash plain">: NodePort</code></div><div class="line number54 index53 alt1"><code class="bash spaces">  </code><code class="bash plain">ports:</code></div><div class="line number55 index54 alt2"><code class="bash spaces">  </code><code class="bash plain">- name: ngx-nfs-svc</code></div><div class="line number56 index55 alt1"><code class="bash spaces">    </code><code class="bash plain">protocol: TCP</code></div><div class="line number57 index56 alt2"><code class="bash spaces">    </code><code class="bash plain">port: 80</code></div><div class="line number58 index57 alt1"><code class="bash spaces">    </code><code class="bash plain">targetPort: 80</code></div><div class="line number59 index58 alt2"><code class="bash spaces">    </code><code class="bash plain">nodePort: 30014</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　应用上述配置清单</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_850469" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl apply -f nfs-static-pvc-demo.yaml </code></div><div class="line number2 index1 alt1"><code class="bash plain">persistentvolume</code><code class="bash plain">/myapp-static-pv</code> <code class="bash plain">created</code></div><div class="line number3 index2 alt2"><code class="bash plain">persistentvolumeclaim</code><code class="bash plain">/myapp-static-pvc</code> <code class="bash plain">created</code></div><div class="line number4 index3 alt1"><code class="bash plain">deployment.apps</code><code class="bash plain">/ngx-nfs-pvc-80</code> <code class="bash plain">created</code></div><div class="line number5 index4 alt2"><code class="bash plain">service</code><code class="bash plain">/ngx-pvc-svc</code> <code class="bash plain">created</code></div><div class="line number6 index5 alt1"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl get pv</code></div><div class="line number7 index6 alt2"><code class="bash plain">NAME              CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                      STORAGECLASS   REASON   AGE</code></div><div class="line number8 index7 alt1"><code class="bash plain">myapp-static-pv   2Gi        RWO            Retain           Bound    default</code><code class="bash plain">/myapp-static-pvc</code>                           <code class="bash plain">4s</code></div><div class="line number9 index8 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl get pvc</code></div><div class="line number10 index9 alt1"><code class="bash plain">NAME               STATUS    VOLUME            CAPACITY   ACCESS MODES   STORAGECLASS   AGE</code></div><div class="line number11 index10 alt2"><code class="bash plain">myapp-static-pvc   Pending   myapp-static-pv   0                                        7s</code></div><div class="line number12 index11 alt1"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># kubectl get pods </code></div><div class="line number13 index12 alt2"><code class="bash plain">NAME                            READY   STATUS    RESTARTS       AGE</code></div><div class="line number14 index13 alt1"><code class="bash plain">deploy-demo-6849bdf444-pvsc9    1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (151m ago)   47h</code></div><div class="line number15 index14 alt2"><code class="bash plain">deploy-demo-6849bdf444-sg8fz    1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (151m ago)   47h</code></div><div class="line number16 index15 alt1"><code class="bash plain">ng-rc-l7xmp                     1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (151m ago)   2d1h</code></div><div class="line number17 index16 alt2"><code class="bash plain">ng-rc-wl5d6                     1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (151m ago)   2d1h</code></div><div class="line number18 index17 alt1"><code class="bash plain">ngx-nfs-pvc-80-f776bb6d-nwwwq   0</code><code class="bash plain">/1</code>     <code class="bash plain">Pending   0              10s</code></div><div class="line number19 index18 alt2"><code class="bash plain">rs-demo-nzmqs                   1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (151m ago)   2d</code></div><div class="line number20 index19 alt1"><code class="bash plain">rs-demo-v2vb6                   1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (151m ago)   2d</code></div><div class="line number21 index20 alt2"><code class="bash plain">rs-demo-x27fv                   1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   1 (151m ago)   2d</code></div><div class="line number22 index21 alt1"><code class="bash functions">test</code>                            <code class="bash plain">1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   7 (151m ago)   18d</code></div><div class="line number23 index22 alt2"><code class="bash plain">test1                           1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   7 (151m ago)   18d</code></div><div class="line number24 index23 alt1"><code class="bash plain">test2                           1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   7 (151m ago)   18d</code></div><div class="line number25 index24 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　在nfs服务器上/data/k8sdata/myserver/myappdata创建index.html，看看对应主页是否能够被访问？</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_539242" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># echo "this page from nfs-server /data/k8sdata/myserver/myappdata/index.html" >> /data/k8sdata/myserver/myappdata/index.html</code></div><div class="line number2 index1 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># cat /data/k8sdata/myserver/myappdata/index.html</code></div><div class="line number3 index2 alt2"><code class="bash plain">this page from nfs-server </code><code class="bash plain">/data/k8sdata/myserver/myappdata/index</code><code class="bash plain">.html</code></div><div class="line number4 index3 alt1"><code class="bash plain">root@harbor:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　访问pod</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_215467" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># curl 192.168.0.36:30014</code></div><div class="line number2 index1 alt1"><code class="bash plain">this page from nfs-server </code><code class="bash plain">/data/k8sdata/myserver/myappdata/index</code><code class="bash plain">.html</code></div><div class="line number3 index2 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　nfs实现动态pvc的使用</p> <p> 　　创建名称空间、服务账号、clusterrole、clusterrolebindding、role、rolebinding</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_473469" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div><div class="line number30 index29 alt1">30</div><div class="line number31 index30 alt2">31</div><div class="line number32 index31 alt1">32</div><div class="line number33 index32 alt2">33</div><div class="line number34 index33 alt1">34</div><div class="line number35 index34 alt2">35</div><div class="line number36 index35 alt1">36</div><div class="line number37 index36 alt2">37</div><div class="line number38 index37 alt1">38</div><div class="line number39 index38 alt2">39</div><div class="line number40 index39 alt1">40</div><div class="line number41 index40 alt2">41</div><div class="line number42 index41 alt1">42</div><div class="line number43 index42 alt2">43</div><div class="line number44 index43 alt1">44</div><div class="line number45 index44 alt2">45</div><div class="line number46 index45 alt1">46</div><div class="line number47 index46 alt2">47</div><div class="line number48 index47 alt1">48</div><div class="line number49 index48 alt2">49</div><div class="line number50 index49 alt1">50</div><div class="line number51 index50 alt2">51</div><div class="line number52 index51 alt1">52</div><div class="line number53 index52 alt2">53</div><div class="line number54 index53 alt1">54</div><div class="line number55 index54 alt2">55</div><div class="line number56 index55 alt1">56</div><div class="line number57 index56 alt2">57</div><div class="line number58 index57 alt1">58</div><div class="line number59 index58 alt2">59</div><div class="line number60 index59 alt1">60</div><div class="line number61 index60 alt2">61</div><div class="line number62 index61 alt1">62</div><div class="line number63 index62 alt2">63</div><div class="line number64 index63 alt1">64</div><div class="line number65 index64 alt2">65</div><div class="line number66 index65 alt1">66</div><div class="line number67 index66 alt2">67</div><div class="line number68 index67 alt1">68</div><div class="line number69 index68 alt2">69</div><div class="line number70 index69 alt1">70</div><div class="line number71 index70 alt2">71</div><div class="line number72 index71 alt1">72</div><div class="line number73 index72 alt2">73</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: Namespace</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name: nfs</code></div><div class="line number5 index4 alt2"><code class="bash plain">---</code></div><div class="line number6 index5 alt1"><code class="bash plain">apiVersion: v1</code></div><div class="line number7 index6 alt2"><code class="bash plain">kind: ServiceAccount</code></div><div class="line number8 index7 alt1"><code class="bash plain">metadata:</code></div><div class="line number9 index8 alt2"><code class="bash spaces">  </code><code class="bash plain">name: nfs-client-provisioner</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash comments"># replace with namespace where provisioner is deployed</code></div><div class="line number11 index10 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: nfs</code></div><div class="line number12 index11 alt1"><code class="bash plain">---</code></div><div class="line number13 index12 alt2"><code class="bash plain">kind: ClusterRole</code></div><div class="line number14 index13 alt1"><code class="bash plain">apiVersion: rbac.authorization.k8s.io</code><code class="bash plain">/v1</code></div><div class="line number15 index14 alt2"><code class="bash plain">metadata:</code></div><div class="line number16 index15 alt1"><code class="bash spaces">  </code><code class="bash plain">name: nfs-client-provisioner-runner</code></div><div class="line number17 index16 alt2"><code class="bash plain">rules:</code></div><div class="line number18 index17 alt1"><code class="bash spaces">  </code><code class="bash plain">- apiGroups: [</code><code class="bash string">""</code><code class="bash plain">]</code></div><div class="line number19 index18 alt2"><code class="bash spaces">    </code><code class="bash plain">resources: [</code><code class="bash string">"nodes"</code><code class="bash plain">]</code></div><div class="line number20 index19 alt1"><code class="bash spaces">    </code><code class="bash plain">verbs: [</code><code class="bash string">"get"</code><code class="bash plain">, </code><code class="bash string">"list"</code><code class="bash plain">, </code><code class="bash string">"watch"</code><code class="bash plain">]</code></div><div class="line number21 index20 alt2"><code class="bash spaces">  </code><code class="bash plain">- apiGroups: [</code><code class="bash string">""</code><code class="bash plain">]</code></div><div class="line number22 index21 alt1"><code class="bash spaces">    </code><code class="bash plain">resources: [</code><code class="bash string">"persistentvolumes"</code><code class="bash plain">]</code></div><div class="line number23 index22 alt2"><code class="bash spaces">    </code><code class="bash plain">verbs: [</code><code class="bash string">"get"</code><code class="bash plain">, </code><code class="bash string">"list"</code><code class="bash plain">, </code><code class="bash string">"watch"</code><code class="bash plain">, </code><code class="bash string">"create"</code><code class="bash plain">, </code><code class="bash string">"delete"</code><code class="bash plain">]</code></div><div class="line number24 index23 alt1"><code class="bash spaces">  </code><code class="bash plain">- apiGroups: [</code><code class="bash string">""</code><code class="bash plain">]</code></div><div class="line number25 index24 alt2"><code class="bash spaces">    </code><code class="bash plain">resources: [</code><code class="bash string">"persistentvolumeclaims"</code><code class="bash plain">]</code></div><div class="line number26 index25 alt1"><code class="bash spaces">    </code><code class="bash plain">verbs: [</code><code class="bash string">"get"</code><code class="bash plain">, </code><code class="bash string">"list"</code><code class="bash plain">, </code><code class="bash string">"watch"</code><code class="bash plain">, </code><code class="bash string">"update"</code><code class="bash plain">]</code></div><div class="line number27 index26 alt2"><code class="bash spaces">  </code><code class="bash plain">- apiGroups: [</code><code class="bash string">"storage.k8s.io"</code><code class="bash plain">]</code></div><div class="line number28 index27 alt1"><code class="bash spaces">    </code><code class="bash plain">resources: [</code><code class="bash string">"storageclasses"</code><code class="bash plain">]</code></div><div class="line number29 index28 alt2"><code class="bash spaces">    </code><code class="bash plain">verbs: [</code><code class="bash string">"get"</code><code class="bash plain">, </code><code class="bash string">"list"</code><code class="bash plain">, </code><code class="bash string">"watch"</code><code class="bash plain">]</code></div><div class="line number30 index29 alt1"><code class="bash spaces">  </code><code class="bash plain">- apiGroups: [</code><code class="bash string">""</code><code class="bash plain">]</code></div><div class="line number31 index30 alt2"><code class="bash spaces">    </code><code class="bash plain">resources: [</code><code class="bash string">"events"</code><code class="bash plain">]</code></div><div class="line number32 index31 alt1"><code class="bash spaces">    </code><code class="bash plain">verbs: [</code><code class="bash string">"create"</code><code class="bash plain">, </code><code class="bash string">"update"</code><code class="bash plain">, </code><code class="bash string">"patch"</code><code class="bash plain">]</code></div><div class="line number33 index32 alt2"><code class="bash plain">---</code></div><div class="line number34 index33 alt1"><code class="bash plain">kind: ClusterRoleBinding</code></div><div class="line number35 index34 alt2"><code class="bash plain">apiVersion: rbac.authorization.k8s.io</code><code class="bash plain">/v1</code></div><div class="line number36 index35 alt1"><code class="bash plain">metadata:</code></div><div class="line number37 index36 alt2"><code class="bash spaces">  </code><code class="bash plain">name: run-nfs-client-provisioner</code></div><div class="line number38 index37 alt1"><code class="bash plain">subjects:</code></div><div class="line number39 index38 alt2"><code class="bash spaces">  </code><code class="bash plain">- kind: ServiceAccount</code></div><div class="line number40 index39 alt1"><code class="bash spaces">    </code><code class="bash plain">name: nfs-client-provisioner</code></div><div class="line number41 index40 alt2"><code class="bash spaces">    </code><code class="bash comments"># replace with namespace where provisioner is deployed</code></div><div class="line number42 index41 alt1"><code class="bash spaces">    </code><code class="bash plain">namespace: nfs</code></div><div class="line number43 index42 alt2"><code class="bash plain">roleRef:</code></div><div class="line number44 index43 alt1"><code class="bash spaces">  </code><code class="bash plain">kind: ClusterRole</code></div><div class="line number45 index44 alt2"><code class="bash spaces">  </code><code class="bash plain">name: nfs-client-provisioner-runner</code></div><div class="line number46 index45 alt1"><code class="bash spaces">  </code><code class="bash plain">apiGroup: rbac.authorization.k8s.io</code></div><div class="line number47 index46 alt2"><code class="bash plain">---</code></div><div class="line number48 index47 alt1"><code class="bash plain">kind: Role</code></div><div class="line number49 index48 alt2"><code class="bash plain">apiVersion: rbac.authorization.k8s.io</code><code class="bash plain">/v1</code></div><div class="line number50 index49 alt1"><code class="bash plain">metadata:</code></div><div class="line number51 index50 alt2"><code class="bash spaces">  </code><code class="bash plain">name: leader-locking-nfs-client-provisioner</code></div><div class="line number52 index51 alt1"><code class="bash spaces">  </code><code class="bash comments"># replace with namespace where provisioner is deployed</code></div><div class="line number53 index52 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: nfs</code></div><div class="line number54 index53 alt1"><code class="bash plain">rules:</code></div><div class="line number55 index54 alt2"><code class="bash spaces">  </code><code class="bash plain">- apiGroups: [</code><code class="bash string">""</code><code class="bash plain">]</code></div><div class="line number56 index55 alt1"><code class="bash spaces">    </code><code class="bash plain">resources: [</code><code class="bash string">"endpoints"</code><code class="bash plain">]</code></div><div class="line number57 index56 alt2"><code class="bash spaces">    </code><code class="bash plain">verbs: [</code><code class="bash string">"get"</code><code class="bash plain">, </code><code class="bash string">"list"</code><code class="bash plain">, </code><code class="bash string">"watch"</code><code class="bash plain">, </code><code class="bash string">"create"</code><code class="bash plain">, </code><code class="bash string">"update"</code><code class="bash plain">, </code><code class="bash string">"patch"</code><code class="bash plain">]</code></div><div class="line number58 index57 alt1"><code class="bash plain">---</code></div><div class="line number59 index58 alt2"><code class="bash plain">kind: RoleBinding</code></div><div class="line number60 index59 alt1"><code class="bash plain">apiVersion: rbac.authorization.k8s.io</code><code class="bash plain">/v1</code></div><div class="line number61 index60 alt2"><code class="bash plain">metadata:</code></div><div class="line number62 index61 alt1"><code class="bash spaces">  </code><code class="bash plain">name: leader-locking-nfs-client-provisioner</code></div><div class="line number63 index62 alt2"><code class="bash spaces">  </code><code class="bash comments"># replace with namespace where provisioner is deployed</code></div><div class="line number64 index63 alt1"><code class="bash spaces">  </code><code class="bash plain">namespace: nfs</code></div><div class="line number65 index64 alt2"><code class="bash plain">subjects:</code></div><div class="line number66 index65 alt1"><code class="bash spaces">  </code><code class="bash plain">- kind: ServiceAccount</code></div><div class="line number67 index66 alt2"><code class="bash spaces">    </code><code class="bash plain">name: nfs-client-provisioner</code></div><div class="line number68 index67 alt1"><code class="bash spaces">    </code><code class="bash comments"># replace with namespace where provisioner is deployed</code></div><div class="line number69 index68 alt2"><code class="bash spaces">    </code><code class="bash plain">namespace: nfs</code></div><div class="line number70 index69 alt1"><code class="bash plain">roleRef:</code></div><div class="line number71 index70 alt2"><code class="bash spaces">  </code><code class="bash plain">kind: Role</code></div><div class="line number72 index71 alt1"><code class="bash spaces">  </code><code class="bash plain">name: leader-locking-nfs-client-provisioner</code></div><div class="line number73 index72 alt2"><code class="bash spaces">  </code><code class="bash plain">apiGroup: rbac.authorization.k8s.io</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　创建sc</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_94439" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: storage.k8s.io</code><code class="bash plain">/v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: StorageClass</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name: managed-nfs-storage</code></div><div class="line number5 index4 alt2"><code class="bash plain">provisioner: k8s-sigs.io</code><code class="bash plain">/nfs-subdir-external-provisioner</code> <code class="bash comments"># or choose another name, must match deployment's env PROVISIONER_NAME'</code></div><div class="line number6 index5 alt1"><code class="bash plain">reclaimPolicy: Retain </code><code class="bash comments">#PV的删除策略，默认为delete，删除PV后立即删除NFS server的数据</code></div><div class="line number7 index6 alt2"><code class="bash plain">mountOptions:</code></div><div class="line number8 index7 alt1"><code class="bash spaces">  </code><code class="bash comments">#- vers=4.1 #containerd有部分参数异常</code></div><div class="line number9 index8 alt2"><code class="bash spaces">  </code><code class="bash comments">#- noresvport #告知NFS客户端在重新建立网络连接时，使用新的传输控制协议源端口</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash plain">- noatime </code><code class="bash comments">#访问文件时不更新文件inode中的时间戳，高并发环境可提高性能</code></div><div class="line number11 index10 alt2"><code class="bash plain">parameters:</code></div><div class="line number12 index11 alt1"><code class="bash spaces">  </code><code class="bash comments">#mountOptions: "vers=4.1,noresvport,noatime"</code></div><div class="line number13 index12 alt2"><code class="bash spaces">  </code><code class="bash plain">archiveOnDelete: </code><code class="bash string">"true"</code>  <code class="bash comments">#删除pod时保留pod数据，默认为false时为不保留数据 </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　创建provision</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_390931" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div><div class="line number30 index29 alt1">30</div><div class="line number31 index30 alt2">31</div><div class="line number32 index31 alt1">32</div><div class="line number33 index32 alt2">33</div><div class="line number34 index33 alt1">34</div><div class="line number35 index34 alt2">35</div><div class="line number36 index35 alt1">36</div><div class="line number37 index36 alt2">37</div><div class="line number38 index37 alt1">38</div><div class="line number39 index38 alt2">39</div><div class="line number40 index39 alt1">40</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: apps</code><code class="bash plain">/v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: Deployment</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name: nfs-client-provisioner</code></div><div class="line number5 index4 alt2"><code class="bash spaces">  </code><code class="bash plain">labels:</code></div><div class="line number6 index5 alt1"><code class="bash spaces">    </code><code class="bash plain">app: nfs-client-provisioner</code></div><div class="line number7 index6 alt2"><code class="bash spaces">  </code><code class="bash comments"># replace with namespace where provisioner is deployed</code></div><div class="line number8 index7 alt1"><code class="bash spaces">  </code><code class="bash plain">namespace: nfs</code></div><div class="line number9 index8 alt2"><code class="bash plain">spec:</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash plain">replicas: 1</code></div><div class="line number11 index10 alt2"><code class="bash spaces">  </code><code class="bash plain">strategy: </code><code class="bash comments">#部署策略</code></div><div class="line number12 index11 alt1"><code class="bash spaces">    </code><code class="bash functions">type</code><code class="bash plain">: Recreate</code></div><div class="line number13 index12 alt2"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number14 index13 alt1"><code class="bash spaces">    </code><code class="bash plain">matchLabels:</code></div><div class="line number15 index14 alt2"><code class="bash spaces">      </code><code class="bash plain">app: nfs-client-provisioner</code></div><div class="line number16 index15 alt1"><code class="bash spaces">  </code><code class="bash plain">template:</code></div><div class="line number17 index16 alt2"><code class="bash spaces">    </code><code class="bash plain">metadata:</code></div><div class="line number18 index17 alt1"><code class="bash spaces">      </code><code class="bash plain">labels:</code></div><div class="line number19 index18 alt2"><code class="bash spaces">        </code><code class="bash plain">app: nfs-client-provisioner</code></div><div class="line number20 index19 alt1"><code class="bash spaces">    </code><code class="bash plain">spec:</code></div><div class="line number21 index20 alt2"><code class="bash spaces">      </code><code class="bash plain">serviceAccountName: nfs-client-provisioner</code></div><div class="line number22 index21 alt1"><code class="bash spaces">      </code><code class="bash plain">containers:</code></div><div class="line number23 index22 alt2"><code class="bash spaces">        </code><code class="bash plain">- name: nfs-client-provisioner</code></div><div class="line number24 index23 alt1"><code class="bash spaces">          </code><code class="bash comments">#image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2 </code></div><div class="line number25 index24 alt2"><code class="bash spaces">          </code><code class="bash plain">image: registry.cn-qingdao.aliyuncs.com</code><code class="bash plain">/zhangshijie/nfs-subdir-external-provisioner</code><code class="bash plain">:v4.0.2 </code></div><div class="line number26 index25 alt1"><code class="bash spaces">          </code><code class="bash plain">volumeMounts:</code></div><div class="line number27 index26 alt2"><code class="bash spaces">            </code><code class="bash plain">- name: nfs-client-root</code></div><div class="line number28 index27 alt1"><code class="bash spaces">              </code><code class="bash plain">mountPath: </code><code class="bash plain">/persistentvolumes</code></div><div class="line number29 index28 alt2"><code class="bash spaces">          </code><code class="bash functions">env</code><code class="bash plain">:</code></div><div class="line number30 index29 alt1"><code class="bash spaces">            </code><code class="bash plain">- name: PROVISIONER_NAME</code></div><div class="line number31 index30 alt2"><code class="bash spaces">              </code><code class="bash plain">value: k8s-sigs.io</code><code class="bash plain">/nfs-subdir-external-provisioner</code></div><div class="line number32 index31 alt1"><code class="bash spaces">            </code><code class="bash plain">- name: NFS_SERVER</code></div><div class="line number33 index32 alt2"><code class="bash spaces">              </code><code class="bash plain">value: 192.168.0.42</code></div><div class="line number34 index33 alt1"><code class="bash spaces">            </code><code class="bash plain">- name: NFS_PATH</code></div><div class="line number35 index34 alt2"><code class="bash spaces">              </code><code class="bash plain">value: </code><code class="bash plain">/data/volumes</code></div><div class="line number36 index35 alt1"><code class="bash spaces">      </code><code class="bash plain">volumes:</code></div><div class="line number37 index36 alt2"><code class="bash spaces">        </code><code class="bash plain">- name: nfs-client-root</code></div><div class="line number38 index37 alt1"><code class="bash spaces">          </code><code class="bash plain">nfs:</code></div><div class="line number39 index38 alt2"><code class="bash spaces">            </code><code class="bash plain">server: 192.168.0.42</code></div><div class="line number40 index39 alt1"><code class="bash spaces">            </code><code class="bash plain">path: </code><code class="bash plain">/data/volumes</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　调用sc创建pvc</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_373401" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">apiVersion: v1</code></div><div class="line number2 index1 alt1"><code class="bash plain">kind: Namespace</code></div><div class="line number3 index2 alt2"><code class="bash plain">metadata:</code></div><div class="line number4 index3 alt1"><code class="bash spaces">  </code><code class="bash plain">name: myserver</code></div><div class="line number5 index4 alt2"><code class="bash plain">---   </code></div><div class="line number6 index5 alt1"><code class="bash comments"># Test PVC</code></div><div class="line number7 index6 alt2"><code class="bash plain">kind: PersistentVolumeClaim</code></div><div class="line number8 index7 alt1"><code class="bash plain">apiVersion: v1</code></div><div class="line number9 index8 alt2"><code class="bash plain">metadata:</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash plain">name: myserver-myapp-dynamic-pvc</code></div><div class="line number11 index10 alt2"><code class="bash spaces">  </code><code class="bash plain">namespace: myserver</code></div><div class="line number12 index11 alt1"><code class="bash plain">spec:</code></div><div class="line number13 index12 alt2"><code class="bash spaces">  </code><code class="bash plain">storageClassName: managed-nfs-storage </code><code class="bash comments">#调用的storageclass 名称</code></div><div class="line number14 index13 alt1"><code class="bash spaces">  </code><code class="bash plain">accessModes:</code></div><div class="line number15 index14 alt2"><code class="bash spaces">    </code><code class="bash plain">- ReadWriteMany </code><code class="bash comments">#访问权限</code></div><div class="line number16 index15 alt1"><code class="bash spaces">  </code><code class="bash plain">resources:</code></div><div class="line number17 index16 alt2"><code class="bash spaces">    </code><code class="bash plain">requests:</code></div><div class="line number18 index17 alt1"><code class="bash spaces">      </code><code class="bash plain">storage: 500Mi </code><code class="bash comments">#空间大小</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　创建app使用pvc</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_706518" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div><div class="line number22 index21 alt1">22</div><div class="line number23 index22 alt2">23</div><div class="line number24 index23 alt1">24</div><div class="line number25 index24 alt2">25</div><div class="line number26 index25 alt1">26</div><div class="line number27 index26 alt2">27</div><div class="line number28 index27 alt1">28</div><div class="line number29 index28 alt2">29</div><div class="line number30 index29 alt1">30</div><div class="line number31 index30 alt2">31</div><div class="line number32 index31 alt1">32</div><div class="line number33 index32 alt2">33</div><div class="line number34 index33 alt1">34</div><div class="line number35 index34 alt2">35</div><div class="line number36 index35 alt1">36</div><div class="line number37 index36 alt2">37</div><div class="line number38 index37 alt1">38</div><div class="line number39 index38 alt2">39</div><div class="line number40 index39 alt1">40</div><div class="line number41 index40 alt2">41</div><div class="line number42 index41 alt1">42</div><div class="line number43 index42 alt2">43</div><div class="line number44 index43 alt1">44</div><div class="line number45 index44 alt2">45</div><div class="line number46 index45 alt1">46</div><div class="line number47 index46 alt2">47</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">kind: Deployment</code></div><div class="line number2 index1 alt1"><code class="bash comments">#apiVersion: extensions/v1beta1</code></div><div class="line number3 index2 alt2"><code class="bash plain">apiVersion: apps</code><code class="bash plain">/v1</code></div><div class="line number4 index3 alt1"><code class="bash plain">metadata:</code></div><div class="line number5 index4 alt2"><code class="bash spaces">  </code><code class="bash plain">labels:</code></div><div class="line number6 index5 alt1"><code class="bash spaces">    </code><code class="bash plain">app: myserver-myapp </code></div><div class="line number7 index6 alt2"><code class="bash spaces">  </code><code class="bash plain">name: myserver-myapp-deployment-name</code></div><div class="line number8 index7 alt1"><code class="bash spaces">  </code><code class="bash plain">namespace: myserver</code></div><div class="line number9 index8 alt2"><code class="bash plain">spec:</code></div><div class="line number10 index9 alt1"><code class="bash spaces">  </code><code class="bash plain">replicas: 1 </code></div><div class="line number11 index10 alt2"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number12 index11 alt1"><code class="bash spaces">    </code><code class="bash plain">matchLabels:</code></div><div class="line number13 index12 alt2"><code class="bash spaces">      </code><code class="bash plain">app: myserver-myapp-frontend</code></div><div class="line number14 index13 alt1"><code class="bash spaces">  </code><code class="bash plain">template:</code></div><div class="line number15 index14 alt2"><code class="bash spaces">    </code><code class="bash plain">metadata:</code></div><div class="line number16 index15 alt1"><code class="bash spaces">      </code><code class="bash plain">labels:</code></div><div class="line number17 index16 alt2"><code class="bash spaces">        </code><code class="bash plain">app: myserver-myapp-frontend</code></div><div class="line number18 index17 alt1"><code class="bash spaces">    </code><code class="bash plain">spec:</code></div><div class="line number19 index18 alt2"><code class="bash spaces">      </code><code class="bash plain">containers:</code></div><div class="line number20 index19 alt1"><code class="bash spaces">        </code><code class="bash plain">- name: myserver-myapp-container</code></div><div class="line number21 index20 alt2"><code class="bash spaces">          </code><code class="bash plain">image: nginx:1.20.0 </code></div><div class="line number22 index21 alt1"><code class="bash spaces">          </code><code class="bash comments">#imagePullPolicy: Always</code></div><div class="line number23 index22 alt2"><code class="bash spaces">          </code><code class="bash plain">volumeMounts:</code></div><div class="line number24 index23 alt1"><code class="bash spaces">          </code><code class="bash plain">- mountPath: </code><code class="bash string">"/usr/share/nginx/html/statics"</code></div><div class="line number25 index24 alt2"><code class="bash spaces">            </code><code class="bash plain">name: statics-datadir</code></div><div class="line number26 index25 alt1"><code class="bash spaces">      </code><code class="bash plain">volumes:</code></div><div class="line number27 index26 alt2"><code class="bash spaces">        </code><code class="bash plain">- name: statics-datadir</code></div><div class="line number28 index27 alt1"><code class="bash spaces">          </code><code class="bash plain">persistentVolumeClaim:</code></div><div class="line number29 index28 alt2"><code class="bash spaces">            </code><code class="bash plain">claimName: myserver-myapp-dynamic-pvc </code></div><div class="line number30 index29 alt1"> </div><div class="line number31 index30 alt2"><code class="bash plain">---</code></div><div class="line number32 index31 alt1"><code class="bash plain">kind: Service</code></div><div class="line number33 index32 alt2"><code class="bash plain">apiVersion: v1</code></div><div class="line number34 index33 alt1"><code class="bash plain">metadata:</code></div><div class="line number35 index34 alt2"><code class="bash spaces">  </code><code class="bash plain">labels:</code></div><div class="line number36 index35 alt1"><code class="bash spaces">    </code><code class="bash plain">app: myserver-myapp-service</code></div><div class="line number37 index36 alt2"><code class="bash spaces">  </code><code class="bash plain">name: myserver-myapp-service-name</code></div><div class="line number38 index37 alt1"><code class="bash spaces">  </code><code class="bash plain">namespace: myserver</code></div><div class="line number39 index38 alt2"><code class="bash plain">spec:</code></div><div class="line number40 index39 alt1"><code class="bash spaces">  </code><code class="bash functions">type</code><code class="bash plain">: NodePort</code></div><div class="line number41 index40 alt2"><code class="bash spaces">  </code><code class="bash plain">ports:</code></div><div class="line number42 index41 alt1"><code class="bash spaces">  </code><code class="bash plain">- name: http</code></div><div class="line number43 index42 alt2"><code class="bash spaces">    </code><code class="bash plain">port: 80</code></div><div class="line number44 index43 alt1"><code class="bash spaces">    </code><code class="bash plain">targetPort: 80</code></div><div class="line number45 index44 alt2"><code class="bash spaces">    </code><code class="bash plain">nodePort: 30015</code></div><div class="line number46 index45 alt1"><code class="bash spaces">  </code><code class="bash plain">selector:</code></div><div class="line number47 index46 alt2"><code class="bash spaces">    </code><code class="bash plain">app: myserver-myapp-frontend</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　应用上述配置清单</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_410969" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml/myapp</code><code class="bash comments"># kubectl apply -f .</code></div><div class="line number2 index1 alt1"><code class="bash plain">namespace</code><code class="bash plain">/nfs</code> <code class="bash plain">created</code></div><div class="line number3 index2 alt2"><code class="bash plain">serviceaccount</code><code class="bash plain">/nfs-client-provisioner</code> <code class="bash plain">created</code></div><div class="line number4 index3 alt1"><code class="bash plain">clusterrole.rbac.authorization.k8s.io</code><code class="bash plain">/nfs-client-provisioner-runner</code> <code class="bash plain">created</code></div><div class="line number5 index4 alt2"><code class="bash plain">clusterrolebinding.rbac.authorization.k8s.io</code><code class="bash plain">/run-nfs-client-provisioner</code> <code class="bash plain">created</code></div><div class="line number6 index5 alt1"><code class="bash plain">role.rbac.authorization.k8s.io</code><code class="bash plain">/leader-locking-nfs-client-provisioner</code> <code class="bash plain">created</code></div><div class="line number7 index6 alt2"><code class="bash plain">rolebinding.rbac.authorization.k8s.io</code><code class="bash plain">/leader-locking-nfs-client-provisioner</code> <code class="bash plain">created</code></div><div class="line number8 index7 alt1"><code class="bash plain">storageclass.storage.k8s.io</code><code class="bash plain">/managed-nfs-storage</code> <code class="bash plain">created</code></div><div class="line number9 index8 alt2"><code class="bash plain">deployment.apps</code><code class="bash plain">/nfs-client-provisioner</code> <code class="bash plain">created</code></div><div class="line number10 index9 alt1"><code class="bash plain">namespace</code><code class="bash plain">/myserver</code> <code class="bash plain">created</code></div><div class="line number11 index10 alt2"><code class="bash plain">persistentvolumeclaim</code><code class="bash plain">/myserver-myapp-dynamic-pvc</code> <code class="bash plain">created</code></div><div class="line number12 index11 alt1"><code class="bash plain">deployment.apps</code><code class="bash plain">/myserver-myapp-deployment-name</code> <code class="bash plain">created</code></div><div class="line number13 index12 alt2"><code class="bash plain">service</code><code class="bash plain">/myserver-myapp-service-name</code> <code class="bash plain">created</code></div><div class="line number14 index13 alt1"><code class="bash plain">root@k8s-deploy:</code></div></div></td></tr></tbody></table></div></div> </div> <p>　　验证：查看sc、pv、pvc是否创建？pod是否正常运行？</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_307725" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml/myapp</code><code class="bash comments"># kubectl get sc</code></div><div class="line number2 index1 alt1"><code class="bash plain">NAME                  PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE</code></div><div class="line number3 index2 alt2"><code class="bash plain">managed-nfs-storage   k8s-sigs.io</code><code class="bash plain">/nfs-subdir-external-provisioner</code>   <code class="bash plain">Retain          Immediate           </code><code class="bash functions">false</code>                  <code class="bash plain">105s</code></div><div class="line number4 index3 alt1"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml/myapp</code><code class="bash comments"># kubectl get pv</code></div><div class="line number5 index4 alt2"><code class="bash plain">NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                 STORAGECLASS          REASON   AGE</code></div><div class="line number6 index5 alt1"><code class="bash plain">pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c   500Mi      RWX            Retain           Bound    myserver</code><code class="bash plain">/myserver-myapp-dynamic-pvc</code>   <code class="bash plain">managed-nfs-storage            107s</code></div><div class="line number7 index6 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml/myapp</code><code class="bash comments"># kubectl get pvc -n myserver</code></div><div class="line number8 index7 alt1"><code class="bash plain">NAME                         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE</code></div><div class="line number9 index8 alt2"><code class="bash plain">myserver-myapp-dynamic-pvc   Bound    pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c   500Mi      RWX            managed-nfs-storage   117s</code></div><div class="line number10 index9 alt1"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml/myapp</code><code class="bash comments"># kubectl get pods -n myserver</code></div><div class="line number11 index10 alt2"><code class="bash plain">NAME                                              READY   STATUS    RESTARTS   AGE</code></div><div class="line number12 index11 alt1"><code class="bash plain">myserver-myapp-deployment-name-65ff65446f-xpd5p   1</code><code class="bash plain">/1</code>     <code class="bash plain">Running   0          2m8s</code></div><div class="line number13 index12 alt2"><code class="bash plain">root@k8s-deploy:</code><code class="bash plain">/yaml/myapp</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　提示：可以看到pv自动由sc创建，pvc自动和pv关联；</p> <p>　　验证：在nfs服务器上的/data/volumes/下创建index.html文件，访问pod service，看看对应文件是否能够正常被访问到？</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_446779" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@harbor:</code><code class="bash plain">/data/volumes</code><code class="bash comments"># ls</code></div><div class="line number2 index1 alt1"><code class="bash plain">myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c</code></div><div class="line number3 index2 alt2"><code class="bash plain">root@harbor:</code><code class="bash plain">/data/volumes</code><code class="bash comments"># cd myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c/</code></div><div class="line number4 index3 alt1"><code class="bash plain">root@harbor:</code><code class="bash plain">/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c</code><code class="bash comments"># ls</code></div><div class="line number5 index4 alt2"><code class="bash plain">root@harbor:</code><code class="bash plain">/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c</code><code class="bash comments"># echo "this page from nfs-server /data/volumes" >> index.html</code></div><div class="line number6 index5 alt1"><code class="bash plain">root@harbor:</code><code class="bash plain">/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c</code><code class="bash comments"># cat index.html </code></div><div class="line number7 index6 alt2"><code class="bash plain">this page from nfs-server </code><code class="bash plain">/data/volumes</code></div><div class="line number8 index7 alt1"><code class="bash plain">root@harbor:</code><code class="bash plain">/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　提示：在nfs服务器上的/data/volumes目录下会自动生成一个使用pvcpod所在名称空间+pvc名字+pv名字的一个目录，这个目录就是由provision创建；</p> <p>　　访问pod</p> <div class="cnblogs_Highlighter sh-gutter"> <div><div id="highlighter_12894" class="syntaxhighlighter bash"><div class="toolbar"><span><a href="#" class="toolbar_item command_help help">?</a></span></div><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># curl 192.168.0.36:30015/statics/index.html</code></div><div class="line number2 index1 alt1"><code class="bash plain">this page from nfs-server </code><code class="bash plain">/data/volumes</code></div><div class="line number3 index2 alt2"><code class="bash plain">root@harbor:~</code><code class="bash comments"># </code></div></div></td></tr></tbody></table></div></div> </div> <p>　　提示：能够访问到我们刚才创建的文件，说明pod正常挂载nfs服务器对应目录；</p> <p>　　PV/PVC总结</p> <p><img src="https://1000bd.com/contentImg/2023/06/12/165433296.png" alt="" loading="lazy" class="medium-zoom-image"></p> <p>　　PV是对底层网络存储的抽象，即将网络存储定义为一种存储资源，将一个整体的存储资源拆分成多份后给不同的业务使用。</p> <p>　　PVC是对PV资源的申请调用，pod是通过PVC将数据保存至PV，PV再把数据保存至真正的硬件存储。</p> <p>　　PersistentVolume参数</p> <p>　　Capacity： #当前PV空间大小，kubectl explain PersistentVolume.spec.capacity</p> <p>　　accessModes ：访问模式，#kubectl explain PersistentVolume.spec.accessModes</p> <p>　　　　ReadWriteOnce – PV只能被单个节点以读写权限挂载，RWO</p> <p>　　　　ReadOnlyMany – PV以可以被多个节点挂载但是权限是只读的,ROX<br>　　　　ReadWriteMany – PV可以被多个节点是读写方式挂载使用,RWX</p> <p>　　persistentVolumeReclaimPolicy #删除机制即删除存储卷卷时候，已经创建好的存储卷由以下删除操作：</p> <p>　　　　Retain – 删除PV后保持原装，最后需要管理员手动删除</p> <p>　　　　Recycle – 空间回收，及删除存储卷上的所有数据(包括目录和隐藏文件),目前仅支持NFS和hostPath</p> <p>　　　　Delete – 自动删除存储卷</p> <p>　　volumeMode #卷类型，kubectl explain PersistentVolume.spec.volumeMode；定义存储卷使用的文件系统是块设备还是文件系统，默认为文件系统</p> <p>　　mountOptions #附加的挂载选项列表，实现更精细的权限控制；</p> <p>　　官方文档：<a href="https://kubernetes.io/zh-cn/docs/concepts/storage/persistent-volumes/" rel="noopener">持久卷 | Kubernetes</a>；</p> <p>　　PersistentVolumeClaim创建参数</p> <p>　　accessModes ：PVC 访问模式，#kubectl explain PersistentVolumeClaim.spec.volumeMode</p> <p>　　　　ReadWriteOnce – PVC只能被单个节点以读写权限挂载，RWO</p> <p>　　　　ReadOnlyMany – PVC以可以被多个节点挂载但是权限是只读的,ROX</p> <p>　　　　ReadWriteMany – PVC可以被多个节点是读写方式挂载使用,RWX</p> <p>　　resources： #定义PVC创建存储卷的空间大小</p> <p>　　selector： #标签选择器，选择要绑定的PV</p> <p>　　　　matchLabels #匹配标签名称</p> <p>　　　　matchExpressions #基于正则表达式匹配</p> <p>　　volumeName #要绑定的PV名称</p> <p>　　volumeMode #卷类型，定义PVC使用的文件系统是块设备还是文件系统，默认为文件系统</p> <p>　　Volume- 存储卷类型</p> <p><img src="https://1000bd.com/contentImg/2023/06/12/165434199.png" alt="" loading="lazy" class="medium-zoom-image"></p> <p>　　static：静态存储卷 ,需要在使用前手动创建PV、然后创建PVC并绑定到PV然后挂载至pod使用，适用于PV和PVC相对比较固定的业务场景。</p> <p>　　dynamin：动态存储卷,先创建一个存储类storageclass，后期pod在使用PVC的时候可以通过存储类动态创建PVC，适用于有状态服务集群如MySQL一主多从、zookeeper集群等。</p> <p>　　存储类官方文档：<a href="https://kubernetes.io/zh-cn/docs/concepts/storage/storage-classes/" rel="noopener">存储类 | Kubernetes</a></p> </div> </div> </li> <li class="list-group-item ul-li"> <b>相关阅读:</b><br> <nobr> <a href="/Article/Index/1025904">目标检测——day49 基于级联注意力与点监督机制的考场目标检测模型</a> <br /> <a href="/Article/Index/1331520">ts的装饰器</a> <br /> <a href="/Article/Index/1399219">2017年高热度编程语言简介</a> <br /> <a href="/Article/Index/669711">嘉立创EDA-PCB产品设计操作概要</a> <br /> <a href="/Article/Index/1302436">【分布式云储存】高性能云存储MinIO简介与Docker部署集群</a> <br /> <a href="/Article/Index/1002218">SAP PA MM 后台配置</a> <br /> <a href="/Article/Index/1128243">lua基础之io</a> <br /> <a href="/Article/Index/1111518">【深入浅出Spring6】第四期——实例化Bean和Bean的生命周期</a> <br /> <a href="/Article/Index/1281781">转债打新监听</a> <br /> <a href="/Article/Index/1406755">软件测试面试丨关于JMeter的问题，看这篇就够了</a> <br /> </nobr> </li> <li class="list-group-item from-a mb-2"> 原文地址：https://www.cnblogs.com/qiuhom-1874/p/17368808.html </li> </ul> </div> <div class="col-lg-4 col-sm-12"> <ul class="list-group" style="word-break:break-all;"> <li class="list-group-item ul-li-bg" aria-current="true"> 最新文章 </li> <li class="list-group-item ul-li"> <nobr> <a href="/Article/Index/1484446">攻防演习之三天拿下官网站群</a> <br /> <a href="/Article/Index/1515268">数据安全治理学习——前期安全规划和安全管理体系建设</a> <br /> <a href="/Article/Index/1759065">企业安全 | 企业内一次钓鱼演练准备过程</a> <br /> <a href="/Article/Index/1485036">内网渗透测试 | Kerberos协议及其部分攻击手法</a> <br /> <a href="/Article/Index/1877332">0day的产生 | 不懂代码的"代码审计"</a> <br /> <a href="/Article/Index/1887576">安装scrcpy-client模块av模块异常，环境问题解决方案</a> <br /> <a href="/Article/Index/1887578">leetcode hot100【LeetCode 279. 完全平方数】java实现</a> <br /> <a href="/Article/Index/1887512">OpenWrt下安装Mosquitto</a> <br /> <a href="/Article/Index/1887520">AnatoMask论文汇总</a> <br /> <a href="/Article/Index/1887496">【AI日记】24.11.01 LangChain、openai api和github copilot</a> <br /> </nobr> </li> </ul> <ul class="list-group pt-2" style="word-break:break-all;"> <li class="list-group-item ul-li-bg" aria-current="true"> 热门文章 </li> <li class="list-group-item ul-li"> <nobr> <a href="/Article/Index/888177">十款代码表白小特效 一个比一个浪漫 赶紧收藏起来吧！！！</a> <br /> <a href="/Article/Index/797680">奉劝各位学弟学妹们，该打造你的技术影响力了！</a> <br /> <a href="/Article/Index/888183">五年了，我在 CSDN 的两个一百万。</a> <br /> <a href="/Article/Index/888179">Java俄罗斯方块，老程序员花了一个周末，连接中学年代！</a> <br /> <a href="/Article/Index/797730">面试官都震惊，你这网络基础可以啊！</a> <br /> <a href="/Article/Index/797725">你真的会用百度吗？我不信 — 那些不为人知的搜索引擎语法</a> <br /> <a href="/Article/Index/797702">心情不好的时候，用 Python 画棵樱花树送给自己吧</a> <br /> <a href="/Article/Index/797709">通宵一晚做出来的一款类似CS的第一人称射击游戏Demo！原来做游戏也不是很难，连憨憨学妹都学会了！</a> <br /> <a href="/Article/Index/797716">13 万字 C 语言从入门到精通保姆级教程2021 年版</a> <br /> <a href="/Article/Index/888192">10行代码集2000张美女图，Python爬虫120例，再上征途</a> <br /> </nobr> </li> </ul> </div> </div> </div> <!-- 主体 --> <!--body结束--> <!--这里是footer模板--> <!--footer--> <nav class="navbar navbar-inverse navbar-fixed-bottom"> <div class="container"> <div class="row"> <div class="col-md-12"> <div class="text-muted center foot-height"> Copyright © 2022 侵权请联系<a href="mailto:2656653265@qq.com">2656653265@qq.com</a>    <a href="https://beian.miit.gov.cn/" target="_blank">京ICP备2022015340号-1</a> </div> <div style="width:300px;margin:0 auto; padding:0px 5px;"> <a href="/regex.html">正则表达式工具</a> <a href="/cron.html">cron表达式工具</a> <a href="/pwdcreator.html">密码生成工具</a> </div> <div style="width:300px;margin:0 auto; padding:5px 0;"> <a target="_blank" href="http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=11010502049817" style="display:inline-block;text-decoration:none;height:20px;line-height:20px;"> <img src="" style="float:left;" /><p style="float:left;height:20px;line-height:20px;margin: 0px 0px 0px 5px; color:#939393;">京公网安备 11010502049817号</p></a> </div> </div> </div> </div> </nav> <!--footer--> <!--footer模板结束--> <script src="/js/plugins/jquery/jquery.js"></script> <script src="/js/bootstrap.min.js"></script> <!--这里是scripts模板--> <!--scripts模板结束--> </body> </html>