kubernetes使用NFS存储卷---csi-driver-nfs

参考资料：
https://www.cnblogs.com/-k8s/p/18004618
https://ubuntu.com/server/docs/service-nfs

简介

csi-driver-nfs 是一个用于 Kubernetes 的 NFS CSI 驱动程序，它可以让 Kubernetes 访问 Linux 节点上的 NFS 服务器。它的 CSI 插件名称是 nfs.csi.k8s.io。这个驱动程序需要已经存在并配置好的 NFSv3 或 NFSv4 服务器，它支持通过创建 NFS 服务器下的新子目录来动态分配持久卷（Persistent Volumes）

这个驱动程序的主要功能和特点有：
支持 NFSv3 和 NFSv4 协议
支持多种参数设置，包括 mountOptions，server，share，subPath，readOnly 等
支持 Kubernetes 1.21+ 版本

安装NFS Server

sudo apt install nfs-kernel-server

配置nfs

mkdir /nfs-csi
echo "/nfs-csi *(rw,async,no_subtree_check,no_root_squash)" >> /etc/exports

启动NFS Server

systemctl start nfs-kernel-server.service

查看NFS版本

root@k8s:~# cat /proc/fs/nfsd/versions 
-2 +3 +4 +4.1 +4.2

查看NFS Server共享目录

root@k8s:~# showmount -e 10.2.20.7
Export list for 10.2.20.7:
/nfs-csi *

安装NFS Driver

https://github.com/kubernetes-csi/csi-driver-nfs

安装脚本
https://github.com/kubernetes-csi/csi-driver-nfs/blob/v4.6.0/deploy/install-driver.sh

部署yaml文件
https://github.com/kubernetes-csi/csi-driver-nfs/tree/v4.6.0/deploy/v4.6.0

镜像

docker pull registry.cn-hangzhou.aliyuncs.com/image-storage/csi-provisioner:v3.6.2
docker pull registry.cn-hangzhou.aliyuncs.com/image-storage/csi-snapshotter:v6.3.2
docker pull registry.cn-hangzhou.aliyuncs.com/image-storage/livenessprobe:v2.11.0        
docker pull registry.cn-hangzhou.aliyuncs.com/image-storage/nfspluginnfsplugin:v4.6.0
docker pull registry.cn-hangzhou.aliyuncs.com/image-storage/csi-node-driver-registrar:v2.9.1
docker pull registry.cn-hangzhou.aliyuncs.com/image-storage/snapshot-controller:v6.3.2

部署

kubectl apply -f rbac-csi-nfs.yaml
kubectl apply -f csi-nfs-driverinfo.yaml
kubectl apply -f csi-nfs-controller.yaml
kubectl apply -f csi-nfs-node.yaml

检查Pod状态

root@master1:~# kubectl -n kube-system get pod -o wide -l app=csi-nfs-node
NAME                 READY   STATUS    RESTARTS   AGE   IP          NODE      NOMINATED NODE   READINESS GATES
csi-nfs-node-9wn4h   3/3     Running   0          3h    10.2.20.7   master1   <none>           <none>

root@master1:~# kubectl -n kube-system get pod -o wide -l app=csi-nfs-controller
NAME                                  READY   STATUS    RESTARTS   AGE    IP          NODE      NOMINATED NODE   READINESS GATES
csi-nfs-controller-787dddc7fb-8h5kw   4/4     Running   0          3h9m   10.2.20.7   master1   <none>           <none>

创建storageclass

cat > storageclass-nfs.yaml <apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-csi
  annotations:
    # 此操作是1.25的以上的一个alpha的新功能，是将此storageclass设置为默认
    storageclass.kubernetes.io/is-default-class: "true"
# 此处指定了csidrivers的名称
provisioner: nfs.csi.k8s.io
parameters:
  # NFS的Server
  server: 10.2.20.7
  # NFS的存储路径
  share: /nfs-csi
  # csi.storage.k8s.io/provisioner-secret is only needed for providing mountOptions in DeleteVolume
  # csi.storage.k8s.io/provisioner-secret-name: "mount-options"
  # csi.storage.k8s.io/provisioner-secret-namespace: "default"
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  # 这里不只可以配置nfs的版本
  - nfsvers=4.1
EOF

kubectl apply -f storageclass-nfs.yaml

查看sc

kubectl get storageclasses.storage.k8s.io 
NAME                PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-csi (default)   nfs.csi.k8s.io   Delete          Immediate           false                  41s

创建资源测试

cat <| sudo tee nfs-pod.yaml 
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-nginx
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: nfs-csi
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-nfs
spec:
  containers:
    - image: nginx
      name: nginx
      ports:
        - containerPort: 80
          protocol: TCP
      volumeMounts:
        - mountPath: /var/www
          name: pvc-nginx
  volumes:
    - name: pvc-nginx
      persistentVolumeClaim:
        claimName: pvc-nginx
EOF

kubectl apply -f nfs-pod.yaml

查看资源

root@master1:~# kubectl get pvc
NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
pvc-nginx   Bound    pvc-e1a0183d-24d3-4329-a8a9-59c626c0e0e6   10Gi       RWX            nfs-csi        <unset>                 3h2m
root@master1:~# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM               STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
pvc-e1a0183d-24d3-4329-a8a9-59c626c0e0e6   10Gi       RWX            Delete           Bound    default/pvc-nginx   nfs-csi        <unset>                          3h2m
root@master1:~# ll /nfs-csi/
total 12
drwxr-xr-x  3 root root 4096 Jun  6 15:03 ./
drwxr-xr-x 21 root root 4096 Jun  6 15:06 ../
drwxr-xr-x  2 root root 4096 Jun  6 12:08 pvc-e1a0183d-24d3-4329-a8a9-59c626c0e0e6/