云原生|kubernetes|kubernetes集群使用私有镜像仓库拉取镜像（harbor或者官方的registry私有镜像仓库）

前言：

在实际的生产中，我们可能会有许多的由开发制作的docker镜像，这也就造成使用这些镜像需要打包成tar文件，然后上传到服务器内然后在导入并使用，但，kubernetes节点很多，有时候并不是明确的要在哪个节点使用，因此需要每个节点都上传或者是在部署的时候做节点亲和。那么，很明显这样的方法是不够优雅的。

那么，我们将需要的镜像统一上传到一个自己搭建的私有镜像仓库是一个比较好的解决方案，只需要在部署阶段指定使用私有镜像仓库就可以解决了，并且harbor这样的私有镜像仓库还有安全漏洞扫描功能，能够提升整个系统的安全性。

下面，将就如何在kubernetes集群内使用私有镜像仓库做一个简单的示例。

一，

在kubernetes内使用私有镜像仓库之前，我们需要先有一个私有镜像仓库，并保证这个仓库是可用的。

本文的私有镜像仓库是harbor仓库，该仓库搭建在服务器IP为192.168.217.23。

具体的搭建过程请移步上一篇文章；harbor仓库的构建及简单使用（修订版）_晚风_END的博客-CSDN博客

二，

检查私有镜像仓库是否可用


[root@node3 manifests]# systemctl status harbor
● harbor.service - Harbor
   Loaded: loaded (/usr/lib/systemd/system/harbor.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2022-12-01 11:43:31 CST; 1h 22min ago
     Docs: http://github.com/vmware/harbor
 Main PID: 2690 (docker-compose)
   Memory: 41.6M
   CGroup: /system.slice/harbor.service
           ├─2690 /usr/bin/docker-compose -f /usr/local/harbor/docker-compose.yml up
           └─2876 /usr/bin/docker-compose -f /usr/local/harbor/docker-compose.yml up
 
Dec 01 11:43:33 node3 docker-compose[2690]: harbor-jobservice is up-to-date
Dec 01 11:43:33 node3 docker-compose[2690]: nginx is up-to-date
Dec 01 11:43:33 node3 docker-compose[2690]: Attaching to harbor-log, redis, harbor-adminserver, registry, harbor-db, harbor-ui, harbor-jobservice, nginx
Dec 01 11:43:33 node3 docker-compose[2690]: harbor-adminserver | WARNING: no logs are available with the 'syslog' log driver
Dec 01 11:43:33 node3 docker-compose[2690]: harbor-db      | WARNING: no logs are available with the 'syslog' log driver
Dec 01 11:43:33 node3 docker-compose[2690]: harbor-jobservice | WARNING: no logs are available with the 'syslog' log driver
Dec 01 11:43:33 node3 docker-compose[2690]: harbor-ui      | WARNING: no logs are available with the 'syslog' log driver
Dec 01 11:43:33 node3 docker-compose[2690]: nginx          | WARNING: no logs are available with the 'syslog' log driver
Dec 01 11:43:33 node3 docker-compose[2690]: redis          | WARNING: no logs are available with the 'syslog' log driver
Dec 01 11:43:33 node3 docker-compose[2690]: registry       | WARNING: no logs are available with the 'syslog' log driver

健康检查，如下。都是healthy即可，如果是unhealthy，表示此镜像仓库不可用，一般这样的情况重启服务器即可恢复。


[root@node3 harbor]# docker-compose ps
       Name                     Command                  State                                    Ports                              
-------------------------------------------------------------------------------------------------------------------------------------
harbor-adminserver   /harbor/start.sh                 Up (healthy)                                                                   
harbor-db            /usr/local/bin/docker-entr ...   Up (healthy)   3306/tcp                                                        
harbor-jobservice    /harbor/start.sh                 Up                                                                             
harbor-log           /bin/sh -c /usr/local/bin/ ...   Up (healthy)   127.0.0.1:1514->10514/tcp                                       
harbor-ui            /harbor/start.sh                 Up (healthy)                                                                   
nginx                nginx -g daemon off;             Up (healthy)   0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp, 0.0.0.0:80->80/tcp
redis                docker-entrypoint.sh redis ...   Up             6379/tcp                                                        
registry             /entrypoint.sh serve /etc/ ...   Up (healthy)   5000/tcp

登录私有镜像仓库，生成登录记录文件（注意，这个文件的生成不管是https的私有仓库还是http的仓库，只需要登录成功即可，有几个私有仓库，它都会记录进来的）：

登录记录文件是kubernetes使用私有镜像仓库的关键文件，一会会使用此文件生成一个secret，在使用私有仓库内的镜像时，将该secret挂载到部署文件内。


[root@node3 harbor]# docker login https://192.168.217.23
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
 
Login Succeeded


[root@node3 harbor]# cat /root/.docker/config.json
{
	"auths": {
		"192.168.217.23": {
			"auth": "YWRtaW46U2hpZ3VhbmdfMzI="
		}
	},
	"HttpHeaders": {
		"User-Agent": "Docker-Client/19.03.9 (linux)"
	}

三，

将密钥进行base64加密

cat /root/.docker/config.json | base64 -w 0

输出如下；

ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjIxNy4yMyI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZVMmhwWjNWaGJtZGZNekk9IgoJCX0KCX0sCgkiSHR0cEhlYWRlcnMiOiB7CgkJIlVzZXItQWdlbnQiOiAiRG9ja2VyLUNsaWVudC8xOS4wMy45IChsaW51eCkiCgl9Cn0=

四，

新建secret部署文件，保存上述生成的密钥


cat >harbor_secret.yaml <<EOF 
apiVersion: v1
kind: Secret
metadata:
  name: harbor-login
type: kubernetes.io/dockerconfigjson
data:
   # 这里添加上述base64加密后的密钥
   .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjIxNy4yMyI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZVMmhwWjNWaGJtZGZNekk9IgoJCX0KCX0sCgkiSHR0cEhlYWRlcnMiOiB7CgkJIlVzZXItQWdlbnQiOiAiRG9ja2VyLUNsaWVudC8xOS4wMy45IChsaW51eCkiCgl9Cn0=
EOF

生成这个secret：

 k apply -f harbor_secret.yaml

查看这个secret：


[root@node3 harbor]# k describe secrets harbor-login 
Name:         harbor-login
Namespace:    default
Labels:       
Annotations:  
Type:         kubernetes.io/dockerconfigjson
 
Data
====
.dockerconfigjson:  152 bytes

五，

kubernetes的部署文件调用此secret

假设已有镜像上传到了私有镜像仓库内，该镜像是nginx，版本是1.20

现在部署nginx的时候指定使用私有镜像仓库里的镜像，那么，部署文件应该是这样的：


cat >nginx.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: 192.168.217.23/library/nginx:1.20
        name: nginx
        resources: {}
      imagePullSecrets:
      - name: harbor-login
status: {}
EOF

主要是在部署的时候增加了这么两行：


      imagePullSecrets:
      - name: harbor-login

在部署一个MySQL，同样也是使用私有镜像仓库：

同样的，增加这么两行：


  imagePullSecrets:
  - name: harbor-login


cat /etc/kubernetes/manifests/mysql.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: mysql
  labels:
    run: mysql
  name: mysql
  namespace: default
spec:
  containers:
  - env:
    - name: MYSQL_ROOT_PASSWORD
      value: shiguang32
    image: 192.168.217.23/test/mysql:5.7.39
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        cpu: 200m
    name: mysql
    volumeMounts:
    - mountPath: /var/lib/mysql
      name: mysql-data
      readOnly: false
    - mountPath: /etc/mysql/mysql.conf.d
      name: mysql-conf
      readOnly: false
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  volumes:
  - name: mysql-data
    hostPath:
      path: /opt/mysql/data
      type: DirectoryOrCreate
  - name: mysql-conf
    hostPath:
      path: /opt/mysql/conf
      type: DirectoryOrCreate
  imagePullSecrets:
  - name: harbor-login
 
  hostNetwork: true
  priorityClassName: system-cluster-critical
status: {}


[root@node3 harbor]# k get po
NAME                     READY   STATUS    RESTARTS   AGE
mysql                    1/1     Running   7          76m
nginx-58bf645545-xtnsn   1/1     Running   1          161m

查看pod详情，看看是不是使用了私有仓库的镜像：

可以看到，确实是正确的下载了私有镜像仓库里的镜像


[root@node3 harbor]# k describe pod nginx-58bf645545-xtnsn 
Name:         nginx-58bf645545-xtnsn
Namespace:    default
Priority:     0
Node:         node3/192.168.217.23
Start Time:   Thu, 01 Dec 2022 10:59:50 +0800
Labels:       app=nginx
              pod-template-hash=58bf645545
Annotations:  
Status:       Running
IP:           10.244.0.243
IPs:
  IP:           10.244.0.243
Controlled By:  ReplicaSet/nginx-58bf645545
Containers:
  nginx:
    Container ID:   docker://07fc2a45709ff4698de6e4c168a175d1c10b9f23c1240c29fc1cb463142193c7
    Image:          192.168.217.23/library/nginx:1.20
    Image ID:       docker-pullable://192.168.217.23/library/nginx@sha256:cba27ee29d62dfd6034994162e71c399b08a84b50ab25783eabce64b1907f774


[root@node3 harbor]# k describe pod mysql 
Name:                 mysql
Namespace:            default
Priority:             2000000000
Priority Class Name:  system-cluster-critical
Node:                 node3/192.168.217.23
Start Time:           Thu, 01 Dec 2022 12:24:50 +0800
Labels:               run=mysql
Annotations:          Status:  Running
IP:                   192.168.217.23
IPs:
  IP:  192.168.217.23
Containers:
  mysql:
    Container ID:   docker://f6b6e9324bc17a2b3425edee382c01f0a3095379fbe4af4209bf4c7dc05bd55d
    Image:          192.168.217.23/test/mysql:5.7.39
    Image ID:       docker-pullable://192.168.217.23/test/mysql@sha256:b39b95329c868c3875ea6eb23c9a2a27168c3531f83c96c24324213f75793636

至此，kubernetes使用私有镜像仓库圆满成功！！！

相关阅读:
锂电池欧姆内阻和极化内阻
 使用Psycopg2连接openGauss 3.0（python2）
未来的趋势是什么？为什么说先进计算是未来的趋势？
一场由Integer引发的血案
 [Unity2D独立/合作开发]实现记录物品在不同场景的存在状态，附：场景的淡入淡出功能和预加载
 Happy 1024
STC89C52定时器/中断快速上手示例讲解
 修复mongodb未授权访问漏洞（漏洞编号：009711D8）
问题：arcpy中计算RasterCalculator输出的栅格一直为double precision
Kubernetes（k8s）的核心设计介绍
原文地址：https://blog.csdn.net/alwaysbefine/article/details/128130267