《OpenShift / RHEL / DevSecOps 汇总目录》
文本已在 OpenShift 4.15 环境中进行验证。
一个 OpenShift 集群节点在运行 Pod 前需要先从 Registry 拉取到相关 Image。这些镜像会保存在节点本地存储中并作为缓存,这样该节点如果再使用这个 Image 就会先使用节点缓存中的镜像,从而提升 Pod 的启动速度。
$ oc get node
NAME STATUS ROLES AGE VERSION
control-plane-cluster-cw8ww-1 Ready control-plane,master,worker 30h v1.28.7+f1b5f6c
$ NODE_NAME=control-plane-cluster-cw8ww-1
$ oc apply -f - << EOF
kind: Deployment
apiVersion: apps/v1
metadata:
name: parksmap
spec:
replicas: 1
selector:
matchLabels:
app: parksmap
template:
metadata:
labels:
app: parksmap
spec:
nodeName: ${NODE_NAME}
containers:
- name: hello-openshift
image: 'quay.io/openshiftroadshow/parksmap:latest'
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 8888
protocol: TCP
imagePullPolicy: IfNotPresent
EOF
$ oc get pod -owide
NAME READY STATUS RESTARTS AGE IP NODE
parksmap-9bdf99cc-km892 1/1 Running 0 32m 10.132.0.114 control-plane-cluster-cw8ww-1
$ oc debug node/$NODE_NAME
sh-4.4# chroot /host
$ cat /var/lib/containers/storage/overlay-images/images.json | jq
......
{
"id": "0c2f55f381ee738ec77722ea7b92ac97fcfa1cb6ef8e323df929b0ab40c70a47",
"digest": "sha256:89d1e324846cb431df9039e1a7fd0ed2ba0c51aafbae73f2abd70a83d5fa173b",
"names": [
"quay.io/openshiftroadshow/parksmap:latest"
],
"names-history": [
"quay.io/openshiftroadshow/parksmap:latest"
],
"layer": "e0c471633f3e337a376fd877385b6bfc287b9144c003aa61c933ec853610e3ce",
"metadata": "{}",
"big-data-names": [
"sha256:0c2f55f381ee738ec77722ea7b92ac97fcfa1cb6ef8e323df929b0ab40c70a47",
"manifest-sha256:89d1e324846cb431df9039e1a7fd0ed2ba0c51aafbae73f2abd70a83d5fa173b",
"manifest"
],
"big-data-sizes": {
"manifest": 1373,
"manifest-sha256:89d1e324846cb431df9039e1a7fd0ed2ba0c51aafbae73f2abd70a83d5fa173b": 1373,
"sha256:0c2f55f381ee738ec77722ea7b92ac97fcfa1cb6ef8e323df929b0ab40c70a47": 5671
},
"big-data-digests": {
"manifest": "sha256:89d1e324846cb431df9039e1a7fd0ed2ba0c51aafbae73f2abd70a83d5fa173b",
"manifest-sha256:89d1e324846cb431df9039e1a7fd0ed2ba0c51aafbae73f2abd70a83d5fa173b": "sha256:89d1e324846cb431df9039e1a7fd0ed2ba0c51aafbae73f2abd70a83d5fa173b",
"sha256:0c2f55f381ee738ec77722ea7b92ac97fcfa1cb6ef8e323df929b0ab40c70a47": "sha256:0c2f55f381ee738ec77722ea7b92ac97fcfa1cb6ef8e323df929b0ab40c70a47"
},
"created": "2021-02-01T12:52:36.138979177Z"
}
$ ls -al /var/lib/containers/storage/overlay-images | grep 0c2f55f381ee738ec77722ea7b92ac97fcfa1cb6ef8e323df929b0ab40c70a47
drwx------. 2 root root 4096 Apr 10 01:15 0c2f55f381ee738ec77722ea7b92ac97fcfa1cb6ef8e323df929b0ab40c70a47
$ oc apply -f - << EOF
kind: Deployment
apiVersion: apps/v1
metadata:
name: hello-openshift-v1
spec:
replicas: 1
selector:
matchLabels:
app: hello-openshift-v1
template:
metadata:
labels:
app: hello-openshift-v1
spec:
nodeName: ${NODE_NAME}
containers:
- name: hello-openshift
image: 'quay.io/dawnskyliu/hello-openshift:v1'
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 8888
protocol: TCP
imagePullPolicy: IfNotPresent
EOF
$ oc get pod
NAME READY STATUS RESTARTS AGE
hello-openshift-v1-6995c77fb5-hvd84 1/1 Running 0 13s
$ oc delete deploy hello-openshift-v1
$ oc apply -f - << EOF
kind: Deployment
apiVersion: apps/v1
metadata:
name: hello-openshift-v1
spec:
replicas: 1
selector:
matchLabels:
app: hello-openshift-v1
template:
metadata:
labels:
app: hello-openshift-v1
spec:
nodeName: ${NODE_NAME}
containers:
- name: hello-openshift
image: 'quay.io/dawnskyliu/hello-openshift:v1'
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 8888
protocol: TCP
imagePullPolicy: IfNotPresent
EOF
$ oc delete deploy hello-openshift-v1
$ oc apply -f - << EOF
kind: Deployment
apiVersion: apps/v1
metadata:
name: hello-openshift-v1
spec:
replicas: 1
selector:
matchLabels:
app: hello-openshift-v1
template:
metadata:
labels:
app: hello-openshift-v1
spec:
nodeName: ${NODE_NAME}
containers:
- name: hello-openshift
image: 'quay.io/dawnskyliu/hello-openshift:v1'
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 8888
protocol: TCP
imagePullPolicy: Always
EOF
随着保存在节点本地存储的镜像缓存的数量增加,会不断消耗节点的本地存储空间,因此需要定期清理镜像缓存以释放存储空间。运行在节点的 kubelet 提供了清理本地镜像缓存的机制,而 OpenShift 4 是无法手动删除节点镜像缓存的。
即便不考虑节点本地的镜像缓存,基于 oc new-app 生成的部署和基于 YAML 的部署之间是存在一定差异。
在使用 oc new-app 命令生成部署的时候,虽然在命令中提供的是远程镜像地址,但 OpenShift 首先会把镜像从远程拉到 OpenShift 内部的镜像库中,然后再在 Deployment 的定义中自动通过 ImageStream 使用已在 OpenShift 内部镜像库中的镜像。
当使用基于 YAML 的部署时候,其容器使用的镜像地址不会发生变化,会直接使用远程镜像库中的镜像。
https://www.redhat.com/en/blog/image-garbage-collection-in-openshift
https://docs.openshift.com/container-platform/4.15/nodes/nodes/nodes-nodes-garbage-collection.html
https://medium.com/kubernetes-tutorials/efficient-node-out-of-resource-management-in-kubernetes-67f158da6e59
https://itnext.io/improve-container-image-availability-and-speed-with-caching-in-kubernetes-870fa7bfa1ed