• 使用 CRD 开启您的 Ingress 可观测之路

    简介

    CRD 全称是 Custom Resource Definition，是 Kubernetes 的一种内置资源类型，允许用户自定义新的资源类型。观测云实现了 CRD 资源的自定义，然后通过 CRD 控制器实现对自定义的 CRD 对象的管理。使用 CRD 采集指标，可以实现与应用的解耦。以 Ingress 指标采集来说，采集指标需要在部署 Ingress 的 yaml 文件中的 Deployment 资源上增加 annotations，这样 DataKit 就可以通过自定义的 annotations 来采集 Ingress Pod 的指标。有个不好的地方就是 annotations 与 Pod 耦合性太强，使用 CRD 就方便多了，只需要知道 Ingress 的 namespace 和 Deployment 的 name 就可以了。接下来就使用 CRD 一步一步实现 Ingress 的可观测。

    前置条件

    • Kubernetes 集群
    • 观测云账号

    环境版本

    本次示例使用版本如下：

    • DataKit 1.4.11
    • Nginx Ingress Controller 1.1.1

    操作步骤

    步骤 1： 部署 DataKit

    1.1 获取 Token

    登录『观测云』，点击『管理』模块，在基本设置界面找到 Token，点击后面的复制图标。

    1.2 下载 DataKit 部署文件

    登录『观测云』，点击『集成』模块，再点击左上角『DataKit』，选择『Kubernetes』，下载 datakit.yaml 文件。

    1.3 部署 DataKit

    打开 datakit.yaml 文件，把复制的 Token 替换文件中的 ，为了区分集群和选举，增加几个环境变量，k8s-containerd 可以自己定义。

    1.         - name: ENV_GLOBAL_HOST_TAGS
    2.           value: host=__datakit_hostname,host_ip=__datakit_ip,cluster_name_k8s=k8s-containerd
    3.         - name: ENV_NAMESPACE   
    4.           value: k8s-containerd
    5.         - name: ENV_GLOBAL_ELECTION_TAGS 
    6.           value: cluster_name_k8s=k8s-containerd

    把 datakit.yaml 上传到 Kubernetes 集群的 master 节点，执行命令部署 DataKit。

    kubectl apply -f datakit.yaml

    步骤 2： 部署 Ingress

    2.1 编写 ingress-deployment.yaml

    1. apiVersion: v1
    2. kind: Namespace
    3. metadata:
    4.   name: ingress-nginx
    5.   labels:
    6.     app.kubernetes.io/name: ingress-nginx
    7.     app.kubernetes.io/instance: ingress-nginx
    8.  
    9. ---
    10. # Source: ingress-nginx/templates/controller-serviceaccount.yaml
    11. apiVersion: v1
    12. kind: ServiceAccount
    13. metadata:
    14.   labels:
    15.     helm.sh/chart: ingress-nginx-4.0.15
    16.     app.kubernetes.io/name: ingress-nginx
    17.     app.kubernetes.io/instance: ingress-nginx
    18.     app.kubernetes.io/version: 1.1.1
    19.     app.kubernetes.io/managed-by: Helm
    20.     app.kubernetes.io/component: controller
    21.   name: ingress-nginx
    22.   namespace: ingress-nginx
    23. automountServiceAccountToken: true
    24. ---
    25. # Source: ingress-nginx/templates/controller-configmap.yaml
    26. apiVersion: v1
    27. kind: ConfigMap
    28. metadata:
    29.   labels:
    30.     helm.sh/chart: ingress-nginx-4.0.15
    31.     app.kubernetes.io/name: ingress-nginx
    32.     app.kubernetes.io/instance: ingress-nginx
    33.     app.kubernetes.io/version: 1.1.1
    34.     app.kubernetes.io/managed-by: Helm
    35.     app.kubernetes.io/component: controller
    36.   name: ingress-nginx-controller
    37.   namespace: ingress-nginx
    38. data:
    39.   allow-snippet-annotations: 'true'
    40. ---
    41. # Source: ingress-nginx/templates/clusterrole.yaml
    42. apiVersion: rbac.authorization.k8s.io/v1
    43. kind: ClusterRole
    44. metadata:
    45.   labels:
    46.     helm.sh/chart: ingress-nginx-4.0.15
    47.     app.kubernetes.io/name: ingress-nginx
    48.     app.kubernetes.io/instance: ingress-nginx
    49.     app.kubernetes.io/version: 1.1.1
    50.     app.kubernetes.io/managed-by: Helm
    51.   name: ingress-nginx
    52. rules:
    53.   - apiGroups:
    54.       - ''
    55.     resources:
    56.       - configmaps
    57.       - endpoints
    58.       - nodes
    59.       - pods
    60.       - secrets
    61.       - namespaces
    62.     verbs:
    63.       - list
    64.       - watch
    65.   - apiGroups:
    66.       - ''
    67.     resources:
    68.       - nodes
    69.     verbs:
    70.       - get
    71.   - apiGroups:
    72.       - ''
    73.     resources:
    74.       - services
    75.     verbs:
    76.       - get
    77.       - list
    78.       - watch
    79.   - apiGroups:
    80.       - networking.k8s.io
    81.     resources:
    82.       - ingresses
    83.     verbs:
    84.       - get
    85.       - list
    86.       - watch
    87.   - apiGroups:
    88.       - ''
    89.     resources:
    90.       - events
    91.     verbs:
    92.       - create
    93.       - patch
    94.   - apiGroups:
    95.       - networking.k8s.io
    96.     resources:
    97.       - ingresses/status
    98.     verbs:
    99.       - update
    100.   - apiGroups:
    101.       - networking.k8s.io
    102.     resources:
    103.       - ingressclasses
    104.     verbs:
    105.       - get
    106.       - list
    107.       - watch
    108. ---
    109. # Source: ingress-nginx/templates/clusterrolebinding.yaml
    110. apiVersion: rbac.authorization.k8s.io/v1
    111. kind: ClusterRoleBinding
    112. metadata:
    113.   labels:
    114.     helm.sh/chart: ingress-nginx-4.0.15
    115.     app.kubernetes.io/name: ingress-nginx
    116.     app.kubernetes.io/instance: ingress-nginx
    117.     app.kubernetes.io/version: 1.1.1
    118.     app.kubernetes.io/managed-by: Helm
    119.   name: ingress-nginx
    120. roleRef:
    121.   apiGroup: rbac.authorization.k8s.io
    122.   kind: ClusterRole
    123.   name: ingress-nginx
    124. subjects:
    125.   - kind: ServiceAccount
    126.     name: ingress-nginx
    127.     namespace: ingress-nginx
    128. ---
    129. # Source: ingress-nginx/templates/controller-role.yaml
    130. apiVersion: rbac.authorization.k8s.io/v1
    131. kind: Role
    132. metadata:
    133.   labels:
    134.     helm.sh/chart: ingress-nginx-4.0.15
    135.     app.kubernetes.io/name: ingress-nginx
    136.     app.kubernetes.io/instance: ingress-nginx
    137.     app.kubernetes.io/version: 1.1.1
    138.     app.kubernetes.io/managed-by: Helm
    139.     app.kubernetes.io/component: controller
    140.   name: ingress-nginx
    141.   namespace: ingress-nginx
    142. rules:
    143.   - apiGroups:
    144.       - ''
    145.     resources:
    146.       - namespaces
    147.     verbs:
    148.       - get
    149.   - apiGroups:
    150.       - ''
    151.     resources:
    152.       - configmaps
    153.       - pods
    154.       - secrets
    155.       - endpoints
    156.     verbs:
    157.       - get
    158.       - list
    159.       - watch
    160.   - apiGroups:
    161.       - ''
    162.     resources:
    163.       - services
    164.     verbs:
    165.       - get
    166.       - list
    167.       - watch
    168.   - apiGroups:
    169.       - networking.k8s.io
    170.     resources:
    171.       - ingresses
    172.     verbs:
    173.       - get
    174.       - list
    175.       - watch
    176.   - apiGroups:
    177.       - networking.k8s.io
    178.     resources:
    179.       - ingresses/status
    180.     verbs:
    181.       - update
    182.   - apiGroups:
    183.       - networking.k8s.io
    184.     resources:
    185.       - ingressclasses
    186.     verbs:
    187.       - get
    188.       - list
    189.       - watch
    190.   - apiGroups:
    191.       - ''
    192.     resources:
    193.       - configmaps
    194.     resourceNames:
    195.       - ingress-controller-leader
    196.     verbs:
    197.       - get
    198.       - update
    199.   - apiGroups:
    200.       - ''
    201.     resources:
    202.       - configmaps
    203.     verbs:
    204.       - create
    205.   - apiGroups:
    206.       - ''
    207.     resources:
    208.       - events
    209.     verbs:
    210.       - create
    211.       - patch
    212. ---
    213. # Source: ingress-nginx/templates/controller-rolebinding.yaml
    214. apiVersion: rbac.authorization.k8s.io/v1
    215. kind: RoleBinding
    216. metadata:
    217.   labels:
    218.     helm.sh/chart: ingress-nginx-4.0.15
    219.     app.kubernetes.io/name: ingress-nginx
    220.     app.kubernetes.io/instance: ingress-nginx
    221.     app.kubernetes.io/version: 1.1.1
    222.     app.kubernetes.io/managed-by: Helm
    223.     app.kubernetes.io/component: controller
    224.   name: ingress-nginx
    225.   namespace: ingress-nginx
    226. roleRef:
    227.   apiGroup: rbac.authorization.k8s.io
    228.   kind: Role
    229.   name: ingress-nginx
    230. subjects:
    231.   - kind: ServiceAccount
    232.     name: ingress-nginx
    233.     namespace: ingress-nginx
    234. ---
    235. # Source: ingress-nginx/templates/controller-service-webhook.yaml
    236. apiVersion: v1
    237. kind: Service
    238. metadata:
    239.   labels:
    240.     helm.sh/chart: ingress-nginx-4.0.15
    241.     app.kubernetes.io/name: ingress-nginx
    242.     app.kubernetes.io/instance: ingress-nginx
    243.     app.kubernetes.io/version: 1.1.1
    244.     app.kubernetes.io/managed-by: Helm
    245.     app.kubernetes.io/component: controller
    246.   name: ingress-nginx-controller-admission
    247.   namespace: ingress-nginx
    248. spec:
    249.   type: ClusterIP
    250.   ports:
    251.     - name: https-webhook
    252.       port: 443
    253.       targetPort: webhook
    254.       appProtocol: https
    255.   selector:
    256.     app.kubernetes.io/name: ingress-nginx
    257.     app.kubernetes.io/instance: ingress-nginx
    258.     app.kubernetes.io/component: controller
    259. ---
    260. # Source: ingress-nginx/templates/controller-service.yaml
    261. apiVersion: v1
    262. kind: Service
    263. metadata:
    264.   #annotations:
    265.   #  prometheus.io/scrape: "true"
    266.   #  prometheus.io/port: "10254"
    267.   labels:
    268.     helm.sh/chart: ingress-nginx-4.0.15
    269.     app.kubernetes.io/name: ingress-nginx
    270.     app.kubernetes.io/instance: ingress-nginx
    271.     app.kubernetes.io/version: 1.1.1
    272.     app.kubernetes.io/managed-by: Helm
    273.     app.kubernetes.io/component: controller
    274.   name: ingress-nginx-controller
    275.   namespace: ingress-nginx
    276. spec:
    277.   type: NodePort
    278.   # externalTrafficPolicy: Local
    279.   # ipFamilyPolicy: SingleStack
    280.   # ipFamilies:
    281.   #   - IPv4
    282.   ports:
    283.     - name: http
    284.       port: 80
    285.       protocol: TCP
    286.       targetPort: http
    287.       appProtocol: http
    288.       nodePort: 30049
    289.     - name: https
    290.       port: 443
    291.       protocol: TCP
    292.       targetPort: https
    293.       appProtocol: https
    294.     #- name: prometheus
    295.     #  port: 10254
    296.     #  targetPort: prometheus
    297.   selector:
    298.     app.kubernetes.io/name: ingress-nginx
    299.     app.kubernetes.io/instance: ingress-nginx
    300.     app.kubernetes.io/component: controller
    301. ---
    302. # Source: ingress-nginx/templates/controller-deployment.yaml
    303. apiVersion: apps/v1
    304. kind: Deployment
    305. metadata:
    306.   labels:
    307.     helm.sh/chart: ingress-nginx-4.0.15
    308.     app.kubernetes.io/name: ingress-nginx
    309.     app.kubernetes.io/instance: ingress-nginx
    310.     app.kubernetes.io/version: 1.1.1
    311.     app.kubernetes.io/managed-by: Helm
    312.     app.kubernetes.io/component: controller
    313.   name: ingress-nginx-controller
    314.   namespace: ingress-nginx
    315. spec:
    316.   selector:
    317.     matchLabels:
    318.       app.kubernetes.io/name: ingress-nginx
    319.       app.kubernetes.io/instance: ingress-nginx
    320.       app.kubernetes.io/component: controller
    321.   revisionHistoryLimit: 10
    322.   minReadySeconds: 0
    323.   template:
    324.     metadata:
    325.       labels:
    326.         app.kubernetes.io/name: ingress-nginx
    327.         app.kubernetes.io/instance: ingress-nginx
    328.         app.kubernetes.io/component: controller
    329.       annotations:
    330.  
    331.     spec:
    332.       dnsPolicy: ClusterFirst
    333.       nodeName: k8s-node2
    334.       containers:
    335.         - name: controller
    336.           image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.1.1
    337.           imagePullPolicy: IfNotPresent
    338.           lifecycle:
    339.             preStop:
    340.               exec:
    341.                 command:
    342.                   - /wait-shutdown
    343.           args:
    344.             - /nginx-ingress-controller
    345.             - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
    346.             - --election-id=ingress-controller-leader
    347.             - --controller-class=k8s.io/ingress-nginx
    348.             - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
    349.             - --validating-webhook=:8443
    350.             - --validating-webhook-certificate=/usr/local/certificates/cert
    351.             - --validating-webhook-key=/usr/local/certificates/key
    352.           securityContext:
    353.             capabilities:
    354.               drop:
    355.                 - ALL
    356.               add:
    357.                 - NET_BIND_SERVICE
    358.             runAsUser: 101
    359.             allowPrivilegeEscalation: true
    360.           env:
    361.             - name: POD_NAME
    362.               valueFrom:
    363.                 fieldRef:
    364.                   fieldPath: metadata.name
    365.             - name: POD_NAMESPACE
    366.               valueFrom:
    367.                 fieldRef:
    368.                   fieldPath: metadata.namespace
    369.             - name: LD_PRELOAD
    370.               value: /usr/local/lib/libmimalloc.so
    371.           livenessProbe:
    372.             failureThreshold: 5
    373.             httpGet:
    374.               path: /healthz
    375.               port: 10254
    376.               scheme: HTTP
    377.             initialDelaySeconds: 10
    378.             periodSeconds: 10
    379.             successThreshold: 1
    380.             timeoutSeconds: 1
    381.           readinessProbe:
    382.             failureThreshold: 3
    383.             httpGet:
    384.               path: /healthz
    385.               port: 10254
    386.               scheme: HTTP
    387.             initialDelaySeconds: 10
    388.             periodSeconds: 10
    389.             successThreshold: 1
    390.             timeoutSeconds: 1
    391.           ports:
    392.             - name: http
    393.               containerPort: 80
    394.               hostPort: 80
    395.               protocol: TCP
    396.             - name: https
    397.               containerPort: 443
    398.               hostPort: 443
    399.               protocol: TCP
    400.             - name: webhook
    401.               containerPort: 8443
    402.               protocol: TCP
    403.           volumeMounts:
    404.             - name: webhook-cert
    405.               mountPath: /usr/local/certificates/
    406.               readOnly: true
    407.           resources:
    408.             requests:
    409.               cpu: 100m
    410.               memory: 90Mi
    411.       nodeSelector:
    412.         kubernetes.io/os: linux
    413.       serviceAccountName: ingress-nginx
    414.       terminationGracePeriodSeconds: 300
    415.       volumes:
    416.         - name: webhook-cert
    417.           secret:
    418.             secretName: ingress-nginx-admission
    419. ---
    420. # Source: ingress-nginx/templates/controller-ingressclass.yaml
    421. # We don't support namespaced ingressClass yet
    422. # So a ClusterRole and a ClusterRoleBinding is required
    423. apiVersion: networking.k8s.io/v1
    424. kind: IngressClass
    425. metadata:
    426.   labels:
    427.     helm.sh/chart: ingress-nginx-4.0.15
    428.     app.kubernetes.io/name: ingress-nginx
    429.     app.kubernetes.io/instance: ingress-nginx
    430.     app.kubernetes.io/version: 1.1.1
    431.     app.kubernetes.io/managed-by: Helm
    432.     app.kubernetes.io/component: controller
    433.   name: nginx
    434.   namespace: ingress-nginx
    435. spec:
    436.   controller: k8s.io/ingress-nginx
    437. ---
    438. # Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
    439. # before changing this value, check the required kubernetes version
    440. # https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
    441. apiVersion: admissionregistration.k8s.io/v1
    442. kind: ValidatingWebhookConfiguration
    443. metadata:
    444.   labels:
    445.     helm.sh/chart: ingress-nginx-4.0.15
    446.     app.kubernetes.io/name: ingress-nginx
    447.     app.kubernetes.io/instance: ingress-nginx
    448.     app.kubernetes.io/version: 1.1.1
    449.     app.kubernetes.io/managed-by: Helm
    450.     app.kubernetes.io/component: admission-webhook
    451.   name: ingress-nginx-admission
    452. webhooks:
    453.   - name: validate.nginx.ingress.kubernetes.io
    454.     matchPolicy: Equivalent
    455.     rules:
    456.       - apiGroups:
    457.           - networking.k8s.io
    458.         apiVersions:
    459.           - v1
    460.         operations:
    461.           - CREATE
    462.           - UPDATE
    463.         resources:
    464.           - ingresses
    465.     failurePolicy: Fail
    466.     sideEffects: None
    467.     admissionReviewVersions:
    468.       - v1
    469.     clientConfig:
    470.       service:
    471.         namespace: ingress-nginx
    472.         name: ingress-nginx-controller-admission
    473.         path: /networking/v1/ingresses
    474. ---
    475. # Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
    476. apiVersion: v1
    477. kind: ServiceAccount
    478. metadata:
    479.   name: ingress-nginx-admission
    480.   namespace: ingress-nginx
    481.   annotations:
    482.     helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    483.     helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    484.   labels:
    485.     helm.sh/chart: ingress-nginx-4.0.15
    486.     app.kubernetes.io/name: ingress-nginx
    487.     app.kubernetes.io/instance: ingress-nginx
    488.     app.kubernetes.io/version: 1.1.1
    489.     app.kubernetes.io/managed-by: Helm
    490.     app.kubernetes.io/component: admission-webhook
    491. ---
    492. # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
    493. apiVersion: rbac.authorization.k8s.io/v1
    494. kind: ClusterRole
    495. metadata:
    496.   name: ingress-nginx-admission
    497.   annotations:
    498.     helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    499.     helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    500.   labels:
    501.     helm.sh/chart: ingress-nginx-4.0.15
    502.     app.kubernetes.io/name: ingress-nginx
    503.     app.kubernetes.io/instance: ingress-nginx
    504.     app.kubernetes.io/version: 1.1.1
    505.     app.kubernetes.io/managed-by: Helm
    506.     app.kubernetes.io/component: admission-webhook
    507. rules:
    508.   - apiGroups:
    509.       - admissionregistration.k8s.io
    510.     resources:
    511.       - validatingwebhookconfigurations
    512.     verbs:
    513.       - get
    514.       - update
    515. ---
    516. # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
    517. apiVersion: rbac.authorization.k8s.io/v1
    518. kind: ClusterRoleBinding
    519. metadata:
    520.   name: ingress-nginx-admission
    521.   annotations:
    522.     helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    523.     helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    524.   labels:
    525.     helm.sh/chart: ingress-nginx-4.0.15
    526.     app.kubernetes.io/name: ingress-nginx
    527.     app.kubernetes.io/instance: ingress-nginx
    528.     app.kubernetes.io/version: 1.1.1
    529.     app.kubernetes.io/managed-by: Helm
    530.     app.kubernetes.io/component: admission-webhook
    531. roleRef:
    532.   apiGroup: rbac.authorization.k8s.io
    533.   kind: ClusterRole
    534.   name: ingress-nginx-admission
    535. subjects:
    536.   - kind: ServiceAccount
    537.     name: ingress-nginx-admission
    538.     namespace: ingress-nginx
    539. ---
    540. # Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
    541. apiVersion: rbac.authorization.k8s.io/v1
    542. kind: Role
    543. metadata:
    544.   name: ingress-nginx-admission
    545.   namespace: ingress-nginx
    546.   annotations:
    547.     helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    548.     helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    549.   labels:
    550.     helm.sh/chart: ingress-nginx-4.0.15
    551.     app.kubernetes.io/name: ingress-nginx
    552.     app.kubernetes.io/instance: ingress-nginx
    553.     app.kubernetes.io/version: 1.1.1
    554.     app.kubernetes.io/managed-by: Helm
    555.     app.kubernetes.io/component: admission-webhook
    556. rules:
    557.   - apiGroups:
    558.       - ''
    559.     resources:
    560.       - secrets
    561.     verbs:
    562.       - get
    563.       - create
    564. ---
    565. # Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
    566. apiVersion: rbac.authorization.k8s.io/v1
    567. kind: RoleBinding
    568. metadata:
    569.   name: ingress-nginx-admission
    570.   namespace: ingress-nginx
    571.   annotations:
    572.     helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    573.     helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    574.   labels:
    575.     helm.sh/chart: ingress-nginx-4.0.15
    576.     app.kubernetes.io/name: ingress-nginx
    577.     app.kubernetes.io/instance: ingress-nginx
    578.     app.kubernetes.io/version: 1.1.1
    579.     app.kubernetes.io/managed-by: Helm
    580.     app.kubernetes.io/component: admission-webhook
    581. roleRef:
    582.   apiGroup: rbac.authorization.k8s.io
    583.   kind: Role
    584.   name: ingress-nginx-admission
    585. subjects:
    586.   - kind: ServiceAccount
    587.     name: ingress-nginx-admission
    588.     namespace: ingress-nginx
    589. ---
    590. # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
    591. apiVersion: batch/v1
    592. kind: Job
    593. metadata:
    594.   name: ingress-nginx-admission-create
    595.   namespace: ingress-nginx
    596.   annotations:
    597.     helm.sh/hook: pre-install,pre-upgrade
    598.     helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    599.   labels:
    600.     helm.sh/chart: ingress-nginx-4.0.15
    601.     app.kubernetes.io/name: ingress-nginx
    602.     app.kubernetes.io/instance: ingress-nginx
    603.     app.kubernetes.io/version: 1.1.1
    604.     app.kubernetes.io/managed-by: Helm
    605.     app.kubernetes.io/component: admission-webhook
    606. spec:
    607.   template:
    608.     metadata:
    609.       name: ingress-nginx-admission-create
    610.       labels:
    611.         helm.sh/chart: ingress-nginx-4.0.15
    612.         app.kubernetes.io/name: ingress-nginx
    613.         app.kubernetes.io/instance: ingress-nginx
    614.         app.kubernetes.io/version: 1.1.1
    615.         app.kubernetes.io/managed-by: Helm
    616.         app.kubernetes.io/component: admission-webhook
    617.     spec:
    618.       containers:
    619.         - name: create
    620.           image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
    621.           imagePullPolicy: IfNotPresent
    622.           args:
    623.             - create
    624.             - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
    625.             - --namespace=$(POD_NAMESPACE)
    626.             - --secret-name=ingress-nginx-admission
    627.           env:
    628.             - name: POD_NAMESPACE
    629.               valueFrom:
    630.                 fieldRef:
    631.                   fieldPath: metadata.namespace
    632.           securityContext:
    633.             allowPrivilegeEscalation: false
    634.       restartPolicy: OnFailure
    635.       serviceAccountName: ingress-nginx-admission
    636.       nodeSelector:
    637.         kubernetes.io/os: linux
    638.       securityContext:
    639.         runAsNonRoot: true
    640.         runAsUser: 2000
    641. ---
    642. # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
    643. apiVersion: batch/v1
    644. kind: Job
    645. metadata:
    646.   name: ingress-nginx-admission-patch
    647.   namespace: ingress-nginx
    648.   annotations:
    649.     helm.sh/hook: post-install,post-upgrade
    650.     helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    651.   labels:
    652.     helm.sh/chart: ingress-nginx-4.0.15
    653.     app.kubernetes.io/name: ingress-nginx
    654.     app.kubernetes.io/instance: ingress-nginx
    655.     app.kubernetes.io/version: 1.1.1
    656.     app.kubernetes.io/managed-by: Helm
    657.     app.kubernetes.io/component: admission-webhook
    658. spec:
    659.   template:
    660.     metadata:
    661.       name: ingress-nginx-admission-patch
    662.       labels:
    663.         helm.sh/chart: ingress-nginx-4.0.15
    664.         app.kubernetes.io/name: ingress-nginx
    665.         app.kubernetes.io/instance: ingress-nginx
    666.         app.kubernetes.io/version: 1.1.1
    667.         app.kubernetes.io/managed-by: Helm
    668.         app.kubernetes.io/component: admission-webhook
    669.     spec:
    670.       containers:
    671.         - name: patch
    672.           image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
    673.           imagePullPolicy: IfNotPresent
    674.           args:
    675.             - patch
    676.             - --webhook-name=ingress-nginx-admission
    677.             - --namespace=$(POD_NAMESPACE)
    678.             - --patch-mutating=false
    679.             - --secret-name=ingress-nginx-admission
    680.             - --patch-failure-policy=Fail
    681.           env:
    682.             - name: POD_NAMESPACE
    683.               valueFrom:
    684.                 fieldRef:
    685.                   fieldPath: metadata.namespace
    686.           securityContext:
    687.             allowPrivilegeEscalation: false
    688.       restartPolicy: OnFailure
    689.       serviceAccountName: ingress-nginx-admission
    690.       nodeSelector:
    691.         kubernetes.io/os: linux
    692.       securityContext:
    693.         runAsNonRoot: true
    694.         runAsUser: 2000

    2.2 部署 Ingress

    把 ingress-deployment.yaml 文件上传到 Kubernetes 集群的 master 节点，执行命令部署 Ingress。

    kubectl apply -f ingress-deployment.yaml

    步骤 3： 部署 Nginx

    3.1 编写 nginx-deployment.yaml

    1. apiVersion: apps/v1
    2. kind: Deployment
    3. metadata:
    4.   name: nginx-deployment
    5. spec:
    6.   selector:
    7.     matchLabels:
    8.       app: backend
    9.   replicas: 1
    10.   template:
    11.     metadata:
    12.       labels:
    13.         app: backend
    14.     spec:
    15.       nodeName: k8s-node2
    16.       containers:
    17.       - name: nginx
    18.         image: nginx:latest
    19.         resources:
    20.           limits:
    21.             memory: "128Mi"
    22.             cpu: "128m"
    23.         ports:
    24.         - containerPort: 80
    25.  
    26. ---
    27.  
    28. apiVersion: v1
    29. kind: Service
    30. metadata:
    31.   name: nginx-service
    32. spec:
    33.   selector:
    34.     app: backend
    35.   ports:
    36.     - port: 80
    37.       targetPort: 80

    3.2 编写 nginx-ingress.yaml

    1. apiVersion: networking.k8s.io/v1
    2. kind: Ingress
    3. metadata:
    4.   name: nodeport-ingress
    5.   namespace: default
    6.   annotations:
    7.     kubernetes.io/ingress.class: "nginx"
    8. spec:
    9.   rules:
    10.   - host: mynginx.com
    11.     http:
    12.       paths:
    13.       - pathType: Prefix
    14.         path: /
    15.         backend:
    16.           service:
    17.             name: nginx-service
    18.             port:
    19.               number: 80

    3.3 部署 Nginx

    把 nginx-deployment.yaml 和 nginx-ingress.yaml 上传到 Kubernetes 集群的 master 节点，执行命令。

    1. kubectl apply -f nginx-deployment.yaml
    2. kubectl apply -f nginx-ingress.yaml

    步骤 4： 创建 CRD

    4.1 编写 datakit-crd.yaml

    1. apiVersion: apiextensions.k8s.io/v1
    2. kind: CustomResourceDefinition
    3. metadata:
    4.   name: datakits.guance.com
    5. spec:
    6.   group: guance.com
    7.   versions:
    8.     - name: v1beta1
    9.       served: true
    10.       storage: true
    11.       schema:
    12.         openAPIV3Schema:
    13.           type: object
    14.           properties:
    15.             spec:
    16.               type: object
    17.               properties:
    18.                 instances:
    19.                   type: array
    20.                   items:
    21.                     type: object
    22.                     properties:
    23.                       k8sNamespace:
    24.                         type: string
    25.                       k8sDeployment:
    26.                         type: string
    27.                       datakit/logs:
    28.                         type: string
    29.                       inputConf:
    30.                         type: string
    31.   scope: Namespaced
    32.   names:
    33.     plural: datakits
    34.     singular: datakit
    35.     kind: Datakit
    36.     shortNames:
    37.     - dk

    4.2 编写 ingress-crd.yaml

    1. apiVersion: "guance.com/v1beta1"
    2. kind: Datakit
    3. metadata:
    4.   name: prom-ingress
    5.   namespace: datakit
    6. spec:
    7.   instances:
    8.     - k8sNamespace: "ingress-nginx"
    9.       k8sDeployment: "ingress-nginx-controller"
    10.       inputConf: |
    11.         [[inputs.prom]]
    12.           url = "http://$IP:10254/metrics"
    13.           source = "prom-ingress"
    14.           metric_types = ["counter", "gauge","histogram"]
    15.           # metric_name_filter = ["cpu"]
    16.           # measurement_prefix = ""
    17.           measurement_name = "prom_ingress"
    18.           interval = "60s"
    19.           tags_ignore = ["build","le","path","method","release","repository"]
    20.           metric_name_filter = ["nginx_process_cpu_seconds_total","nginx_process_resident_memory_bytes","request_size","response_size","requests","success","config_last_reload_successful"]
    21.           [[inputs.prom.measurements]]
    22.             prefix = "nginx_ingress_controller_"
    23.             name = "prom_ingress"
    24.           [inputs.prom.tags]
    25.             namespace = "$NAMESPACE"

    4.3 部署 CRD

    把文件上传到 Kubernets 集群的 master 节点，执行命令。

    1. kubectl apply -f datakit-crd.yaml
    2. kubectl apply -f ingress-crd.yaml

    步骤 5： 场景视图

    5.1 访问 nginx

    执行 kubectl get svc -n ingress-nginx 获取到 80 对应的节点端口是 30049。

    使用节点 8.136.207.182 来访问 Nginx。

    while true; do sleep 1;curl -v http://8.136.207.182 -H 'host: mynginx.com'; done

    5.2 Ingress 可观测

    登录『观测云』，点击『场景』模块，新建仪表板，搜索 ingress，点击『确定』。

    上步执行的 while 命令会模拟访问 mynginx.com，监控视图展示出 Ingress 的使用情况。

  • 相关阅读:
    Java IO之网络的简介说明
    终于辞职了，全职ue了
    java IO流 序列化流
    软件开发项目文档系列之五如何撰写需求规格说明书
    mybatis动态表名
    QT定时器简单应用
    【CVPR 2022】NeRFReN: Neural Radiance Fields with Reflections
    Sketch在mac运行时崩溃，什么是安全模式以及如何启用它？
    【OAuth2】十五、客户端认证流程-自定义授权页面和客户端认证
    主辅助服务市场出清模型研究【旋转备用】（Matlab代码实现）
  • 原文地址：https://blog.csdn.net/DataFlux/article/details/126727973