Kubernetes 审计(Auditing) 功能提供了与安全相关的、按时间顺序排列的记录集, 记录每个用户、使用 Kubernetes API 的应用以及控制面自身引发的活动。
审计功能使得集群管理员能够回答以下问题:
审计记录最初产生于 kube-apiserver 内部。每个请求在不同执行阶段都会生成审计事件;这些审计事件会根据特定策略 被预处理并写入后端。策略确定要记录的内容和用来存储记录的后端。 当前的后端支持日志文件和 webhook。
每个请求都可被记录其相关的 阶段(stage)。已定义的阶段有:
审计日志记录功能会增加 API server 的内存消耗,因为需要为每个请求存储审计所需的某些上下文。 此外,内存消耗取决于审计日志记录的配置。
审计政策定义了关于应记录哪些事件以及应包含哪些数据的规则。 审计策略对象结构定义在 audit.k8s.io 的API组中。
处理事件时,将按顺序与规则列表进行比较。第一个匹配规则设置事件的审计级别(Audit Level)。已定义的审计级别有:
也可以使用 --audit-policy-file 标志将包含策略的文件传递给 kube-apiserver。 如果不设置该参数,则不记录事件。 注意 rules 字段必须在审计策略文件中提供。
审计后端实现将审计事件导出到外部存储。Kube-apiserver 默认提供两个后端:
cd /etc/kubernetes
vim audit-policy.yaml
apiVersion: audit.k8s.io/v1 # This is required.
kind: Policy
# Don't generate audit events for all requests in RequestReceived stage.
omitStages:
- "RequestReceived"
rules:
# Log pod changes at RequestResponse level
- level: RequestResponse
resources:
- group: ""
# Resource "pods" doesn't match requests to any subresource of pods,
# which is consistent with the RBAC policy.
resources: ["pods"]
# Log "pods/log", "pods/status" at Metadata level
- level: Metadata
resources:
- group: ""
resources: ["pods/log", "pods/status"]
# Don't log requests to a configmap called "controller-leader"
- level: None
resources:
- group: ""
resources: ["configmaps"]
resourceNames: ["controller-leader"]
# Don't log watch requests by the "system:kube-proxy" on endpoints or services
- level: None
users: ["system:kube-proxy"]
verbs: ["watch"]
resources:
- group: "" # core API group
resources: ["endpoints", "services"]
# Don't log authenticated requests to certain non-resource URL paths.
- level: None
userGroups: ["system:authenticated"]
nonResourceURLs:
- "/api*"
- "/version"
# Log the request body of configmap changes in kube-system.
- level: Request
resources:
- group: "" # core API group
resources: ["configmaps"]
# This rule only applies to resources in the "kube-system" namespace.
# The empty string "" can be used to select non-namespaced resources.
namespaces: ["kube-system"]
# Log configmap and secret changes in all other namespaces at the Metadata level.
- level: Metadata
resources:
- group: "" # core API group
resources: ["secrets", "configmaps"]
# Log all other resources in core and extensions at the Request level.
- level: Request
resources:
- group: "" # core API group
- group: "extensions"
# A catch-all rule to log all other requests at the Metadata level.
- level: Metadata
# Long-running requests like watches that fall under this rule will not
# generate an audit event in RequestReceived.
omitStages:
- "RequestReceived"
可以使用最低限度的审计策略文件在 Metadata 级别记录所有请求:
# 在 Metadata 级别为所有请求生成日志
apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
- level: Metadata
vim /etc/kubernetes/manifests/kube-apiserver.yaml
- --audit-policy-file=/etc/kubernetes/audit-policy.yaml
# 指定用来写入审计事件的日志文件路径。不指定此标志会禁用日志后端。
- --audit-log-path=/var/log/audit.log
# 定义审计日志文件的最大大小(兆字节)
- --audit-log-maxsize=100
# 定义了保留旧审计日志文件的最大天数
- --audit-log-maxage=5
# 定义了要保留的审计日志文件的最大数量
- --audit-log-maxbackup=3
# 开启日志压缩
- --audit-log-compress
# 开启截断
- --audit-log-truncate-enabled=true
volumeMounts:
- mountPath: /etc/kubernetes/audit-policy.yaml
name: audit
readOnly: true
- mountPath: /var/log/k8s
name: audit-log
- name: audit
hostPath:
path: /etc/kubernetes/audit-policy.yaml
type: File
- name: audit-log
hostPath:
path: /var/log/k8s
type: DirectoryOrCreate
保存后等待apiserver重启完成。
我这里审计日志没有自动轮转,而是到了100m后日志就不动了。查看apiserver日志有如下报错:
yum -y install logrotate
cat /etc/logrotate.d/audit
/var/log/k8s/audit.log {
# 指定文件属性
create 0644 root root
# 不压缩
nocompress
# 备份日志文件但是不截断
nocopytruncate
# 如果是空文件的话,不转储
notifempty
# 如果日志丢失,不报错继续滚动下一个日志
missingok
# 转储后的日志文件和当前日志文件放在同一个目录下
noolddir
# 使用当期日期作为命名格式
dateext
# 指定日志文件删除之前转储的个数,5指保留5个备份
rotate 10
# 当日志文件到达指定的大小时才转储
size=100MB
*/5 * * * * /usr/sbin/logrotate -f /etc/logrotate.d/audit
ll /var/log/k8s/