ELK集添加安全认证

一、ElasticSearch配置

1、elasticsearch集群启用SSL
编辑所有elasticsearch节点 elasticsearch/config/elasticsearch.yml 配置文件新增以下内容：

vi elasticsearch.yml

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
1
2
3
4
5
6
7

2、生成证书
注意：如果是在docker中部署的，需要进入到容器中进行操作
生成的证书默认都在 /usr/share/elasticsearch 目录下：

cd /usr/share/elasticsearch/bin
 ./elasticsearch-certutil cert -out /usr/share/elasticsearch/elastic-certificates.p12 -pass ""
 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.bouncycastle.jcajce.provider.drbg.DRBG (file:/home/estestuser/elasticsearch-7.1.1/lib/tools/security-cli/bcprov-jdk15on-1.61.jar) to constructor sun.security.provider.Sun()
WARNING: Please consider reporting this to the maintainers of org.bouncycastle.jcajce.provider.drbg.DRBG
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires a SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.

    * All certificates generated by this tool will be signed by a certificate authority (CA).
    * The tool can automatically generate a new CA for you, or you can provide your own with the
         -ca or -ca-cert command line options.

By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -keep-ca-key (retain generated CA key)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files


Certificates written to /home/estestuser/elasticsearch-7.1.1/config/elastic-certificates.p12

This file should be properly secured as it contains the private key for 
your instance.

This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

查看是否成功生成证书

ls -l /usr/share/elasticsearch/elastic-*                
-rw------- 1 root root 3596 May 27 05:58 /usr/share/elasticsearch/elastic-certificates.p12
-rw------- 1 root root 2672 May 27 05:53 /usr/share/elasticsearch/elastic-stack-ca.p12
1
2
3

把这两个证书移动到 /usr/share/elasticsearch/config/certs(elasticsearch.yml同级目录下) 目录里

mkdir /usr/share/elasticsearch/config/certs
mv /usr/share/elasticsearch/elastic-* /usr/share/elasticsearch/config/certs
1
2

另外，需要注意修改证书的权限
生成的elastic-certificates.p12文件拷贝到每个节点的/usr/share/elasticsearch/config/certs目录下
3、重启 ES 集群
 ES 集群不启动，下面的添加密码操作执行不了，所以依次重启 3 个节点。首先重启节点 1。

ps -ef|grep elasticsearch
kill -9 pid

bin/elasticsearch -d
1
2
3
4

依次重启节点 2 和节点 3。
4、创建 Elasticsearch 集群密码
在主节点 上执行如下命令，设置用户密码。设置完之后，数据会自动同步到其他节点。
bin/elasticsearch-setup-passwords interactive 设置密码
bin/elasticsearch-setup-passwords auto 自动生成密码

[root@wecity elasticsearch-7.1.1]$ bin/elasticsearch-setup-passwords interactive

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

5、访问验证

 curl http://localhost:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
1
2

输入帐号：elastic，密码：password（此密码非真实密码，仅为了写博客记录），再次访问，发现成功。

[estestuser@vm-10-201-42-9 elasticsearch-7.1.1]$ curl --user elastic:password http://localhost:9200
{
  "name" : "node-1",
  "cluster_name" : "my-application",
  "cluster_uuid" : "l-kTWdoxRCuTIm6x8ekG7w",
  "version" : {
    "number" : "7.1.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "7a013de",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

为集群添加用户安全认证成功。

二、Kibana配置

1、 在kibana.yml中添加配置用户名和密码

elasticsearch.username: "kibana"
elasticsearch.password: "*****"
server.publicBaseUrl: "http://10.185.1.178:5600" #默认访问地址，如果挂了域名指向，这里可以写域名地址
i18n.locale: "zh-CN" #kibana的默认语言配置，默认是英文，我这里改成了中文，英文好的同学可以忽略
1
2
3
4

2、重启Kibana

service kibana  restart
1

重启后，输入http://ip:5600打开登录页面，使用elastic账号登录。登录进入系统后进入管理按钮(Management)，就可以看到多了一块东西，就是用户与角色权限配置，在角色和用户管理中添加用户指定 索引用于访问ES。

三、Logstash配置

1、在logstash.yml中配置用户名和密码

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: *****
xpack.monitoring.elasticsearch.hosts: ["http://ip:9200"]
1
2
3
4

2、在logstash指定的conf文件中加上用户名密码
在本案例中获取nginx的日志，文件是 logstash/conf.d/nginx.conf。在这个文件的output 方法下增加user 和 password。

output {
    elasticsearch {
        hosts => ["192.168.1.240:9200"]
        index => "nginx-access-log-%{+YYYY.MM.dd}"
        user => "elastic"
        password => "******"
    }
    stdout { codec => rubydebug }
}
1
2
3
4
5
6
7
8
9

2、重启Logstash
启动命令 bin/logstash -f 上面配置的文件位置

bin/logstash -f nginx.conf
1

小知识验证logstash

bin/logstash -e 'input { stdin { } } output { stdout {} }'
1