• elasticSearch+kibana+logstash+filebeat集群改成https认证

    文章目录

    一、生成相关证书

    ps：主节点操作

    1. 切换用户：su es

    2. 进入目录：cd /home/es/elasticsearch-7.6.2

    3. 创建文件：vi instances.yml

      instances:
  - name: "master" 
    ip: 
      - "192.168.248.10"
  - name: "slave1"
    ip:
      - "192.168.248.11"
  - name: "slave2"
    ip:
      - "192.168.248.12"
  - name: "kibana"
    ip:
      - "192.168.248.10"
  - name: "logstash"
    ip:
      - "192.168.248.10"   
  - name: "filebeat"
    ip:
      - "192.168.248.10"   

      • 1
      • 2
      • 3
      • 4
      • 5
      • 6
      • 7
      • 8
      • 9
      • 10
      • 11
      • 12
      • 13
      • 14
      • 15
      • 16
      • 17
      • 18
      • 19

    4. 生成证书：/home/es/elasticsearch-7.6.2/bin/elasticsearch-certutil cert ca --pem --in instances.yml --out certs.zip

    5. 解压得到各个证书：unzip certs.zip

    二、配置elasticSearh

    ps：三个节点

    1. 切换用户：su es

    2. 将解压得到的三个文件夹文件拷贝到各个节点的/home/es/elasticsearch-7.6.2/config下，如master节点：ca.crt、master.crt、master.key

    3. 三个节点配置，末尾添加配置：vi /home/es/elasticsearch-7.6.2/config/elasticsearch.yml
      ps1：根据名字配置master和slave1和slave2
      ps2：如果之前配置过密码，需要将密码的配置先移除

      xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: master.key
xpack.security.http.ssl.certificate: master.crt
xpack.security.http.ssl.certificate_authorities: ca.crt

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: master.key
xpack.security.transport.ssl.certificate: master.crt
xpack.security.transport.ssl.certificate_authorities: ["ca.crt"]

      • 1
      • 2
      • 3
      • 4
      • 5
      • 6
      • 7
      • 8
      • 9
      • 10

    4. 如果是用root用户拷贝的，记得赋权：chown -R es:es /home/es

    5. 启动

      cd /home/es/elasticsearch-7.6.2/bin
nohup /home/es/elasticsearch-7.6.2/bin/elasticsearch &

      • 1
      • 2

    6. 设置密码：/home/es/elasticsearch-7.6.2/bin/elasticsearch-setup-passwords interactive

      ps1：可以统一设置一个密码ffcsict123

      ps2：如果已经设置过密码了，可以忽略。或者也可以删除es的 .security-7 索引，重新执行设置密码的操作也可以

    三、配置kibana

    1. 将kibana证书放到/home/es/kibana-7.6.2-linux-x86_64/config下：kibana.crt、ca.crt、kibana.key

    2. 配置：vi /home/es/kibana-7.6.2-linux-x86_64/config/kibana.yml

      # 修改
elasticsearch.hosts: ["https://192.168.248.10:9200","https://192.168.248.11:9200","https://192.168.248.12:9200"]

# 末尾添加
# 这三个路径写成相对路径会被错，写绝对路径才行，不知道为啥
server.ssl.enabled: true
server.ssl.certificate: /home/es/kibana-7.6.2-linux-x86_64/config/kibana.crt
server.ssl.key: /home/es/kibana-7.6.2-linux-x86_64/config/kibana.key
elasticsearch.ssl.certificateAuthorities: ["/home/es/kibana-7.6.2-linux-x86_64/config/ca.crt"]

elasticsearch.username: "kibana"
elasticsearch.password: "ffcsict123"

      • 1
      • 2
      • 3
      • 4
      • 5
      • 6
      • 7
      • 8
      • 9
      • 10
      • 11
      • 12

    3. 如果是用root用户拷贝的，记得赋权：chown -R es:es /home/es

    4. 启动：nohup /home/es/kibana-7.6.2-linux-x86_64/bin/kibana &

    5. 访问：https://192.168.248.10:5601
      在这里插入图片描述

    四、配置logstash

    1. 将logstash证书放到/home/es/logstash-7.6.2/config下：logstash.crt、ca.crt、logstash.key

    2. 修改配置文件：vi /home/es/logstash-7.6.2/config/logstash.yml

      xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: ffcsict123
xpack.monitoring.elasticsearch.hosts: ["https://192.168.248.10:9200"]
xpack.monitoring.elasticsearch.ssl.certificate_authority: "/home/es/logstash-7.6.2/config/ca.crt"

      • 1
      • 2
      • 3
      • 4
      • 5

    3. 修改配置文件：vi /home/es/logstash-7.6.2/config/logstash-sample.conf

      output {
   elasticsearch {
    hosts => ["https://192.168.248.10:9200","https://192.168.248.11:9200","https://192.168.248.12:9200"]
    index => "testlog-%{+YYYY.MM.dd}"
    user => "elastic"
    password => "ffcsict123"
    ssl => true
    cacert => "/home/es/logstash-7.6.2/config/ca.crt"
   }
 }

      • 1
      • 2
      • 3
      • 4
      • 5
      • 6
      • 7
      • 8
      • 9
      • 10

    4. 启动：nohup /home/es/logstash-7.6.2/bin/logstash -f /home/es/logstash-7.6.2/config/logstash-sample.conf &

    五、配置filebeat

    --------------------如果logstash不需要转https，则可以忽略以下步骤-----------------

    1. 杀死logstash进程

    2. 将 logstash.key 转换为 PKCS#8 格式

      cd /home/es/logstash-7.6.2/config
openssl pkcs8 -in logstash.key -topk8 -nocrypt -out logstash.pkcs8.key

      • 1
      • 2

    3. 由于我们一个logstash服务，会有很多filebeat服务写日志进来。如果将logstash改为https访问，需要将所有涉及的filebeat都进行改配置。所以如果只是要求es改造为https，可以不改造logstash。如果需要改造，则修改配置文件：vi /home/es/logstash-7.6.2/config/logstash-sample.conf

      input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/home/es/logstash-7.6.2/config/ca.crt"]
    ssl_certificate => "/home/es/logstash-7.6.2/config/logstash.crt"
    ssl_key => "/home/es/logstash-7.6.2/config/logstash.pkcs8.key"
    ssl_verify_mode => "force_peer"
  }
}

      • 1
      • 2
      • 3
      • 4
      • 5
      • 6
      • 7
      • 8
      • 9
      • 10

    4. 启动logstash：nohup /home/es/logstash-7.6.2/bin/logstash -f /home/es/logstash-7.6.2/config/logstash-sample.conf &

    5. 将filebeat证书放到/home/es/filebeat-7.6.2-linux-x86_64下：filebeat.crt、filebeat.crt、filebeat.key

    6. 配置filebeat：vi /home/es/filebeat-7.6.2-linux-x86_64/filebeat.yml

      output.logstash:
  hosts: ["192.168.248.10:5044"]
  ssl.certificate_authorities: ["/home/es/filebeat-7.6.2-linux-x86_64/ca.crt"]
  ssl.certificate: "/home/es/filebeat-7.6.2-linux-x86_64/filebeat.crt"
  ssl.key: "/home/es/filebeat-7.6.2-linux-x86_64/filebeat.key"

      • 1
      • 2
      • 3
      • 4
      • 5

    7. 启动：nohup /home/es/filebeat-7.6.2-linux-x86_64/filebeat -e -c /home/es/filebeat-7.6.2-linux-x86_64/filebeat.yml &

    六、连接https es的java api

    import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.SSLContexts;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.client.*;
import org.elasticsearch.cluster.metadata.AliasMetadata;
import org.springframework.core.io.ClassPathResource;

import javax.net.ssl.SSLContext;
import java.io.IOException;
import java.io.InputStream;
import java.security.KeyStore;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
import java.util.*;

/**
 * @author 天真热
 * @create 2023-09-02 20:25
 * @desc
 **/
public class elkDemo {
    public static String ip = "192.168.248.10";
    public static String port = "9200";
    public static String esUsername = "elastic";
    public static String esPassword = "ffcsict123";

    public static void main(String[] args) throws IOException {
        {

            RestHighLevelClient clinet = getConnection();
            List<Map<String, Object>> indexs = getIndex(clinet);
            System.out.println(indexs);
        }
    }

    /**
     * 创建链接
     * @return
     */
    public static RestHighLevelClient getConnection() {
        // 创建凭据提供程序
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(esUsername, esPassword));


        RestClientBuilder http = RestClient.builder(new HttpHost(ip, Integer.parseInt(port), "https"))
                .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
                    @Override
                    public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
                        requestConfigBuilder.setConnectTimeout(700000);
                        requestConfigBuilder.setSocketTimeout(600000);
                        requestConfigBuilder.setConnectionRequestTimeout(100000);
                        return requestConfigBuilder;
                    }
                }).setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpAsyncClientBuilder) {
                        return httpAsyncClientBuilder.setSSLContext(buildSSLContext())
                                .setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE).setDefaultCredentialsProvider(credentialsProvider);
                    }
                });
        return new RestHighLevelClient(http);
    }

    /**
     * 获取所有索引
     */
    public static List<Map<String, Object>> getIndex(RestHighLevelClient esHighInit) throws IOException {
        List<Map<String, Object>> resultList = new ArrayList();
        GetAliasesRequest request = new GetAliasesRequest();
        GetAliasesResponse alias = esHighInit.indices().getAlias(request, RequestOptions.DEFAULT);
        Map<String, Set<AliasMetadata>> map = alias.getAliases();
        map.forEach((k, v) -> {
            if (!k.startsWith(".")) {//忽略elasticesearch 默认的
                Map map1 = new HashMap();
                map1.put("indexName", k);
                resultList.add(map1);
            }
        });

        return resultList;
    }


    /**
     * 创建证书验证
     * @return
     */
    private static SSLContext buildSSLContext() {
        ClassPathResource resource = new ClassPathResource("master.crt");

        SSLContext sslContext = null;
        try {
            CertificateFactory factory = CertificateFactory.getInstance("X.509");
            Certificate trustedCa;
            try (InputStream is = resource.getInputStream()) {
                trustedCa = factory.generateCertificate(is);
            }
            KeyStore trustStore = KeyStore.getInstance("pkcs12");
            trustStore.load(null, null);
            trustStore.setCertificateEntry("ca", trustedCa);
            SSLContextBuilder sslContextBuilder = SSLContexts.custom()
                    .loadTrustMaterial(trustStore, null);
            sslContext = sslContextBuilder.build();
        } catch (Exception e) {
            e.printStackTrace();
        }

        return sslContext;
    }
}


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94
    • 95
    • 96
    • 97
    • 98
    • 99
    • 100
    • 101
    • 102
    • 103
    • 104
    • 105
    • 106
    • 107
    • 108
    • 109
    • 110
    • 111
    • 112
    • 113
    • 114
    • 115
    • 116
    • 117
    • 118
    • 119
    • 120
  • 相关阅读:
    使用Cisco进行模拟配置OSPF路由协议
    面试真题汇总430家
    DeepStream系列之6.1版本安装及测试
    iOS开发UITableView的使用，区别Plain模式和Grouped模式
    leetcode 739. Daily Temperatures 每日温度(中等)
    基于openeuler的NAT服务器查看流向内部服务端口的流量
    JVM垃圾回收器
    vue3 antv 静态登录页面
    解决git@github.com: Permission denied (publickey)
    含文档+PPT+源码等]精品基于Uniapp+SSM实现的校园心理健康APP[包运行成功]Android毕业设计Java项目源码论文
  • 原文地址：https://blog.csdn.net/weixin_40496191/article/details/132650865