• Elasticsearch (一) 基于Docker-compose 搭建集群

    基于Docker-compose 搭建Elasticsearch集群

    1.前言

    基于docker-compose 安装 elasticsearch 集群，kibana可视化组件， 通过cerebro工具监控集群信息

    Elasticsearch 官网：https://www.elastic.co/cn/

    Docker

    安装

    yum install -y docker

    • 1

    启动

    systemctl start docker

    • 1

    测试

    docker --version

    • 1

    Compose

    Docker Compose是一个用来定义和运行复杂应用的Docker工具。一个使用Docker容器的应用，通常由多个容器组成。使用Docker Compose不再需要使用shell脚本来启动容器。
    Compose 通过一个配置文件来管理多个Docker容器，在配置文件中，所有的容器通过services来定义，然后使用docker-compose脚本来启动，停止和重启应用，和应用中的服务以及所有依赖服务的容器，非常适合组合使用多个容器进行开发的场景。

    安装

     sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

    • 1

    对二进制文件应用可执行权限：

    sudo chmod +x /usr/local/bin/docker-compose

    • 1

    测试

    docker-compose --version

    • 1

    Compose和Docker兼容性

    compose文件格式版本 docker版本
    3.4 17.09.0+
    3.3 17.06.0+
    3.2 17.04.0+
    3.1 1.13.1+
    3.0 1.13.0+
    2.3 17.06.0+
    2.2 1.13.0+
    2.1 1.12.0+
    2.0 1.10.0+
    1.0 1.9.1.+

    配置文件

    官方文档

    创建 docker-compose-elasticsearch.yaml

    version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/config/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/config/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - data02:/usr/share/elasticsearch/data
    networks:
      - elastic
  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/config/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - data03:/usr/share/elasticsearch/data
    networks:
      - elastic
  kibana:
    image: docker.elastic.co/kibana/kibana:7.15.2
    container_name: kibana
    volumes:
      - ./kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
    ports:
      - 5601:5601
    environment:
      ELASTICSEARCH_URL: http://es01:9200
      ELASTICSEARCH_HOSTS: http://es01:9200
    depends_on:
      - es01
    networks:
      - elastic
volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89

    elasticsearch/config/elasticsearch.yml

    network.host: 0.0.0.0
http.port: 9200
# 开启es跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
# 开启安全控制
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/elastic-certificates.p12

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12

    kibana/config/kibana.yml

    server.name: kibana
server.host: "0.0.0.0"
xpack.monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.username: "elastic"  # es账号
elasticsearch.password: "*******"   # es密码 进入容器设置的密码
i18n.locale: zh-CN # 中文

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    上传并运行

    上传文件 docker-compose-elasticsearch.yaml 到自己创建的目录
    在这里插入图片描述

    elasticsearch 安全策略

    生成证书
    依次执行命令
    1.创建临时容器
    2.进入容器
    3.创建ca [直接回车不用输入密码]
    4.创建证书 [直接回车不用输入密码]
    5.退出容器 并将容器中的证书拷贝出来
    6.删除这个临时容器

    docker run -d  docker.elastic.co/elasticsearch/elasticsearch:7.15.2 --name=es
docker exec -it es /bin/bash
./bin/elasticsearch-certutil ca
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
docker cp es:/usr/share/elasticsearch/elastic-certificates.p12 .
docker rm -f es

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    如果创建证书的时候输入了密码 需要在容器内执行否则这个证书在集群启动的时候会认证不通过。
    ./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
    ./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

    启动所有容器

    docker-compose -f docker-compose-elasticsearch.yaml up -d

    进入容器修改密码

    docker exec -it es01 /bin/bash

    [root@localhost elasticsearch-cluster]# docker exec -it es01 /bin/bash
bash-4.4# ./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana_system]: 
Reenter password for [kibana_system]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

通过命令 docker-compose -f docker-compose-elasticsearch.yaml up -d 运行

```bash
[root@localhost elasticsearch-cluster]# docker-compose -f docker-compose-elasticsearch.yaml up -d
Creating network "elasticsearch-cluster_elastic" with driver "bridge"
Creating volume "elasticsearch-cluster_data01" with local driver
Creating volume "elasticsearch-cluster_data02" with local driver
Creating volume "elasticsearch-cluster_data03" with local driver
Pulling es01 (docker.elastic.co/elasticsearch/elasticsearch:7.15.2)...
Trying to pull repository docker.elastic.co/elasticsearch/elasticsearch ... 
7.15.2: Pulling from docker.elastic.co/elasticsearch/elasticsearch
009c11f4ddee: Pull complete
8772b99d888d: Pull complete
bd8b744bf3bf: Pull complete
2a41be2c565a: Pull complete
e7e9200dd33e: Pull complete
Digest: sha256:a1dce08d504b22e87adc849c94dcae53f6a0bd12648a4d99d7f9fc07bb2e8a3e
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.15.2
Creating es02 ... done
Creating es01 ... done
Creating es03 ... done

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48

    通过 docker ps 命令查看运行中的容器

    [root@localhost elasticsearch-cluster]# docker ps
CONTAINER ID        IMAGE                                                  COMMAND                  CREATED             STATUS              PORTS                              NAMES
f43b017dd23a        docker.elastic.co/elasticsearch/elasticsearch:7.15.2   "/bin/tini -- /usr..."   17 seconds ago      Up 12 seconds       9200/tcp, 9300/tcp                 es03
7ed565d7eb4e        docker.elastic.co/elasticsearch/elasticsearch:7.15.2   "/bin/tini -- /usr..."   17 seconds ago      Up 12 seconds       0.0.0.0:9200->9200/tcp, 9300/tcp   es01
fb89e106eea2        docker.elastic.co/elasticsearch/elasticsearch:7.15.2   "/bin/tini -- /usr..."   17 seconds ago      Up 12 seconds       9200/tcp, 9300/tcp                 es02

    • 1
    • 2
    • 3
    • 4
    • 5

    通过 docker logs -f es01 查看容器的运行日志

    [root@localhost elasticsearch-cluster]# docker logs -f es01
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.elasticsearch.bootstrap.Elasticsearch (file:/usr/share/elasticsearch/lib/elasticsearch-7.15.2.jar)
WARNING: Please consider reporting this to the maintainers of org.elasticsearch.bootstrap.Elasticsearch
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.elasticsearch.bootstrap.Security (file:/usr/share/elasticsearch/lib/elasticsearch-7.15.2.jar)
WARNING: Please consider reporting this to the maintainers of org.elasticsearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
{"type": "server", "timestamp": "2021-12-30T04:49:03,608Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "version[7.15.2], pid[6], build[default/docker/93d5a7f6192e8a1a12e154a2b81bf6fa7309da0c/2021-11-04T14:04:42.515624022Z], OS[Linux/3.10.0-1160.el7.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.1/17.0.1+12]" }
..........
..........
..........

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13

    开放防火墙端口：9200，9300

    [root@localhost elasticsearch-cluster]# firewall-cmd --zone=public --add-port=9200/tcp --add-port=9300/tcp --permanent&&firewall-cmd --reload
success
success
[root@localhost elasticsearch-cluster]# firewall-cmd --list-ports
6379/tcp 9200/tcp 9300/tcp
[root@localhost elasticsearch-cluster]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    访问Kibana
    在这里插入图片描述
    在这里插入图片描述

    查看节点

    [root@localhost elasticsearch-cluster]# curl -X GET "localhost:9200/_cat/nodes?v=true&pretty"
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
172.18.0.4           64          89   6    0.11    0.18     0.19 cdfhilmrstw *      es02
172.18.0.3           68          89   6    0.11    0.18     0.19 cdfhilmrstw -      es01
172.18.0.2           53          89   6    0.11    0.18     0.19 cdfhilmrstw -      es03
[root@localhost elasticsearch-cluster]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    运行Cerebro

    下载地址：https://github.com/lmenezes/cerebro/releases
    在这里插入图片描述

    解压运行：cerebro.bat
    运行之前先修改conf/application.conf 中的es 密码

    在这里插入图片描述
    双击cerebro.bat 运行
    在这里插入图片描述
    在这里插入图片描述
    在这里插入图片描述

    在这里插入图片描述

  • 相关阅读:
    关于idea 右键找不到Diagrams 按钮（类的继承关系图）
    Spring_第3章_AOP+事务
    RabbitMQ原理(二)：SpringAMQP编程
    信息系统项目管理师---第七章项目成本管理历年考题
    charles劫持修改js文件
    Java 字符串拼接原理分析
    dart 学习 之 在 构造方法中使用 this
    第二部分 Makefile 总述
    基于Dockerfile搭建LNMP
    Linux性能优化-网络篇-DNS问题排查
  • 原文地址：https://blog.csdn.net/m0_67392931/article/details/126358456