-
linux zookeeper kafka_2.12-2.2.0 集群
预先准备：
上传软件至服务器192.168.105.125、192.168.105.129、192.168.130
kafka_2.12-2.2.0、zookeeper-3.4.8.tar.gz一、zookeeper服务搭建
1. 下载
同时在125、129、130节点依次执行
mkdir /app cd /app wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.8/zookeeper-3.4.8.tar.gz- 1
- 2
- 3
2. 解压
tar -zxvf zookeeper-3.4.8.tar.gz- 1
3. 创建目录
cd /app/zookeeper-3.4.8 mkdir data logs- 1
- 2
4. 调整配置
cp conf/zoo_sample.cfg conf/zoo.cfg vim conf/zoo.cfg- 1
- 2
第一处：配置dataDir
dataDir=/app/zookeeper-3.4.8/data- 1
第二处：新增配置
server.1=192.168.105.125:2288:3388 server.2=192.168.105.129:2288:3388 server.3=192.168.105.130:2288:3388- 1
- 2
- 3
5. 配置myid
- 125节点
vim /app/zookeeper-3.4.8/data/myid- 1
内容：1
- 129节点
vim /app/zookeeper-3.4.8/data/myid- 1
内容：2
- 130节点
vim /app/zookeeper-3.4.8/data/myid- 1
内容：3
6. 开放防火墙
125/129/130节点依次执行
firewall-cmd --zone=public --add-port=2181/tcp --permanent firewall-cmd --zone=public --add-port=2288/tcp --permanent firewall-cmd --zone=public --add-port=3388/tcp --permanent firewall-cmd --reload firewall-cmd --list-ports- 1
- 2
- 3
- 4
- 5
7.启动验证zk
/app/zookeeper-3.4.8/bin/zkServer.sh start /app/zookeeper-3.4.8/conf/zoo.cfg /app/zookeeper-3.4.8/bin/zkServer.sh status /app/zookeeper-3.4.8/conf/zoo.cfg- 1
- 2
二、kafka集群搭建
同时在125、129、130节点依次执行
2.1. 下载软件
wget https://archive.apache.org/dist/kafka/2.2.0/kafka_2.12-2.2.0.tgz- 1
2.2. 解压
tar -zxvf kafka_2.12-2.2.0.tgz- 1
2.3. 配置
cd /app/kafka_2.12-2.2.0/config cp server.properties server.properties.bak- 1
- 2
- 125节点
vim server.properties- 1
内容如下：
broker.id=1 listeners=PLAINTEXT://192.168.105.125:9092 advertised.listeners=PLAINTEXT://192.168.105.125:9092 port=9092 host.name=192.168.105.125 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/app/kafka/data num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=3 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=24 log.retention.bytes=1073741824 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=192.168.105.125:2181,192.168.105.129:2181,192.168.105.130:2181 zookeeper.connection.timeout.ms=6000 auto.create.topics.enable = false delete.topic.enable=true message.max.byte=52428880 log.cleanup.policy=delete log.segment.delete.delay.ms=0 group.initial.rebalance.delay.ms=0- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 129节点
vim server.properties- 1
内容如下：
broker.id=2 listeners=PLAINTEXT://192.168.105.129:9092 advertised.listeners=PLAINTEXT://192.168.105.129:9092 port=9092 host.name=192.168.105.125 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/app/kafka/data num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=3 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=24 log.retention.bytes=1073741824 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=192.168.105.125:2181,192.168.105.129:2181,192.168.105.130:2181 zookeeper.connection.timeout.ms=6000 auto.create.topics.enable = false delete.topic.enable=true message.max.byte=52428880 log.cleanup.policy=delete log.segment.delete.delay.ms=0 group.initial.rebalance.delay.ms=0- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 130节点
vim server.properties- 1
内容如下：
broker.id=3 listeners=PLAINTEXT://192.168.105.130:9092 advertised.listeners=PLAINTEXT://192.168.105.130:9092 port=9092 host.name=192.168.105.130 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/app/kafka/data num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=3 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=24 log.retention.bytes=1073741824 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=192.168.105.125:2181,192.168.105.129:2181,192.168.105.130:2181 zookeeper.connection.timeout.ms=6000 auto.create.topics.enable = false delete.topic.enable=true message.max.byte=52428880 log.cleanup.policy=delete log.segment.delete.delay.ms=0 group.initial.rebalance.delay.ms=0 `` ###### 2.4. 防火墙 125/129/130节点依次执行 ```bash firewall-cmd --zone=public --add-port=9092/tcp --permanent firewall-cmd --reload firewall-cmd --list-ports- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
2.5. 启动kafka
启动kafka（第一次前台）
cd /app/kafka_2.12-2.2.0 ./bin/kafka-server-start.sh config/server.properties- 1
- 2
启动kafka（前台）
cd /app/kafka_2.12-2.2.0 ./bin/kafka-server-start.sh -daemon config/server.properties- 1
- 2
125节点
129节点
130节点
三、测试验证
3.1. 创建一个主题
使用kafka，创建一个主题（topic）
bin/kafka-topics.sh --create --zookeeper 192.168.105.125:2181,192.168.105.129:2181,192.168.105.130:2181 --replication-factor 1 --partitions 1 --topic pis-business- 1
3.2. 发送消息
bin/kafka-console-producer.sh --broker-list 192.168.105.125:9092 --topic pis-business- 1
3.3. 消费消息
bin/kafka-console-consumer.sh --bootstrap-server 192.168.105.125:9092 --from-beginning --topic pis-business- 1
-
相关阅读:
【牛客刷题-SQL大厂面试真题】NO1.某音短视频
Dockerfile自定义镜像以及案例分析
为什么图片传输要使用base64编码
【疯狂世界杯】css 动画实现跳动的足球
Jira Software Enterprise Crack
101. 对称二叉树
爬虫是什么？爬虫的原理及应用
一文速学-Base64算法及编解码方法+Python代码
使用C语言实现单链表的功能（带头节点）
二进制信号量
- 原文地址：https://blog.csdn.net/weixin_40816738/article/details/128023679
