• 微服务架构师封神之路13-RabbitMQ集群与高可用|RabbitMQ clustering and HA

    目录

    几个关键技术点

    节点间相互验证，.erlang.cookie

    nodename的唯一性

    port冲突与配置

    Management UI plugin安装

    Queue（message）replicas

    Queue leader strategy

    配置文件详细

    Node 1

    Installation path

    .erlang.cookie

    rabbitmq-env.conf

    rabbitmq.conf

    enabled_plugins

    Node 2

     Installation path

    .erlang.cookie

    rabbitmq-env.conf

    rabbitmq.conf

    enabled_plugins

    Node 3

     Installation path

    .erlang.cookie

    rabbitmq-env.conf

    rabbitmq.conf

    enabled_plugins

    Springboot连接与编程

    application.yml

    Queue and AnonymousQueue

    Queue

    AnonymousQueue

    本文利用一台Linux示范三节点RabbitMQ集群。RabbitMQ版本：3.12.11.

    重点：

    1. 关键技术点
      1. 节点间相互验证，.erlang.cookie
      2. nodename的唯一性
      3. port冲突与配置
      4. Management UI plugin安装
      5. Queue replicas
      6. Queue leader strategy
    2. 配置文件详细
    3. Springboot连接与编程

    几个关键技术点

    节点间相互验证，.erlang.cookie

    .erlang.cookie是erlang框架实现分布式的必要文件。三节点集群需要每个节点的.erlang.cookie文件相同。文件的位置：$home/.erlang.cookie.

    erlang安装之后文件就已经存在，如果使用三台不同的机器，需要将其中一台的文件拷贝三份。用一台机器做示范，我们不需要再做额外的动作。

    nodename的唯一性

    RabbitMQ的nodename有固定的格式@。hostname可以通过操作系统的hostname命令获得。prefix可以自己决定。但是集群中的nodename必须是唯一的。

    如果是部署在不同的机器上，所有的hostname需要明确配置在/etc/hosts文件中。

    nodename配置在rabbitmq-env.conf文件中。

    port冲突与配置

    部署在同一台机器上，需要解决三个端口的冲突。

    1. amqp通信端口5672 - 相关配置文件rabbitmq.conf or rabbitmq-env.conf
    2. management UI端口15672 - 相关配置文件rabbitmq.conf
    3. 内部管理Command Line Interface端口25672 - 相关配置文件rabbitmq-env.conf

    Management UI plugin安装

    信息要放在enabled_plugin文件中。这个文件要手工添加，插件在RabbitMQ第一次启动的时候初始化。

    Queue（message）replicas

    高可用需要同一条消息要同时复制到三个节点上面，这样应用连接的当前节点下线后，还可以从其它另外两个节点读取还未消费完的消息。没有replica，虽然Rabbitmq依然可用，但会有消息丢失的情况。

    要实现消息node间复制，需要创建Queue的时候, queue type设置成Quorum。

    两种方式：

    • 通过管理界面创建Queue

    • Java应用在启动的时候配置Queue
    1.         Map QUEUE_ARGS = new HashMap();    
    2.         QUEUE_ARGS.put("x-queue-type", "quorum");
    3.  
    4.         new Queue("queue_name", true, false, false, QUEUE_ARGS);

    对于匿名Queue不能配置这个属性，当一个node down以后，应用会在新的节点重新创建临时Queue。可能产生message丢失的情况。

    Queue leader strategy

    在rabbitmq.conf中配置，两种client-local（by default）和balanced.

    配置文件详细

    Download rabbitmq package, rabbitmq-server-generic-unix-3.12.11.tar.xz. 解压后拷贝三份，作为三个node。

    下面的文件除了.erlang.cookie都需要手动添加。

    Node 1

    Installation path

    /opt/rabbitmq/rabbitmq-node-1

    .erlang.cookie

    $home/.erlang.cookie

    rabbitmq-env.conf

    File path: /opt/rabbitmq/rabbitmq-node-1/etc/rabbitmq/rabbitmq-env.conf

    1. # Example rabbitmq-env.conf file entries. Note that the variables
    2. # do not have the RABBITMQ_ prefix.
    3. #
    4. # Overrides node name
    5. RABBITMQ_NODENAME=rabbitmq-node-1@DESKTOP-B5WANG
    6.  
    7. # Specifies new style config file location
    8. RABBITMQ_CONFIG_FILE=/opt/rabbitmq/rabbitmq-node-1/etc/rabbitmq/rabbitmq.conf
    9.  
    10. # Specifies advanced config file location
    11. # RABBITMQ_ADVANCED_CONFIG_FILE=/etc/rabbitmq/advanced.config
    12.  
    13. # CLI port
    14. RABBITMQ_DIST_PORT=25672

    rabbitmq.conf

    File path: /opt/rabbitmq/rabbitmq-node-1/etc/rabbitmq/rabbitmq.conf

    1. # port
    2. listeners.tcp.default = 5672
    3. # listeners.tcp.1 = 0.0.0.0:5672
    4.  
    5. # default user
    6. default_vhost = /
    7. default_user = admin
    8. default_pass = admin
    9. default_permissions.configure = .*
    10. default_permissions.read = .*
    11. default_permissions.write = .*
    12. default_user_tags.administrator = true
    13. default_user_tags.management = true
    14. default_user_tags.custom_tag = true
    15.  
    16. # management plugin
    17. management.tcp.port = 15672
    18. management.tcp.ip   = 0.0.0.0
    19.  
    20. # cluster
    21. cluster_name = rabbitmq-cluster
    22. cluster_keepalive_interval = 10000
    23. cluster_formation.peer_discovery_backend = classic_config
    24. cluster_formation.classic_config.nodes.1 = rabbitmq-node-1@DESKTOP-B5WANG
    25. cluster_formation.classic_config.nodes.2 = rabbitmq-node-2@DESKTOP-B5WANG
    26. cluster_formation.classic_config.nodes.3 = rabbitmq-node-3@DESKTOP-B5WANG
    27.  
    28. # queue location strategy
    29. queue_master_locator = client-local

    enabled_plugins

    File path: /opt/rabbitmq/rabbitmq-node-1/etc/rabbitmq/enabled_plugins

    [rabbitmq_management].

    Node 2

     Installation path

    /opt/rabbitmq/rabbitmq-node-2

    .erlang.cookie

    $home/.erlang.cookie

    rabbitmq-env.conf

    File path: /opt/rabbitmq/rabbitmq-node-2/etc/rabbitmq/rabbitmq-env.conf

    1. # Example rabbitmq-env.conf file entries. Note that the variables
    2. # do not have the RABBITMQ_ prefix.
    3. #
    4. # Overrides node name
    5. RABBITMQ_NODENAME=rabbitmq-node-2@DESKTOP-B5WANG
    6.  
    7. # Specifies new style config file location
    8. RABBITMQ_CONFIG_FILE=/opt/rabbitmq/rabbitmq-node-2/etc/rabbitmq/rabbitmq.conf
    9.  
    10. # Specifies advanced config file location
    11. # RABBITMQ_ADVANCED_CONFIG_FILE=/etc/rabbitmq/advanced.config
    12.  
    13. # CLI port
    14. RABBITMQ_DIST_PORT=25673

    rabbitmq.conf

    File path: /opt/rabbitmq/rabbitmq-node-2/etc/rabbitmq/rabbitmq.conf

    1. # port
    2. listeners.tcp.default = 5673
    3. # listeners.tcp.1 = 0.0.0.0:5673
    4.  
    5. # default user
    6. default_vhost = /
    7. default_user = admin
    8. default_pass = admin
    9. default_permissions.configure = .*
    10. default_permissions.read = .*
    11. default_permissions.write = .*
    12. default_user_tags.administrator = true
    13. default_user_tags.management = true
    14. default_user_tags.custom_tag = true
    15.  
    16. # management plugin
    17. management.tcp.port = 15673
    18. management.tcp.ip   = 0.0.0.0
    19.  
    20. # cluster
    21. cluster_name = rabbitmq-cluster
    22. cluster_keepalive_interval = 10000
    23. cluster_formation.peer_discovery_backend = classic_config
    24. cluster_formation.classic_config.nodes.1 = rabbitmq-node-1@DESKTOP-B5WANG
    25. cluster_formation.classic_config.nodes.2 = rabbitmq-node-2@DESKTOP-B5WANG
    26. cluster_formation.classic_config.nodes.3 = rabbitmq-node-3@DESKTOP-B5WANG
    27.  
    28. # queue location strategy
    29. queue_master_locator = client-local

    enabled_plugins

    File path: /opt/rabbitmq/rabbitmq-node-2/etc/rabbitmq/enabled_plugins

    [rabbitmq_management].

    Node 3

     Installation path

    /opt/rabbitmq/rabbitmq-node-3

    .erlang.cookie

    $home/.erlang.cookie

    rabbitmq-env.conf

    File path: /opt/rabbitmq/rabbitmq-node-3/etc/rabbitmq/rabbitmq-env.conf

    1. # Example rabbitmq-env.conf file entries. Note that the variables
    2. # do not have the RABBITMQ_ prefix.
    3. #
    4. # Overrides node name
    5. RABBITMQ_NODENAME=rabbitmq-node-3@DESKTOP-B5WANG
    6.  
    7. # Specifies new style config file location
    8. RABBITMQ_CONFIG_FILE=/opt/rabbitmq/rabbitmq-node-3/etc/rabbitmq/rabbitmq.conf
    9.  
    10. # Specifies advanced config file location
    11. # RABBITMQ_ADVANCED_CONFIG_FILE=/etc/rabbitmq/advanced.config
    12.  
    13. # CLI port
    14. RABBITMQ_DIST_PORT=25674

    rabbitmq.conf

    File path: /opt/rabbitmq/rabbitmq-node-3/etc/rabbitmq/rabbitmq.conf

    1. # port
    2. listeners.tcp.default = 5674
    3. # listeners.tcp.1 = 0.0.0.0:5674
    4.  
    5. # default user
    6. default_vhost = /
    7. default_user = admin
    8. default_pass = admin
    9. default_permissions.configure = .*
    10. default_permissions.read = .*
    11. default_permissions.write = .*
    12. default_user_tags.administrator = true
    13. default_user_tags.management = true
    14. default_user_tags.custom_tag = true
    15.  
    16. # management plugin
    17. management.tcp.port = 15674
    18. management.tcp.ip   = 0.0.0.0
    19.  
    20. # cluster
    21. cluster_name = rabbitmq-cluster
    22. cluster_keepalive_interval = 10000
    23. cluster_formation.peer_discovery_backend = classic_config
    24. cluster_formation.classic_config.nodes.1 = rabbitmq-node-1@DESKTOP-B5WANG
    25. cluster_formation.classic_config.nodes.2 = rabbitmq-node-2@DESKTOP-B5WANG
    26. cluster_formation.classic_config.nodes.3 = rabbitmq-node-3@DESKTOP-B5WANG
    27.  
    28. # queue location strategy
    29. queue_master_locator = client-local

    enabled_plugins

    File path: /opt/rabbitmq/rabbitmq-node-3/etc/rabbitmq/enabled_plugins

    [rabbitmq_management].

    Springboot连接与编程

    application.yml

    1. spring:
    2.   rabbitmq:
    3.     # Connect rabbitmq cluster
    4.     addresses: localhost:5672,localhost:5673,localhost:5674
    5.     username: admin
    6.     password: admin

    Queue and AnonymousQueue

    Queue

    1.         Map arg = new HashMap();
    2.         arg.put("x-queue-type", "quorum");
    3.         arg.put("x-queue-leader-locator", "client-local");
    4.         new Queue(QUEUE_NAME_TEXT_MESSAGE, true, false, false, arg);

    AnonymousQueue

    AnonymousQueue aq = new AnonymousQueue(new Base64UrlNamingStrategy("queue-prefix-"));

    AnonymousQueue本质上还是Queue，只是属性不同。所以不需要配置quorum，否则应用会报错。

    1. public class AnonymousQueue extends Queue {
    2.     public AnonymousQueue() {
    3.         this((Map)null);
    4.     }
    5.  
    6.     public AnonymousQueue(Map arguments) {
    7.         this(Base64UrlNamingStrategy.DEFAULT, arguments);
    8.     }
    9.  
    10.     public AnonymousQueue(NamingStrategy namingStrategy) {
    11.         this(namingStrategy, (Map)null);
    12.     }
    13.  
    14.     public AnonymousQueue(NamingStrategy namingStrategy, Map arguments) {
    15.         super(namingStrategy.generateName(), false, true, true, arguments);
    16.         if (!this.getArguments().containsKey("x-queue-master-locator")) {
    17.             this.setLeaderLocator("client-local");
    18.         }
    19.  
    20.     }
    21. }

  • 相关阅读:
    精品基于NET实现的数码新闻发布系统
    数据平滑处理-均值|中值|Savitzky-Golay滤波器
    剑指 Offer 44. 数字序列中某一位的数字
    TensorBoard的使用2（add_image函数）
    docker安装rocketMQ并测试
    【Java】实现文件夹复制
    Linux学习笔记(12) -- 登录、注销及关机
    字符串——重复的子字符串
    小程序与公众号：编程的异同
    微信小程序低功耗蓝牙BLE快速开发js
  • 原文地址：https://blog.csdn.net/b5wang/article/details/136229195