• clickhouse 单节点多实例部署


    环境说明:

    主机名:cmcc01为例 

    操作系统:centos7

    安装部署软件版本部署方式
    centos7
    zookeeperzookeeper-3.4.10伪分布式
    hadoophadoop-3.1.3伪分布式
    hivehive-3.1.3-bin伪分布式
    clickhouse21.11.10.1-2单节点多实例
    dolphinscheduler3.0.0单节点
    kettlepdi-ce-9.3.0.0单节点
    sqoopsqoop-1.4.7单节点
    seatunnelseatunnel-incubating-2.1.2单节点
    sparkspark-2.4.8单节点

    部署服务:

           clickhouserver

           clickhouserver2

    单节点服务多实例部署

    1.下载并上传clickhouse 至服务节点

      官网:https://packages.clickhouse.com/rpm/stable/

     2.安装clickhouse:

    1. # 安装必要环境
    2. yum install -y yum-utils expect autoconf
    3. # 安装clickhouse
    4. rpm -ivh clickhouse-*

       安装过程中会提示输入密码,需要记住

    3.修改配置文件: vim /etc/clickhouse-server/config.xml

      提示前面数字为行号,非配置信息

    1. # 修改配置
    2. 104 <tcp_port>9998tcp_port>
    3. # 添加主机配置,注意:cmcc01 为我的hostname,此处填写你的
    4. 158 <interserver_http_host>cmcc01interserver_http_host>
    5. # 添加监听配置,运行远程登录
    6. 180 <listen_host>::listen_host>
    7. # 添加集群配置
    8. 720 <shard2_replica1>
    9. 721 <shard>
    10. 722 <internal_replication>trueinternal_replication>
    11. 723 <replica>
    12. 724 <host>cmcc01host>
    13. 725 <port>9998port>
    14. 726 <user>defaultuser>
    15. 727 <password>123qwepassword>
    16. 728 replica>
    17. 729 shard>
    18. 730 <shard>
    19. 731 <internal_replication>trueinternal_replication>
    20. 732 <replica>
    21. 733 <host>cmcc01host>
    22. 734 <port>9999port>
    23. 735 <user>defaultuser>
    24. 736 <password>123qwepassword>
    25. 737 replica>
    26. 738 shard>
    27. 739 shard2_replica1>
    28. # 添加zookeeper配置
    29. 786 <zookeeper>
    30. 787 <node>
    31. 788 <host>cmcc01host>
    32. 789 <port>2181port>
    33. 790 node>
    34. 791 <node>
    35. 792 <host>cmcc01host>
    36. 793 <port>2182port>
    37. 794 node>
    38. 795 <node>
    39. 796 <host>cmcc01host>
    40. 797 <port>2183port>
    41. 798 node>
    42. 799 zookeeper>
    43. # 配置层次分片副本
    44. 813 <macros>
    45. 814 <layer>01layer>
    46. 815 <shard>01shard>
    47. 816 <replica>shard2_replica1-01replica>
    48. 817 macros>

     

     

    4.复制配置文件

    cp /etc/clickhouse-server/config.xml /etc/clickhouse-server/config2.xml
    

     修改配置信息

    1. vim /etc/clickhouse-server/config2.xml
    2. # 修改以下内容
    3. <log>/var/log/clickhouse-server/clickhouse-server2.loglog>
    4. <errorlog>/var/log/clickhouse-server/clickhouse-server2.err.logerrorlog>
    5. <http_port>8124http_port>
    6. <tcp_port>9999tcp_port>
    7. <mysql_port>9014mysql_port>
    8. <postgresql_port>9015postgresql_port>
    9. <interserver_http_port>9019interserver_http_port>
    10. <path>/var/lib/clickhouse2/path>
    11. <tmp_path>/var/lib/clickhouse2/tmp/tmp_path>
    12. <user_files_path>/var/lib/clickhouse2/user_files/user_files_path>
    13. <local_directory>
    14. <path>/var/lib/clickhouse2/access/path>
    15. local_directory>
    16. <macros>
    17. <layer>01layer>
    18. <shard>02shard>
    19. <replica>shard2_replica1-01replica>
    20. macros>

     提示:我的用户都设置为一样所有公用一个配置文件,如需每个主机设置不同的用户需要修改配置,并cp的user.xml 

    1. <users_xml>
    2. <path>users2.xmlpath>
    3. users_xml>

     

     

     5.复制启动服务文件

    1. # 复制启动文件
    2. cp /etc/systemd/system/clickhouse-server.service /etc/systemd/system/clickhouse-server2.service
    3. # 修改启动文件
    4. vi /etc/systemd/system/clickhouse-server2.service
    5. ExecStart=/usr/bin/clickhouse-server --config=/etc/clickhouse-server/config2.xml --pid-file=/run/clickhouse-server/clickhouse-server2.pid

     6.启动服务

    1. # 复制文件
    2. cp /var/log/clickhouse-server/clickhouse-server.log /var/log/clickhouse-server/clickhouse-server2.log
    3. cp /var/log/clickhouse-server/clickhouse-server.err.log /var/log/clickhouse-server/clickhouse-server2.err.log
    4. cp -r /var/lib/clickhouse /var/lib/clickhouse2
    5. # 修改所属用户
    6. chown clickhouse:clickhouse /var/log/clickhouse-server/clickhouse-server2.log
    7. chown clickhouse:clickhouse /var/log/clickhouse-server/clickhouse-server2.err.log
    8. chown clickhouse:clickhouse -R /var/lib/clickhouse2
    9. # 启动服务
    10. systemctl start clickhouse-server
    11. systemctl start clickhouse-server2
    12. # 查看服务状态
    13. systemctl status clickhouse-server
    14. systemctl status clickhouse-server2

    注意:如启动报错,或者启动不了服务先检查以下文件权限是否正确,所有文件的所属用户是否为clickhouse

    /var/log/clickhouse-server/clickhouse-server2.log
    /var/log/clickhouse-server/clickhouse-server2.err.log
    /var/lib/clickhouse2
    /etc/clickhouse-server/config2.xml

    权限参照原目录文件,所属用户必须要clickhouse

    启动成功后的状态 

    7.登录clickhouse

    1. # 登录服务1 clickhouse
    2. clickhouse-client --host cmcc01 --port 9998 -u default --password 123qwe --multiline
    3. # 创建测试数据库
    4. create database test on CLUSTER shard2_replica1;
    5. # 创建测试本地表
    6. CREATE TABLE test.student_test on CLUSTER shard2_replica1
    7. (
    8. id int,
    9. name String,
    10. age int,
    11. grade String
    12. )
    13. ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/test/student_test/{layer}-{shard}', '{replica}')
    14. PARTITION BY grade
    15. ORDER BY id
    16. SETTINGS index_granularity = 8192;
    17. # 创建分布式表
    18. CREATE TABLE test.student_test_all on CLUSTER shard2_replica1 as test.student_test ENGINE = Distributed('shard2_replica1', 'test', 'student_test', rand());
    19. # 插入数据
    20. insert into test.student_test values (1, '李四', 21, '1班'),(2, '王五', 21, '1班')(3, '张三', 21, '2班');
    21. # 查询本地表
    22. cmcc01.hand.com :) select * from test.student_test;
    23. SELECT *
    24. FROM test.student_test
    25. Query id: 04702719-3beb-4a9b-8ba7-33e77f0e5961
    26. ┌─id─┬─name─┬─age─┬─grade─┐
    27. │ 1 │ 李四 │ 21 │ 1班 │
    28. │ 2 │ 王五 │ 21 │ 1班 │
    29. └────┴──────┴─────┴───────┘
    30. ┌─id─┬─name─┬─age─┬─grade─┐
    31. │ 3 │ 张三 │ 21 │ 2班 │
    32. └────┴──────┴─────┴───────┘
    33. 3 rows in set. Elapsed: 0.005 sec.
    34. cmcc01.hand.com :)
    35. # 查询分布式表
    36. select * from test.student_test_all;
    37. # 登录服务2 clickhouse
    38. clickhouse-client --host cmcc01 --port 9999 -u default --password 123qwe --multiline
    39. # 查询本地表,正常是没有数据的
    40. select * from test.student_test;
    41. # 查询分布式表,是有数据的和 9998的保持一致
    42. select * from test.student_test_all;

    至此安装部署测试完成

  • 相关阅读:
    Leetcode 15:三数之和
    CompletableFuture-FutureTask结合线程池提升性能
    xxl-job中多节点分片的时候如何在linux服务器开启多个执行器实例?
    Python之文件 打开与关闭
    java计算机毕业设计风情旅游网站源码+mysql数据库+系统+lw文档+部署
    linux系统环境下mysql安装和基本命令学习
    爬某网站延禧宫率第一集视频
    Linux易混淆知识点
    HTTP代理与SOCKS5代理,有什么区别?
    SpringBoot之Swagger
  • 原文地址:https://blog.csdn.net/QYmufeng/article/details/125908506