• 【Kingbase FlySync】命令模式:安装部署同步软件，实现KES到KES实现同步

    概述

    • Kingbase FlySync是面向同城/异地灾备、数据库平滑升级替换、数据集中共享与分发、应用上云迁移、数据库负载均衡等场景的数据同步产品。该产品基于增量日志解析技术，性能高、时延低、资源占用极少，能够实现异构数据源之间大规模增量数据的任意方向流转和实时同步，帮助用户打破数据孤岛，轻松共享分发数据。Kingbase FlySync支持丰富的软硬件平台和数据源，同步拓扑可以一对一、一对多、多对一、级联等方式任意组合延伸，并可在数据同步过程中保证端到端的事务级数据完整性和高可用性。

    准备环境

    • 两台虚拟机
    虚拟机IP数据库软件版本角色操作系统
    K2node1192.168.40.111(Kingbase) KingbaseES V8R6主库CenterOS7.2
    K2node2192.168.40.112(Kingbase) KingbaseES V8R6备库CenterOS7.2
    • 数据库安装要求
    项目node1node2
    数据库版本KingbaseES V8R6KingbaseES V8R6
    安装路径/KingbaseES/V8//KingbaseES/V8/
    data路径/data/data
    端口5432154321
    用户名systemsystem
    密码kingbasekingbase

    目标

    2. 同步程序安装部署
    3. 使用命令方式，完成数据迁移，并保证数据一致。
    4. 按照以下规范完成部署
    类别源端目标端
    replicater安装路径/home/flysync/replicator/home/flysync/replicator
    replicator端口rmi-port：11000/kufl-port:3112rmi-port:11000/kufl-port：3112
    kfs服务名kingbase8_sourcekingbase8_target
    KFS 同步所使用的用户flysync/flysyncflysync/flysync
    映射规则1examexam[0017]
    映射规则2exam.courseexam0017.course0017

    资源

    1.测试虚拟机下载地址包含node1,node2

    • 虚拟机快照都还原至：优化数据库
    • root用户密码为kingbase.123
    • kingbase用户密码为kingbase

    点击下载：百度网盘

    2.同步工具下载地址

    点击下载：百度网盘

    3.临时授权下载地址

    点击下载：百度网盘

    4.ruby工具下载地址

    点击下载：百度网盘

    5.EXAMv0.11.sql下载地址

    点击下载：百度网盘

    实操：同步软件安装部署

    1.node1准备安装环境

    (1)增加flysync 用户并设置密码

    [root@node1 ~]# adduser flysync
[root@node1 ~]# usermod -G kingbase flysync
[root@node1 ~]# echo flysync | passwd --stdin flysync
更改用户 flysync 的密码 。
passwd：所有的身份验证令牌已经成功更新。
[root@node1 ~]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    (2)调整flysync的最大文件句柄数（open files）、允许创建的最大进程数量（max user processes）

    [root@node1 ~]# cat >> /etc/security/limits.conf << EOF
flysync - nofile 65535
flysync - nproc 8096
EOF
[root@node1 ~]# cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#            
#
#Where:
# can be:
#        - a user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
# can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
# can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open file descriptors
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#
#                 
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4

#add by kingbase
kingbase soft  nproc   65536
kingbase  hard  nproc   65536
kingbase  soft  nofile  65536
kingbase  hard  nofile  65536
#kingbase  soft  stack   10240
#kingbase  hard  stack   32768
kingbase soft core unlimited
kingbase hard core unlimited

flysync - nofile 65535
flysync - nproc 8096
[root@node1 ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78

    (3)hosts 文件配置

    [root@node1 ~]# cat >> /etc/hosts << EOF
192.168.40.111 node1
192.168.40.112 node2
EOF
[root@node1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.111 node1
192.168.40.112 node2
[root@node1 ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    (4)network配置

    [root@node1 ~]# cat >> /etc/sysconfig/network << EOF
> NETWORKING = yes
> EOF
[root@node1 ~]# cat /etc/sysconfig/network
# Created by anaconda
NETWORKING = yes
HOSTNAME = node1
[root@node1 ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8

    (5)上传并安装ruby工具

    • 使用FTP工具上传rbenv_2.2.2_linux_x64.zip到/home/flysync目录下
    [root@node1 flysync]# ls -l
总用量 6832
-rw-r--r-- 1 root root 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]# pwd
/home/flysync
[root@node1 flysync]# [root@node1 flysync]# chown flysync:flysync /home/flysync/rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 解压rbenv_2.2.2_linux_x64.zip包
    [root@node1 flysync]su flysync
[flysync@node1 flysync]# unzip -q /home/flysync/rbenv_2.2.2_linux_x64.zip
[flysync@node1 flysync]# ls -l
总用量 6836
-rwxr-xr-x 1 root root     340 11月  4 2019 fspm
drwxrwxr-x 3 flysync flysync      17 11月  4 2019 rbenv
-rw-r--r-- 1 flysync flysync 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 把ruby添加到环境变量
    [flysync@node1 flysync]# vi ~/.bash_profile
[flysync@node1 flysync]# source ~/.bash_profile
[flysync@node1 flysync]# cat ~/.bash_profile 
[flysync@node1 ~]$ cat ~/.bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
	. ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
export PATH

[flysync@node1 flysync]# ruby  -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@node1 flysync]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21

    (6)node1数据库配置

    • 登录数据库
    [kingbase@node1 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node1 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 添加flysync用户并授权
    test=# CREATE USER flysync PASSWORD 'flysync';
CREATE ROLE
test=# ALTER USER flysync REPLICATION;
ALTER ROLE
test=# GRANT ALL PRIVILEGES ON DATABASE TEST TO flysync ;
GRANT
test=# ALTER USER flysync SUPERUSER;
ALTER ROLE
test=# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 配置服务器参数
    test=# ALTER SYSTEM SET wal_level ='logical';
ALTER SYSTEM
test=# ALTER SYSTEM SET wal_keep_segments = 4;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_wal_senders =10;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_replication_slots =10;
ALTER SYSTEM
test=# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 退出数据库并修改数据库认证文件
    test=# \q
[kingbase@node1 bin]$ cat >> /data/sys_hba.conf << EOF
host    replication     all             0.0.0.0/0              scram-sha-256
EOF
[kingbase@node1 bin]$ cat /data/sys_hba.conf
# Kingbase Client Authentication Configuration File
# ===================================================
#
# Refer to the "Client Authentication" section in the Kingbase
# documentation for a complete description of this file.  A short
# synopsis follows.
#
# This file controls: which hosts are allowed to connect, how clients
# are authenticated, which Kingbase user names they can use, which
# databases they can access.  Records take one of these forms:
#
# local      DATABASE  USER  METHOD  [OPTIONS]
# host       DATABASE  USER  ADDRESS  METHOD  [OPTIONS]
# hostssl    DATABASE  USER  ADDRESS  METHOD  [OPTIONS]
# hostnossl  DATABASE  USER  ADDRESS  METHOD  [OPTIONS]
#
# (The uppercase items must be replaced by actual values.)
#
# The first field is the connection type: "local" is a Unix-domain
# socket, "host" is either a plain or SSL-encrypted TCP/IP socket,
# "hostssl" is an SSL-encrypted TCP/IP socket, and "hostnossl" is a
# plain TCP/IP socket.
#
# DATABASE can be "all", "sameuser", "samerole", "replication", a
# database name, or a comma-separated list thereof. The "all"
# keyword does not match "replication". Access to replication
# must be enabled in a separate record (see example below).
#
# USER can be "all", a user name, a group name prefixed with "+", or a
# comma-separated list thereof.  In both the DATABASE and USER fields
# you can also write a file name prefixed with "@" to include names
# from a separate file.
#
# ADDRESS specifies the set of hosts the record matches.  It can be a
# host name, or it is made up of an IP address and a CIDR mask that is
# an integer (between 0 and 32 (IPv4) or 128 (IPv6) inclusive) that
# specifies the number of significant bits in the mask.  A host name
# that starts with a dot (.) matches a suffix of the actual host name.
# Alternatively, you can write an IP address and netmask in separate
# columns to specify the set of hosts.  Instead of a CIDR-address, you
# can write "samehost" to match any of the server's own IP addresses,
# or "samenet" to match any address in any subnet that the server is
# directly connected to.
#
# METHOD can be "trust", "reject", "md5", "password", "scram-sha-256",
# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert".
# Note that "password" sends passwords in clear text; "md5" or
# "scram-sha-256" are preferred since they send encrypted passwords.
#
# OPTIONS are a set of options for the authentication in the format
# NAME=VALUE.  The available options depend on the different
# authentication methods -- refer to the "Client Authentication"
# section in the documentation for a list of which options are
# available for which authentication methods.
#
# Database and user names containing spaces, commas, quotes and other
# special characters must be quoted.  Quoting one of the keywords
# "all", "sameuser", "samerole" or "replication" makes the name lose
# its special character, and just match a database or username with
# that name.
#
# This file is read on server startup and when the server receives a
# SIGHUP signal.  If you edit the file on a running system, you have to
# SIGHUP the server for the changes to take effect, run "sys_ctl reload",
# or execute "SELECT pg_reload_conf()".
#
# Put your actual configuration here
# ----------------------------------
#
# If you want to allow non-local connections, you need to add more
# "host" records.  In that case you will also need to make Kingbase
# listen on a non-local interface via the listen_addresses
# configuration parameter, or via the -i or -h command line switches.

# CAUTION: Configuring the system for local "trust" authentication
# allows any local user to connect as any Kingbase user, including
# the database superuser.  If you do not trust all your local users,
# use another authentication method.


# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            scram-sha-256
host    all             all             0.0.0.0/0               scram-sha-256
# IPv6 local connections:
host    all             all             ::1/128                 scram-sha-256
host    all             all             ::0/0                   scram-sha-256
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     trust
host    replication     all             127.0.0.1/32            scram-sha-256
host    replication     all             ::1/128                 scram-sha-256
host    replication     all             0.0.0.0/32               scram-sha-256
host    replication     all             0.0.0.0/0              scram-sha-256
[kingbase@node1 bin]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94
    • 95
    • 96
    • 97
    • 98
    • 99
    • 100
    • 101
    • 102
    • 103

    (7)重启数据库使配置生效

    [kingbase@node1 bin]$ sys_ctl restart -D /data
waiting for server to shut down.... done
server stopped
waiting for server to start....2023-11-16 03:43:14.182 GMT [5233] 警告:  max_connections should be less than or equal than 10 (restricted by license)
2023-11-16 03:43:14.182 GMT [5233] 提示:  the value of max_connect is set 10
2023-11-16 03:43:14.183 GMT [5233] 警告:  max_connections should be less than or equal than 10 (restricted by license)
2023-11-16 03:43:14.183 GMT [5233] 提示:  the value of max_connect is set 10
2023-11-16 11:43:14.189 CST [5233] 日志:  sepapower extension initialized
2023-11-16 11:43:14.191 CST [5233] 日志:  太多后台工作进程
2023-11-16 11:43:14.191 CST [5233] 详细信息:  当前设置里最多可以注册2个后台工作进程.
2023-11-16 11:43:14.191 CST [5233] 提示:  考虑增大配置参数 "max_worker_processes"的值.
2023-11-16 11:43:14.191 CST [5233] 日志:  太多后台工作进程
2023-11-16 11:43:14.191 CST [5233] 详细信息:  当前设置里最多可以注册2个后台工作进程.
2023-11-16 11:43:14.191 CST [5233] 提示:  考虑增大配置参数 "max_worker_processes"的值.
2023-11-16 11:43:14.192 CST [5233] 日志:  正在启动 KingbaseES V008R006C005B0023 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
2023-11-16 11:43:14.192 CST [5233] 日志:  正在监听IPv4地址"0.0.0.0"，端口 54321
2023-11-16 11:43:14.192 CST [5233] 日志:  正在监听IPv6地址"::"，端口 54321
2023-11-16 11:43:14.193 CST [5233] 日志:  在Unix套接字 "/tmp/.s.KINGBASE.54321"上侦听
2023-11-16 11:43:14.244 CST [5233] 日志:  日志输出重定向到日志收集进程
2023-11-16 11:43:14.244 CST [5233] 提示:  后续的日志输出将出现在目录 "sys_log"中.
 done
server started
[kingbase@node1 bin]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23

    (8)导入测试数据

    [kingbase@node1 install]$ ksql -U flysync -d test -f /install/EXAM_v0.11.sql
ksql:/install/EXAM_v0.11.sql:8: 注意:  串联删除10个其它对象
描述:  递归删除 序列 exam.seq_student_sno
递归删除 表 exam.student
递归删除 表 exam.course
递归删除 表 exam.instructor
递归删除 表 exam.instructor_history
递归删除 表 exam.score
递归删除 视图 exam.v_score_information
递归删除 function exam.get_avg_score(numeric)
递归删除 procedure exam.retire_instructor(numeric)
递归删除 表 exam.tmpbak01
DROP SCHEMA
CREATE SCHEMA
SET
CREATE SEQUENCE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE VIEW
CREATE FUNCTION
CREATE PROCEDURE
CREATE TRIGGER
ksql:/install/EXAM_v0.11.sql:202: 警告:  没有事物在运行中
COMMIT
[kingbase@node1 install]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94

    (9)最后检查下环境

    • 检查机器名
    [kingbase@node1 install]$ exit
登出
[root@node2 flysync]# uname -n
node2


    • 1
    • 2
    • 3
    • 4
    • 5
    • 检查操作系统属性检查及配置
    [root@node1 ~]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 11210
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 8096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 检查IP 地址
    [root@node1 flysync]# hostname --ip-address
192.168.40.111 192.168.40.111
[root@node1 flysync]# 

    • 1
    • 2
    • 3
    • 检查其他机器的 IP 地址是否正确
    [root@node1 ~]$ ping -c 4 node1
PING node1 (192.168.40.111) 56(84) bytes of data.
64 bytes from node1 (192.168.40.111): icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from node1 (192.168.40.111): icmp_seq=2 ttl=64 time=0.110 ms
64 bytes from node1 (192.168.40.111): icmp_seq=3 ttl=64 time=0.036 ms
64 bytes from node1 (192.168.40.111): icmp_seq=4 ttl=64 time=0.037 ms

--- node1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.036/0.054/0.110/0.033 ms
[flysync@node1 ~]$ ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=1.65 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=1.43 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.262 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.301 ms

--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 0.262/0.913/1.656/0.637 ms


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 检查时区
    [root@node1 ~]$ date
2023年 11月 16日 星期四 11:48:34 CST

    • 1
    • 2

    2.node2准备安装环境

    (1)增加flysync 用户并设置密码

    [root@node2 ~]# userdel -r flysync
[root@node2 ~]# adduser flysync
[root@node2 ~]# usermod -G kingbase flysync
[root@node2 ~]# echo flysync |passwd --stdin flysync
更改用户 flysync 的密码 。
passwd：所有的身份验证令牌已经成功更新。
[root@node2 ~]# 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8

    (2)调整flysync的最大文件句柄数（open files）、允许创建的最大进程数量（max user processes）

    [root@node2 ~]# cat >> /etc/security/limits.conf << EOF
flysync - nofile 65535
flysync - nproc 8096
EOF
[root@node2 ~]# cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#            
#
#Where:
# can be:
#        - a user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
# can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
# can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open file descriptors
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#
#                 
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4

# End of file
#add by kingbase
kingbase soft  nproc   65536
kingbase  hard  nproc   65536
kingbase  soft  nofile  65536
kingbase  hard  nofile  65536
kingbase  soft  stack   10240
kingbase  hard  stack   32768
kingbase soft core unlimited
kingbase hard core unlimited
flysync - nofile 65535
flysync - nproc 8096
[root@node2 ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78

    (3)hosts 文件配置

    [root@node2 ~]# cat >> /etc/hosts << EOF
192.168.40.111 node1
192.168.40.112 node2
EOF
[root@node2 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.111 node1
192.168.40.112 node2
[root@node2 ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    (4)network配置

    [root@node1 ~]# cat >> /etc/sysconfig/network << EOF
NETWORKING = yes
EOF
[root@node2 ~]# cat /etc/sysconfig/network
# Created by anaconda
NETWORKING = yes
HOSTNAME = node2
[root@node2 ~]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8

    (5)拷贝并安装ruby工具

    ```bash
[flysync@node2 ~]# scp root@node1:/home/flysync/rbenv_2.2.2_linux_x64.zip /home/flysync/
The authenticity of host 'node1 (192.168.40.111)' can't be established.
ECDSA key fingerprint is b8:54:e0:74:c5:e2:33:46:0d:7d:39:91:e3:b5:5a:6d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node1,192.168.40.111' (ECDSA) to the list of known hosts.
root@node1's password: 
rbenv_2.2.2_linux_x64.zip                                                                                                                                                                                      100% 6830KB   6.7MB/s   00:00    
[flysync@node2 ~]# cd /home/flysync/
[flysync@node2 flysync]# ls rbenv_2.2.2_linux_x64.zip
rbenv_2.2.2_linux_x64.zip
[flysync@node2 flysync]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 解压rbenv_2.2.2_linux_x64.zip包
    [root@node2 flysync]su flysync
[flysync@node2 flysync]# unzip -q /home/flysync/rbenv_2.2.2_linux_x64.zip
[flysync@node2 flysync]# ls -l
总用量 6836
-rwxr-xr-x 1 root root     340 11月  4 2019 fspm
drwxrwxr-x 3 flysync flysync      17 11月  4 2019 rbenv
-rw-r--r-- 1 flysync flysync 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]#

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 把ruby添加到环境变量
    [flysync@node2 ~]$ vi ~/.bash_profile
[flysync@node2 ~]$ source ~/.bash_profile
[flysync@node2 ~]$ cat ~/.bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
	. ~/.bashrc
fi

# User specific environment and startup programs
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH
[flysync@node2 flysync]# ruby  -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@node2 flysync]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19

    (6)node2数据库添加用户并授权

    • 登录数据库
    [flysync@node2 ~]$ exit
exit
[root@node2 flysync]# su kingbase
[kingbase@node2 flysync]$ 
[kingbase@node2 flysync]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$  ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.

test=# 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 添加flysync用户并授权
    test=# CREATE USER flysync PASSWORD 'flysync';
CREATE ROLE
ttest=#  GRANT ALL PRIVILEGES ON DATABASE TEST TO flysync ;
GRANT
test=# ALTER USER flysync SUPERUSER;
ALTER ROLE
test=# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7

    (7)最后检查下环境

    • 检查机器名
    [kingbase@node1 install]$ exit
登出
[root@node2 flysync]# uname -n
node2

    • 1
    • 2
    • 3
    • 4
    • 检查操作系统属性检查及配置
    [root@node2 flysync]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 9838
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 9838
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@node2 flysync]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 检查IP 地址
    [root@node2 flysync]# hostname --ip-address
192.168.40.112
[root@node2 flysync]# 

    • 1
    • 2
    • 3
    • 检查其他机器的 IP 地址是否正确
    [root@node2 flysync]# ping -c 4 node1
PING node1 (192.168.40.111) 56(84) bytes of data.
64 bytes from node1 (192.168.40.111): icmp_seq=1 ttl=64 time=0.296 ms
64 bytes from node1 (192.168.40.111): icmp_seq=2 ttl=64 time=0.274 ms
64 bytes from node1 (192.168.40.111): icmp_seq=3 ttl=64 time=0.370 ms
64 bytes from node1 (192.168.40.111): icmp_seq=4 ttl=64 time=0.325 ms

--- node1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.274/0.316/0.370/0.038 ms
[root@node2 flysync]# ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=0.043 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.038 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.036 ms

--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.028/0.036/0.043/0.006 ms
[root@node2 flysync]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 检查时区
    [root@node2 flysync]# date
2023年 11月 16日 星期四 13:19:32 CST
[root@node2 flysync]# 

    • 1
    • 2
    • 3

    3.node1节点部署replicator同步程序

    (1).上传并解压replicator同步程序

    [root@node1 ~]#  su flysync
[flysync@node1 install]# cd /install/
[flysync@node1 install]# ls KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node1 install]$ tar -zxf KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node1 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator
cluster-home  extensions  flysync-replicator  tools
[root@node1 install]# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8

    (2).拷贝驱动包

    [flysync@node1 install]$ cd KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/lib
[flysync@node1 lib]$  cp /install/KingbaseFlySync-V002R002C004B20230320-replicator/extensions/jdbc/kingbase8-8.6.0.jar .
[flysync@node1 lib]$ ls -l |grep kingbase8-8.6.0.jar
-rw-rw-r-- 1 flysync flysync 1019069 11月 16 13:35 kingbase8-8.6.0.jar
[flysync@node1 lib]$ 

    • 1
    • 2
    • 3
    • 4
    • 5

    (3).配置静默安装文件

    [flysync@node1 lib]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@node1 ini]$ cp flysync_kingbase8_source.ini ~/flysync.ini
[flysync@node1 ini]$ vi ~/flysync.ini 

    • 1
    • 2
    • 3

    参数如下：

    [defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000

# 服务名
[kingbase8_source]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=node1
# 当前主机名
members=node1

# 数据源类型
datasource-type=kingbase
# 同步数据库信息
replication-host=node1
replication-user=flysync
replication-password=flysync
replication-port=54321

# 同步服务角色
role=master

## 指定连接的数据库名称; 源端时，需指定解析方式
kingbase-dbname=test
kingbase-extractor-method=xlogical

## kingbase 版本号
datasource-version=8

## 过滤器开关配置
# svc_extractor_filters=replicate,casetransform,dropcolumn,rename
svc_extractor_filters=replicate

## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521

### 过滤器详细配置
## replicate 表过滤
# property=replicator.filter.replicate.do=PUBLIC.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
# property=replicator.filter.casetransform.to_upper_case=true
property=replicator.filter.replicate.do=exam.*
## 优化参数
# 达到多少条记录后，分事务，默认 500
property=replicator.extractor.dbms.maxRowsByBlock=500
# 若多少秒没有新的事务产生，那么就直接返回该事务，不再等待合并
property=replicator.extractor.dbms.maxWaitingTime=2
# 大事务优化参数
property=replicator.extractor.dbms.minRowsPerBlock=4000

# 自动恢复尝试次数，系统 OFFLINE 时，尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时，尝试 ONLINE 之前的等待时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65

    (4).执行安装

    [flysync@node1 ini]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN  >> node1 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN  >> node1 >> Linux swappiness is currently set to 30, on restart it will be 1, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
...License :  /home/flysync/replicator/license.dat
.

#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the 
cluster will be available.

  /home/flysync/replicator/flysync/cluster-home/bin/startall

Wait a minute for the services to start up and configure themselves.  After 
that you may proceed.

We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.

Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server.

  /home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl services

Configuration is now complete.  For further information, please consult
FlySync documentation.

NOTE  >> Command successfully completed
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33

    (5).上传并更新同步软件授权

    [flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license.dat ~/replicator/license.dat
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/license.dat 
/home/flysync/replicator/license.dat

    • 1
    • 2
    • 3

    (6).让环境变量生效

    [flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ 

    • 1
    • 2

    (7).配置源端过滤策略

    • 只抽取exam 模式下的增量数据
    [flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$  vi ~/flysync.ini
cat ~/flysync.ini | grep -E "svc_extractor_filters|replicate.do"
# svc_extractor_filters=replicate,casetransform,dropcolumn,rename
svc_extractor_filters=replicate
# property=replicator.filter.replicate.do=PUBLIC.*
property=replicator.filter.replicate.do=exam.*
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7

    (8).启动同步程序并置于offline 状态

    [flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator  start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:23406

    • 1
    • 2
    • 3
    • 4

    (9).查看同步软件的运行状态

    [flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: -1
appliedLatency  : -1.0
role            : master
serviceName     : kingbase8_source
serviceType     : local
started         : true
state           : OFFLINE:NORMAL
Finished services command...
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14

    4.node2节点部署replicator同步程序

    (1).上传并解压replicator同步程序

    [root@node2 ~]# su flysync
[flysync@node2 root]$ cd /install/
[flysync@node2 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz 
KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node2 install]$ tar -zxf KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz 
[flysync@node2 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator
cluster-home  extensions  flysync-replicator  tools
[flysync@node2 install]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9

    (2).拷贝驱动包

    [flysync@node2 install]$ cd KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/lib
[flysync@node2 lib]$  cp /install/KingbaseFlySync-V002R002C004B20230320-replicator/extensions/jdbc/kingbase8-8.6.0.jar .
[flysync@node2 lib]$ ls -l |grep kingbase8-8.6.0.jar
-rw-rw-r-- 1 flysync flysync 1019069 11月 16 22:30 kingbase8-8.6.0.jar
[flysync@node2 lib]$ 

    • 1
    • 2
    • 3
    • 4
    • 5

    (3).配置静默安装文件

    [flysync@node2 lib]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@node2 ini]$ cp flysync_kingbase8_target.ini ~/flysync.ini
[flysync@node2 ini]$ vi ~/flysync.ini 
[flysync@node2 ini]$ cat ~/flysync.ini 
[defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000

# 服务名
[kingbase8_target]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=node1
# 当前主机名
members=node2

##指定连接master监听的kufl端口
master-kufl-port=3112

# 数据源类型
datasource-type=kingbase
# 同步数据库信息
replication-host=node2
replication-user=flysync
replication-password=flysync
replication-port=54321

# 同步服务角色
role=slave

## 指定连接的数据库名称
kingbase-dbname=test

## kingbase 版本号
datasource-version=8

## 过滤器开关配置
# svc-remote-filters=casetransform,dropcolumn,rename
svc-remote-filters=rename
## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521

### 过滤器详细配置
## replicate 表过滤
# property=replicator.filter.replicate.do=PUBLIC.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
# property=replicator.filter.casetransform.to_upper_case=true

## 优化参数
# 开启小事务合并入库
property=replicator.applier.dbms.optimizeRowEvents=true
# 指定单表一次入库的数据量
property=replicator.applier.dbms.maxRowBatchSize=5000

# 自动恢复尝试次数，系统 OFFLINE 时，尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时，尝试 ONLINE 之前的等待时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s
[flysync@node2 ini]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70

    (4).执行安装

    [flysync@node2 ini]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN  >> node2 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN  >> node2 >> Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
License :  /home/flysync/replicator/license.dat
.

#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the 
cluster will be available.

  /home/flysync/replicator/flysync/cluster-home/bin/startall

Wait a minute for the services to start up and configure themselves.  After 
that you may proceed.

We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.

Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server.

  /home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl services

Configuration is now complete.  For further information, please consult
FlySync documentation.

NOTE  >> Command successfully completed
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33

    (5).上传并更新同步软件授权

    [flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license.dat ~/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/license.dat 
/home/flysync/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 

    • 1
    • 2
    • 3
    • 4

    (6).让环境变量生效

    [flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 

    • 1
    • 2

    (7).配置映射策略

    • 将exam 模式下的增量数据映射到exam0017下。
    • 将exam.course 表的增量数据映射到 exam0017.course0017下。
    [flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/replicator/filters-config/rename.csv
#######################################
# RenameFilter sample definition file #
#######################################

# CSV format:
#
# originalSchema,originalTable,originalColumn,newSchema,newTable,newColumn
#
# Left side (first three columns) may contain a special character:
#
# * - match all [schemas/tables/columns] (part-string matching not supported)
#
# Right side (last three columns) may contain a special character:
#
# - - leave as original

# In case of multiple entries matching the same request, order of preference is
# important. Consider the following example:
#
# schemac,*,*,schemac1,-,-
# schemac,tablew,*,schemac2,-,-
# 
# In this example all tables are moved to "schemac1" except "tablew", which is
# moved to "schemac2". Here's the rules being followed:
#
# Order of preference when looking up schema name:
#
# 1. schema.table
# 2. schema.*
#
# Order of preference when looking up table name:
#
# 1. schema.table
# 2. *.table
#
# Order of preference when looking up column name:
#
# 1. schema.table
# 2. schema.*
# 3. *.table
# 4. *.*
#
# By the way, order of preference is fixed in order to have a fast lookup.

# Notes:
#
# * Case sensitivity matters. If it is required to ignore case, use another
#   filter in the pipeline before to transform case.
#
# * For renaming columns, make sure to have column names in KUFL events. For
#   some DBMS types a separate filter (eg. ColumnNameFilter) might be needed.

############
# Examples #
############

# Rename schema for all tables.
schemac,*,*,schemac1,-,-

# This table moved to a different schema.
schemac,tablew,*,schemac2,-,-

# Table renamed and moved to a different schema.
schemav,tablee,*,schemav1,tableee,-

# Rename table in a particular schema.
schemab,tabler,*,-,tabler1,-

# Rename each table with a given name across all schemas.
*,tablet,*,-,tablet1,-

# Rename column of this particular table.
schemaz,tableq,cola,-,-,colaa # Column renaming doesn't change schema/table.

# Rename column of each table with a given name across all schemas.
*,tablew,colb,-,-,colbb

# Rename column occurring in any table across all schemas.
*,*,colc,-,-,colcc

# Rename column across all tables in the given schema.
schemax,*,cold,-,-,coldd

# Renamed schema, but one table is left in it and some of its columns renamed.
scheman,*,*,scheman2,-,-
scheman,tabley,*,-,tabley2,- # Leave table in original schema.
scheman,tabley,colj,-,-,colj2
scheman,tabley,colk,-,-,colk2

# Renamed schema and one table with one column in it.
schemam,*,*,schemam2,-,-
schemam,tableu,*,schemam2,tableu2,- # Move renamed table to new schema too.
schemam,tableu,colj,-,-,colj2 # Column renaming doesn't change schema/table.
# --以下为新增内容--
exam,*,*,exam0017,-,-
exam,course,*,exam0017,course0017,-
flysync_kingbase8_source,trep_commit_seqno,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,consistency,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,heartbeat,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,trep_shard,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,trep_shard_channel,*,flysync_kingbase8_target,-,-
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94
    • 95
    • 96
    • 97
    • 98
    • 99
    • 100
    • 101
    • 102
    • 103
    • 104
    • 105

    (8).启动同步程序并置于offline 状态

    [flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator  start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:53248

    • 1
    • 2
    • 3
    • 4

    (9).查看同步软件的运行状态

    [flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: -1
appliedLatency  : -1.0
role            : slave
serviceName     : kingbase8_target
serviceType     : local
started         : true
state           : OFFLINE:NORMAL
Finished services command...
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13

    5.KFS存量数据搬迁

    (1)检查状态

    • 源端node1
    [flysync@node1 ~]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: -1
appliedLatency  : -1.0
role            : master
serviceName     : kingbase8_source
serviceType     : local
started         : true
state           : OFFLINE:NORMAL
Finished services command...
[flysync@node1 ~]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 目标端node2
    [flysync@node2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: -1
appliedLatency  : -1.0
role            : slave
serviceName     : kingbase8_target
serviceType     : local
started         : true
state           : OFFLINE:NORMAL
Finished services command...
[flysync@node2 ~]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14

    ddlscan 和loader迁移模式选其中一种就好

    (2).ddlscan结构搬迁

    a.新建模式exam0017
    [root@node2 ~]# su kingbase
[kingbase@node2 root]$ exit
exit
[root@node2 ~]# su - kingbase
上一次登录：四 11月 16 22:58:52 CST 2023pts/0 上
[kingbase@node2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$ ./ksql -U flysync -d test
ksql (V8.0)
输入 "help" 来获取帮助信息.

test=# CREATE SCHEMA exam0017;
CREATE SCHEMA
test=# \q
[kingbase@node2 bin]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    b.开始迁移数据
    [kingbase@node2 bin]$ exit
[root@node2 ~]# su - flysync
上一次登录：四 11月 16 23:04:05 CST 2023pts/0 上
[flysync@node2 ~]$ ddlscan -target.service kingbase8_target -source.user flysync -source.pass flysync -source.db test -source.dbtype kingbase8 -source.host node1 -source.port 54321 -source.schema exam -target.db test -mgType 0
                | [ - main] INFO  ddlscan.DDLScanWorker Tables or TableFile not specified - extracting everything!
                | [ - main] INFO  ddlscan.DDLScanWorker Connect to source database with url: jdbc:kingbase8://node1:54321/test
                | [ - main] INFO  database.AbstractDatabase Loading database driver: com.kingbase8.Driver
                | [ - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [ - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [ - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [ - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [ - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [ - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [ - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [ - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [ - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [ - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [ - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [ - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [ - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [ - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [ - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [ - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [ - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [ - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [ - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [ - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [ - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [ - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [ - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [ - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [ - main] INFO  ddlscan.DDLScanWorker There is 5 tables to migration.
                | [ - main] INFO  ddlscan.TargetDDLScanWorker Connect to target database with url: jdbc:kingbase8://node2:54321/test
                | [ - main] INFO  ddlscan.TargetDDLScanWorker Drop tables on target database.
                | [ - main] INFO  ddlscan.DDLScanCtrl 
--------------------------------------------------------------------------------
TOTAL:             5
SUCCESS:           5
FAILURE:           0
TOTAL TIME:        1s
FINISHED AT:       2023-11-16 23:04:18
OUT:               /tmp/ddlscan_20231116_230417.sql
ERROR:             
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|NUM  |SOURCE_TABLE                                                                                                                    |TARGET_TABLE                                                                                                                    |STATUS  |MESSAGE                                 |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|1    |exam.course                                                                                                                     |exam0017.course0017                                                                                                             |Success |                                        |
|2    |exam.instructor                                                                                                                 |exam0017.instructor                                                                                                             |Success |                                        |
|3    |exam.instructor_history                                                                                                         |exam0017.instructor_history                                                                                                     |Success |                                        |
|4    |exam.score                                                                                                                      |exam0017.score                                                                                                                  |Success |                                        |
|5    |exam.student                                                                                                                    |exam0017.student                                                                                                                |Success |                                        |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+

[flysync@node2 ~]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54

    (3).loader数据搬迁(极速模式)

    [flysync@node2 ~]$ loader -source.service kingbase8_source -source.user flysync -source.pass flysync \
> -source.db test -source.dbtype kingbase8 -source.host node1 \
> -source.port 54321 -source.schema exam \
> -source.rmiHost node1 -source.rmiPort 11000 \
> -target.rmiHost node2 -target.rmiPort 11000 \
> -target.service kingbase8_target -mgType 0 -clean
                | [ - main] INFO  database.AbstractDatabase Loading database driver: com.kingbase8.Driver
                | [ - main] INFO  loader.LoaderWorker Tables not specified - extracting everything!
                | [ - main] INFO  loader.LoaderWorker Import table exam.course
                | [ - main] INFO  loader.LoaderWorker Import table exam.instructor
                | [ - main] INFO  loader.LoaderWorker Import table exam.instructor_history
                | [ - main] INFO  loader.LoaderWorker Import table exam.score
                | [ - main] INFO  loader.LoaderWorker Import table exam.student
                | [ - main] INFO  conf.ReplicatorRuntime Replicator role: slave
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
                | [kingbase8_target - main] INFO  pipeline.Pipeline Configuring pipeline: slave
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=extractor
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=file_metadata
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
                | [kingbase8_target - main] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Preparing pipeline for use: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Preparing pipeline: slave
                | [kingbase8_target - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata
                | [kingbase8_target - main] INFO  datasource.FileDataSource Service directory does not exist, creating: /home/flysync/replicator/metadata/applier/kingbase8_target
                | [kingbase8_target - main] INFO  datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
                | [kingbase8_target - main] INFO  datasource.FileCommitSeqno Initializing file-based seqno tracking: directory=/home/flysync/replicator/metadata/applier/kingbase8_target file=commitseqno.0
                | [kingbase8_target - pool-1-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
                | [kingbase8_target - main] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
                | [kingbase8_target - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - main] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  pipeline.Pipeline Starting pipeline: slave
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 6
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task event count: 6
                | [kingbase8_target - pool-1-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Shutting down pipeline: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Releasing pipeline: slave
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Replicator role: slave
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
                | [kingbase8_target - main] INFO  pipeline.Pipeline Configuring pipeline: slave
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=extractor
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=file_metadata
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
                | [kingbase8_target - main] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Preparing pipeline for use: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Preparing pipeline: slave
                | [kingbase8_target - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata
                | [kingbase8_target - main] INFO  datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
                | [kingbase8_target - main] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
                | [kingbase8_target - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
                | [kingbase8_target - pool-5-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - main] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - main] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - main] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  pipeline.Pipeline Starting pipeline: slave
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 6
                | [kingbase8_target - main-q-to-dbms-0] INFO  applier.JdbcApplier nologging: exam0017.course0017
                | [kingbase8_target - main-q-to-dbms-0] INFO  applier.JdbcApplier nologging: exam0017.score
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task event count: 3
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-q-to-dbms-1] INFO  applier.JdbcApplier nologging: exam0017.instructor
                | [kingbase8_target - main-q-to-dbms-2] INFO  applier.JdbcApplier nologging: exam0017.instructor_history
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-q-to-dbms-1] INFO  applier.JdbcApplier nologging: exam0017.student
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Task event count: 3
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Task event count: 2
                | [kingbase8_target - pool-5-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Shutting down pipeline: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Releasing pipeline: slave
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Replicator role: slave
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
                | [kingbase8_target - pipeline-0] INFO  pipeline.Pipeline Configuring pipeline: slave
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=global
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
                | [kingbase8_target - pipeline-0] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - pipeline-0] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=extractor
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=file_metadata
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
                | [kingbase8_target - pipeline-0] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - pipeline-0] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - pipeline-0] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
                | [kingbase8_target - pipeline-0] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
                | [kingbase8_target - pipeline-0] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false
                | [kingbase8_target - pipeline-0] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
                | [kingbase8_target - pipeline-0] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
                | [kingbase8_target - pipeline-0] INFO  conf.ReplicatorRuntime Preparing pipeline for use: slave
                | [kingbase8_target - pipeline-0] INFO  pipeline.Pipeline Preparing pipeline: slave
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata
                | [kingbase8_target - pipeline-0] INFO  datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
                | [kingbase8_target - pipeline-0] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceService Preparing and initializing data source: name=global
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
                | [kingbase8_target - pipeline-0] INFO  datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
                | [kingbase8_target - pool-11-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
                | [kingbase8_target - pipeline-0] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - pipeline-0] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - pipeline-0] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - pipeline-0] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - pipeline-0] WARN  pipeline.StageTaskGroup [pipeline-0-q-to-dbms-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - pipeline-0] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
                | [kingbase8_target - pipeline-0] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
                | [kingbase8_target - pipeline-0] WARN  pipeline.StageTaskGroup [pipeline-0-loader-to-q-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - pipeline-0] INFO  pipeline.Pipeline Starting pipeline: slave
                | [kingbase8_target - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - pipeline-0-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "exam"."course":0-3
                | [kingbase8_target - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "exam"."instructor":0-7
                | [kingbase8_target - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "exam"."instructor_history":0-1
                | [kingbase8_target - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "exam"."score":0-15
                | [kingbase8_target - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "exam"."student":0-8
                | [kingbase8_target - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 6
                | [kingbase8_target - pipeline-0-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - pipeline-0-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - pipeline-0-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - pipeline-0-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task event count: 6
                | [kingbase8_target - pool-11-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
                | [kingbase8_target - pipeline-0] INFO  pipeline.Pipeline Shutting down pipeline: slave
                | [kingbase8_target - pipeline-0] INFO  pipeline.Pipeline Releasing pipeline: slave
                | [kingbase8_target - pipeline-0] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
                | [kingbase8_target - pipeline-0] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
                | [kingbase8_target - pipeline-0] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  loader.TargetLoaderWorker Tables imported
                | [kingbase8_target - main] INFO  loader.LoaderWorker | -------------------------打印执行结果开始-------------------------
                | [kingbase8_target - main] INFO  loader.LoaderWorker | exam.instructor_history : success
                | [kingbase8_target - main] INFO  loader.LoaderWorker | exam.course : success
                | [kingbase8_target - main] INFO  loader.LoaderWorker | exam.student : success
                | [kingbase8_target - main] INFO  loader.LoaderWorker | exam.score : success
                | [kingbase8_target - main] INFO  loader.LoaderWorker | exam.instructor : success
                | [kingbase8_target - main] INFO  loader.LoaderWorker | -------------------------打印执行结果结束-------------------------
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Replicator role: slave
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
                | [kingbase8_target - main] INFO  pipeline.Pipeline Configuring pipeline: slave
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=extractor
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
                | [kingbase8_target - main] INFO  datasource.DataSourceService Configuring data source: name=file_metadata
                | [kingbase8_target - main] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings
                | [kingbase8_target - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
                | [kingbase8_target - main] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  conf.ReplicatorRuntime Preparing pipeline for use: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Preparing pipeline: slave
                | [kingbase8_target - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata
                | [kingbase8_target - main] INFO  datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
                | [kingbase8_target - main] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
                | [kingbase8_target - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
                | [kingbase8_target - main] INFO  datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
                | [kingbase8_target - pool-15-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - main] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - main] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.*.* -> schemac1.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablet.* -> -.tablet1.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.tablew.colb -> -.-.colbb
                | [kingbase8_target - main] INFO  filter.RenameDefinitions *.*.colc -> -.-.colcc
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemax.*.cold -> -.-.coldd
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.*.* -> scheman2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.*.* -> schemam2.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.*.* -> exam0017.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
                | [kingbase8_target - main] INFO  applier.JdbcApplier Connecting to data source
                | [kingbase8_target - main] INFO  applier.JdbcApplier Suppressing logging on privileged slave
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
                | [kingbase8_target - main] WARN  pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position
                | [kingbase8_target - main] INFO  pipeline.Pipeline Starting pipeline: slave
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 6
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-q-to-dbms-0] INFO  pipeline.SingleThreadStageTask Task event count: 3
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-q-to-dbms-1] INFO  pipeline.SingleThreadStageTask Task event count: 3
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Starting stage task thread
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Task has been cancelled
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
                | [kingbase8_target - main-q-to-dbms-2] INFO  pipeline.SingleThreadStageTask Task event count: 2
                | [kingbase8_target - pool-15-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Shutting down pipeline: slave
                | [kingbase8_target - main] INFO  pipeline.Pipeline Releasing pipeline: slave
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
                | [kingbase8_target - main] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
                | [kingbase8_target - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
                | [kingbase8_target - main] INFO  loader.LoaderCtrl Total time:2
                | [kingbase8_target - main] INFO  loader.LoaderCtrl 
--------------------------------------------------------------------------------
TOTAL:             5
SUCCESS:           5
FAILURE:           0
TOTAL TIME:        2s
FINISHED AT:       2023-11-16 23:07:02
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|NUM  |SOURCE_TABLE                                                                                                                    |TARGET_TABLE                                                                                                                    |STATUS  |TOTAL     |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|1    |exam.score                                                                                                                      |exam0017.score                                                                                                                  |Success |15        |
|2    |exam.student                                                                                                                    |exam0017.student                                                                                                                |Success |8         |
|3    |exam.instructor_history                                                                                                         |exam0017.instructor_history                                                                                                     |Success |1         |
|4    |exam.course                                                                                                                     |exam0017.course0017                                                                                                             |Success |3         |
|5    |exam.instructor                                                                                                                 |exam0017.instructor                                                                                                             |Success |7         |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+

[flysync@node2 ~]$

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94
    • 95
    • 96
    • 97
    • 98
    • 99
    • 100
    • 101
    • 102
    • 103
    • 104
    • 105
    • 106
    • 107
    • 108
    • 109
    • 110
    • 111
    • 112
    • 113
    • 114
    • 115
    • 116
    • 117
    • 118
    • 119
    • 120
    • 121
    • 122
    • 123
    • 124
    • 125
    • 126
    • 127
    • 128
    • 129
    • 130
    • 131
    • 132
    • 133
    • 134
    • 135
    • 136
    • 137
    • 138
    • 139
    • 140
    • 141
    • 142
    • 143
    • 144
    • 145
    • 146
    • 147
    • 148
    • 149
    • 150
    • 151
    • 152
    • 153
    • 154
    • 155
    • 156
    • 157
    • 158
    • 159
    • 160
    • 161
    • 162
    • 163
    • 164
    • 165
    • 166
    • 167
    • 168
    • 169
    • 170
    • 171
    • 172
    • 173
    • 174
    • 175
    • 176
    • 177
    • 178
    • 179
    • 180
    • 181
    • 182
    • 183
    • 184
    • 185
    • 186
    • 187
    • 188
    • 189
    • 190
    • 191
    • 192
    • 193
    • 194
    • 195
    • 196
    • 197
    • 198
    • 199
    • 200
    • 201
    • 202
    • 203
    • 204
    • 205
    • 206
    • 207
    • 208
    • 209
    • 210
    • 211
    • 212
    • 213
    • 214
    • 215
    • 216
    • 217
    • 218
    • 219
    • 220
    • 221
    • 222
    • 223
    • 224
    • 225
    • 226
    • 227
    • 228
    • 229
    • 230
    • 231
    • 232
    • 233
    • 234
    • 235
    • 236
    • 237
    • 238
    • 239
    • 240
    • 241
    • 242
    • 243
    • 244
    • 245
    • 246
    • 247
    • 248
    • 249
    • 250
    • 251
    • 252
    • 253
    • 254
    • 255
    • 256
    • 257
    • 258
    • 259
    • 260
    • 261
    • 262
    • 263
    • 264
    • 265
    • 266
    • 267
    • 268
    • 269
    • 270
    • 271
    • 272
    • 273
    • 274
    • 275
    • 276
    • 277
    • 278
    • 279
    • 280
    • 281
    • 282
    • 283
    • 284
    • 285
    • 286
    • 287
    • 288
    • 289
    • 290
    • 291
    • 292
    • 293
    • 294
    • 295
    • 296
    • 297
    • 298
    • 299
    • 300
    • 301
    • 302
    • 303
    • 304
    • 305
    • 306
    • 307
    • 308
    • 309
    • 310
    • 311
    • 312
    • 313
    • 314
    • 315
    • 316
    • 317
    • 318
    • 319
    • 320
    • 321
    • 322
    • 323
    • 324
    • 325
    • 326
    • 327
    • 328
    • 329
    • 330
    • 331
    • 332
    • 333
    • 334
    • 335
    • 336
    • 337
    • 338
    • 339
    • 340
    • 341
    • 342
    • 343
    • 344
    • 345
    • 346
    • 347
    • 348
    • 349
    • 350
    • 351
    • 352
    • 353
    • 354
    • 355
    • 356
    • 357
    • 358
    • 359
    • 360
    • 361
    • 362
    • 363
    • 364
    • 365
    • 366
    • 367
    • 368
    • 369
    • 370
    • 371
    • 372
    • 373
    • 374
    • 375
    • 376
    • 377
    • 378
    • 379
    • 380
    • 381
    • 382
    • 383
    • 384
    • 385
    • 386
    • 387
    • 388
    • 389
    • 390
    • 391
    • 392
    • 393
    • 394
    • 395
    • 396
    • 397
    • 398
    • 399
    • 400
    • 401
    • 402
    • 403
    • 404
    • 405
    • 406
    • 407
    • 408
    • 409
    • 410
    • 411
    • 412
    • 413
    • 414
    • 415
    • 416
    • 417
    • 418
    • 419
    • 420
    • 421
    • 422
    • 423
    • 424
    • 425
    • 426
    • 427
    • 428
    • 429
    • 430
    • 431
    • 432
    • 433
    • 434
    • 435
    • 436
    • 437
    • 438
    • 439
    • 440
    • 441
    • 442
    • 443
    • 444
    • 445
    • 446
    • 447
    • 448
    • 449
    • 450
    • 451
    • 452
    • 453
    • 454
    • 455
    • 456
    • 457
    • 458
    • 459
    • 460
    • 461
    • 462
    • 463
    • 464
    • 465
    • 466
    • 467
    • 468
    • 469
    • 470
    • 471
    • 472
    • 473
    • 474
    • 475
    • 476
    • 477
    • 478
    • 479
    • 480
    • 481
    • 482
    • 483
    • 484
    • 485
    • 486
    • 487
    • 488
    • 489
    • 490
    • 491
    • 492
    • 493
    • 494
    • 495
    • 496
    • 497
    • 498
    • 499
    • 500
    • 501
    • 502
    • 503
    • 504

    6.增量数据同步

    (1).node1节点启动同步服务，数据开始同步

    [flysync@node1 ~]$  fsrepctl -service kingbase8_source online
[flysync@node1 ~]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: 0
appliedLatency  : 0.115
role            : master
serviceName     : kingbase8_source
serviceType     : local
started         : true
state           : ONLINE
Finished services command...
[flysync@node1 ~]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15

    (2).node2节点启动同步服务，数据开始同步

    [flysync@node2 ~]$ fsrepctl -service kingbase8_target online
[flysync@node2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: 0
appliedLatency  : 99.334
role            : slave
serviceName     : kingbase8_target
serviceType     : local
started         : true
state           : ONLINE
Finished services command...
[flysync@node2 ~]$ 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14

    (3).同步进程状态：查看同步是否正常。

    a.查看node1同步状态
    [flysync@node1 ~]$ fsrepctl -service kingbase8_source  status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : kb:31973688:31973840
appliedLastSeqno       : 0
appliedLatency         : 0.115
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : kingbase8_source
currentEventId         : kb:31973688:31974112
currentTimeMillis      : 1700147650924
dataServerHost         : node1
extensions             : 
host                   : node1
latestEpochNumber      : 0
masterConnectUri       : kufl://localhost:/
masterListenUri        : kufl://node1:3112/
maximumStoredSeqNo     : 0
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : UNKNOWN
relativeLatency        : 234.457
resourcePrecedence     : 99
rmiPort                : 11000
role                   : master
seqnoType              : java.lang.Long
serviceName            : kingbase8_source
serviceType            : local
simpleServiceName      : kingbase8_source
siteName               : default
sourceId               : node1
state                  : ONLINE
timeInStateSeconds     : 234.437
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 3541.458
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node1 ~]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    b.查看node2同步状态
    [flysync@node2 ~]$ fsrepctl -service kingbase8_target  status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : kb:31973688:31973840
appliedLastSeqno       : 0
appliedLatency         : 99.334
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : kingbase8_target
currentEventId         : NONE
currentTimeMillis      : 1700147694828
dataServerHost         : node2
extensions             : 
host                   : node2
latestEpochNumber      : 0
masterConnectUri       : kufl://node1:3112/
masterListenUri        : kufl://node2:3112/
maximumStoredSeqNo     : 0
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : kufl://node1:3112/
relativeLatency        : 278.361
resourcePrecedence     : 99
rmiPort                : 11000
role                   : slave
seqnoType              : java.lang.Long
serviceName            : kingbase8_target
serviceType            : local
simpleServiceName      : kingbase8_target
siteName               : default
sourceId               : node2
state                  : ONLINE
timeInStateSeconds     : 179.158
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 1304.402
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$ 


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48

    (4).数据同步验证

    a.登录node1的数据库生成数据
    [flysync@node1 ~]$ exit
exit
[root@node1 install]# su - kingbase
上一次登录：四 11月 16 18:09:03 CST 2023pts/1 上
[kingbase@node1 ~]$ cd /KingbaseES/V8/
ClientTools/  desktops/     DtsTools/     Help/         Interface/    jre/          Logs/         ManagerTools/ Scripts/      Server/       Uninstall/    xulrunner/    
[kingbase@node1 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node1 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# \d exam.course;
                        数据表 "exam.course"
   栏位    |            类型            | 校对规则 |  可空的  | 预设 
-----------+----------------------------+----------+----------+------
 cno       | numeric(2,0)               |          | not null | 
 cname     | character varying(30 char) |          | not null | 
 clevel    | character varying(13 char) |          |          | 
 pass_mark | numeric(5,0)               |          | not null | 
索引：
    "cno_pk" PRIMARY KEY, btree (cno)
检查约束限制
    "pass_mark_ck" CHECK (pass_mark > 0::numeric)
由引用：
    TABLE "exam.score" CONSTRAINT "cno_fk" FOREIGN KEY (cno) REFERENCES exam.course(cno)
test=# INSERT INTO exam.course VALUES(40,'KFS','middle',80);
INSERT 0 1
test=# INSERT INTO exam.instructor VALUES(108,'fenxiaoyue','Intermediate Instructor');
INSERT 0 1
test=# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    b.登录node2的数据库查询刚才在node1插入的数据
    [flysync@node2 ~]$ exit
登出
[root@node2 ~]# su - kingbase
上一次登录：四 11月 16 23:03:45 CST 2023pts/0 上
[kingbase@node2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$ ./ksql -U system test
ksql (V8.0)
输入 "help" 来获取帮助信息.

test=# \d exam0017.course0017;
                    数据表 "exam0017.course0017"
   栏位    |            类型            | 校对规则 |  可空的  | 预设 
-----------+----------------------------+----------+----------+------
 cno       | numeric(2,0)               |          | not null | 
 cname     | character varying(30 char) |          | not null | 
 clevel    | character varying(13 char) |          |          | 
 pass_mark | numeric(5,0)               |          | not null | 
索引：
    "course0017_pkey" PRIMARY KEY, btree (cno)
test=#SELECT * FROM exam0017.course0017 WHERE cno=40;
 cno | cname | clevel | pass_mark 
-----+-------+--------+-----------
  40 | KFS   | middle |        80
(1 行记录)

test=# SELECT * FROM exam0017.instructor WHERE ino=108;
 ino |   iname    |          title          
-----+------------+-------------------------
 108 | fenxiaoyue | Intermediate Instructor
(1 行记录)
test=# 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31

    到这里我们的本次教程就结束了

    报错处理

    1.启动报错

    • 请以下面命令方式查看报错信息
    [flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ find /home/flysync/replicator/ -name *.log
/home/flysync/replicator/service_logs/replicator-user.log
/home/flysync/replicator/service_logs/xtrabackup.log
/home/flysync/replicator/service_logs/mysqldump.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/replicator-user.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/fsrepsvc.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/failure_record_file.log

[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -100 /home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/fsrepsvc.log

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
  • 相关阅读:
    Tableau 10 中的顶级可视化技术
    踩坑记 BSS段的初始化
    【JavaSE API 】生成随机数的2种方法：Random类和Math类的Random方法
    PHP反序列化漏洞
    ARIMA模型与ARIMA-GARCH模型预测时间序列
    C# 上位机软件开发之数据采集
    linux学习：视频输入+V4L2
    本地部署Jellyfin影音服务器并实现远程访问影音库
    百趣代谢组学文献分享:真假肥胖？代谢组说了算
    [附源码]java毕业设计婴幼儿玩具共享租售平台
  • 原文地址：https://blog.csdn.net/sinat_36528886/article/details/134434495