• MySQL高可用集群解决方案之：lvs+keepalived+mysql cluster实现负载均衡

    一：前言

    Mysql-cluster本身实现了高可用，但是其sql节点并没有实现负载均衡。怎样让程序能均衡地访问sql节点呢，程序里写哪个数据库服务器IP呢？

    LVS是实现负载均衡作用的，即将客户端的需求采用特定的负载均衡算法分发到后端的Web应用服务器上。

    Keepalived是用来实现高可用的，提供健康检查，故障转移，即保证主LVS宕机后，从LVS可以在很短时间顶上，从而保证了系统或网站的稳定性。Keepalived在后端的realserver接收不到主节点的信息之后，keepalived能够自己调用ipvsadm命令生成规则，能够自动实现，将主节点的VIP以及ipvs规则“拿过来”，应用在从节点上，继续为用户服务。

    程序里写虚拟IP即可。

    拓扑图：
    在这里插入图片描述

    二：实验环境

    已经安装好了mysql cluster
    Mysql cluster安装,请参考：https://blog.csdn.net/hualinger/article/details/131092300
    在这里插入图片描述

    三：实验步骤

    3.1：安装ipvsadm

    分别在lvs-master(主调度器）和lvs-backup(从调度器）上安装.

    #下载ipvsadm

    点击这里下载ipvsadm:

    http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz

    #查看内核版本

    uname -r

    2.6.32-431.el6.x86_64

    先进cd /usr/src/kernels/看有没有该文件：

    2.6.32-431.el6.x86_64

    如果没有，需要：yum -y install kernel-devel

    否则，在后面编译的时候回报错：如libipvs.h:14:23: error: net/ip_vs.h: No such file or directory

    #创建软连接

    ln -s /usr/src/kernels/2.6.32-573.3.1.el6.x86_64/ /usr/src/linux

    注意：软连接时，需要提供2.6.32-573.3.1.el6.x86_64的绝对路径，如果直接进入/usr/src/kernels目录，这样软连接是不行的：ln -s 2.6.32-573.3.1.el6.x86_64/ /usr/src/linux

    #编译

    cd /download
tar -xvf ipvsadm-1.24.tar.gz 
cd ipvsadm-1.24
[root@ser6-70 ipvsadm-1.24]# make
make -C libipvs
make[1]: Entering directory `/download/ipvsadm-1.24/libipvs'
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include  -DHAVE_NET_IP_VS_H -c -o libipvs.o libipvs.c
ar rv libipvs.a libipvs.o
ar: creating libipvs.a
a - libipvs.o
make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs'
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\"  -DHAVE_NET_IP_VS_H -c -o ipvsadm.o ipvsadm.c
ipvsadm.c: In function ‘print_largenum’:
ipvsadm.c:1564: warning: field width should have type ‘int’, but argument 2 has type ‘size_t’
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\"  -DHAVE_NET_IP_VS_H -c -o config_stream.o config_stream.c
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\"  -DHAVE_NET_IP_VS_H -c -o dynamic_array.o dynamic_array.c
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -o ipvsadm ipvsadm.o config_stream.o dynamic_array.o libipvs/libipvs.a 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    [root@ser6-70 ipvsadm-1.24]# make install
make -C libipvs
make[1]: Entering directory `/download/ipvsadm-1.24/libipvs'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs'
if [ ! -d /sbin ]; then mkdir -p /sbin; fi
install -m 0755 -s ipvsadm /sbin
install -m 0755 ipvsadm-save /sbin
install -m 0755 ipvsadm-restore /sbin
[ -d /usr/man/man8 ] || mkdir -p /usr/man/man8
install -m 0644 ipvsadm.8 /usr/man/man8
install -m 0644 ipvsadm-save.8 /usr/man/man8
install -m 0644 ipvsadm-restore.8 /usr/man/man8
if [ -d /etc/rc.d/init.d ]; then \
  install -m 0755 ipvsadm.sh /etc/rc.d/init.d/ipvsadm; \
fi

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16

    #验证是否安装成功
    [root@ser6-70 ipvsadm-1.24]# /sbin/ipvsadm -v

    ipvsadm v1.24 2005/12/10 (compiled with getopt_long and IPVS v1.2.1)

    说明安装成功。

    3.2：安装keepalived

    分别在lvs_master(主调度器）和lvs-backup(从调度器）上安装。

    安装keepalived前先要安装popt-devel，openssl :

    yum install popt-devel -y
yum install -y openssl openssl-devel

    • 1
    • 2

    #安装keepalived

    wget http://www.keepalived.org/software/keepalived-1.1.19.tar.gz

    • 1

    #解压

    tar zxvf keepalived-1.1.19.tar.gz

    • 1

    #初始化

    [root@ser6-70 linux]# cd keepalived-1.1.19
[root@ser6-70 keepalived-1.1.19]# ./configure --prefix=/usr/local/keepalived
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a BSD-compatible install... /usr/bin/install -c
checking for strip... strip
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/wait.h that is POSIX.1 compatible... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking syslog.h usability... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking for unistd.h... (cached) yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking openssl/ssl.h usability... yes
checking openssl/ssl.h presence... yes
checking for openssl/ssl.h... yes
checking openssl/md5.h usability... yes
checking openssl/md5.h presence... yes
checking for openssl/md5.h... yes
checking openssl/err.h usability... yes
checking openssl/err.h presence... yes
checking for openssl/err.h... yes
checking for MD5_Init in -lcrypto... yes
checking for SSL_CTX_new in -lssl... yes
checking for poptGetContext in -lpopt... yes
checking for kernel version... 2.6.32
checking net/ip_vs.h usability... yes
checking net/ip_vs.h presence... yes
checking for net/ip_vs.h... yes
checking for IPVS syncd support... yes
checking for an ANSI C-conforming const... yes
checking for pid_t... yes
checking whether time.h and sys/time.h may both be included... yes
checking whether gcc needs -traditional... no
checking for working memcmp... yes
checking return type of signal handlers... void
checking for gettimeofday... yes
checking for select... yes
checking for socket... yes
checking for strerror... yes
checking for strtol... yes
checking for uname... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating genhash/Makefile
config.status: WARNING:  'genhash/Makefile.in' seems to ignore the --datarootdir setting
config.status: creating keepalived/core/Makefile
config.status: creating keepalived/include/config.h
config.status: creating keepalived.spec
config.status: creating keepalived/Makefile
config.status: WARNING:  'keepalived/Makefile.in' seems to ignore the --datarootdir setting
config.status: creating lib/Makefile
config.status: creating keepalived/vrrp/Makefile
config.status: creating keepalived/check/Makefile
config.status: creating keepalived/libipvs-2.6/Makefile
 
Keepalived configuration
------------------------
Keepalived version       : 1.1.19
Compiler                 : gcc
Compiler flags           : -g -O2
Extra Lib                : -lpopt -lssl -lcrypto 
Use IPVS Framework       : Yes
IPVS sync daemon support : Yes
Use VRRP Framework       : Yes
Use Debug flags          : No

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93

    以下全部为 YES, 表示配置成功。

    Use IPVS Framework       : Yes
IPVS sync daemon support : Yes
Use VRRP Framework       : Yes

    • 1
    • 2
    • 3

    #编译

    [root@ser6-70 keepalived-1.1.19]# make
make -C lib || exit 1;
make[1]: Entering directory `/download/keepalived-1.1.19/lib'
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c memory.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c utils.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c notify.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c timer.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c scheduler.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c vector.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c list.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c html.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c parser.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c signals.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes  -c logger.c
make[1]: Leaving directory `/download/keepalived-1.1.19/lib'
make -C keepalived
make[1]: Entering directory `/download/keepalived-1.1.19/keepalived'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/core'
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_  -c main.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_  -c daemon.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_  -c pidfile.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_  -c layer4.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_  -c smtp.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_  -c global_data.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_  -c global_parser.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/core'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/check'
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_daemon.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_data.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_parser.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_api.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_tcp.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_http.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_ssl.c
check_ssl.c: In function ‘build_ssl_ctx’:
check_ssl.c:79: warning: assignment discards qualifiers from pointer target type
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_smtp.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c check_misc.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c ipwrapper.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_  -c ipvswrapper.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/check'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/vrrp'
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_daemon.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_data.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_parser.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_notify.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_scheduler.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_sync.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_index.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_netlink.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_arp.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_if.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_track.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_ipaddress.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_iproute.c
gcc -g -O2  -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_  -c vrrp_ipsecah.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/vrrp'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6'
gcc -g -O2  -I/usr/src/linux/include -Wall -Wunused   -c -o libipvs.o libipvs.c
ar rv libipvs.a libipvs.o
ar: creating libipvs.a
a - libipvs.o
rm libipvs.o
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6'
Building ../bin/keepalived
strip ../bin/keepalived
 
Make complete
make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived'
make -C genhash
make[1]: Entering directory `/download/keepalived-1.1.19/genhash'
gcc -g -O2  -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o main.o main.c
gcc -g -O2  -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o sock.o sock.c
gcc -g -O2  -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o layer4.o layer4.c
gcc -g -O2  -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o http.o http.c
gcc -g -O2  -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o ssl.o ssl.c
ssl.c: In function ‘init_ssl’:
ssl.c:50: warning: assignment discards qualifiers from pointer target type
Building ../bin/genhash
strip ../bin/genhash
 
Make complete
make[1]: Leaving directory `/download/keepalived-1.1.19/genhash'
 
Make complete

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    [root@ser6-70 keepalived-1.1.19]# make install
make -C keepalived install
make[1]: Entering directory `/download/keepalived-1.1.19/keepalived'
install -d /usr/local/keepalived/sbin
install -m 700 ../bin/keepalived /usr/local/keepalived/sbin/
install -d /usr/local/keepalived/etc/rc.d/init.d
install -m 755 etc/init.d/keepalived.init /usr/local/keepalived/etc/rc.d/init.d/keepalived
install -d /usr/local/keepalived/etc/sysconfig
install -m 755 etc/init.d/keepalived.sysconfig /usr/local/keepalived/etc/sysconfig/keepalived
install -d /usr/local/keepalived/etc/keepalived/samples
install -m 644 etc/keepalived/keepalived.conf /usr/local/keepalived/etc/keepalived/
install -m 644 ../doc/samples/* /usr/local/keepalived/etc/keepalived/samples/
install -d /usr/local/keepalived/share/man/man5
install -d /usr/local/keepalived/share/man/man8
install -m 644 ../doc/man/man5/keepalived.conf.5 /usr/local/keepalived/share/man/man5
install -m 644 ../doc/man/man8/keepalived.8 /usr/local/keepalived/share/man/man8
make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived'
make -C genhash install
make[1]: Entering directory `/download/keepalived-1.1.19/genhash'
install -d /usr/local/keepalived/bin
install -m 755 ../bin/genhash /usr/local/keepalived/bin/
install -d /usr/local/keepalived/share/man/man1
install -m 644 ../doc/man/man1/genhash.1 /usr/local/keepalived/share/man/man1
make[1]: Leaving directory `/download/keepalived-1.1.19/genhash'
[root@ser6-70 keepalived-1.1.19]# 
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29

    3.3：配置keepadlived

    3.3.1 lvs-master配置

    vi /etc/keepalived/keepalived.conf

    • 1

    添加内容：

    global_defs {
notification_email {
               baidandan@gooagoo.com    #email 通知
        }
        notification_email_from baidandan@gooagoo.com
        smtp_server smtp.mxhichina.com 
        smtp_connect_timeout 30
        router_id LVS1          # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_sync_group test {           #设置vrrp组
group {
        loadbalance
	}
}
 
vrrp_instance loadbalance {
        state MASTER       #设置lvs的状态，报错MASTER和BACKUP两种，必须大写
        interface eth0     #设置对外服务的接口
        lvs_sync_daemon_inteface eth0   #设置lvs监听的接口
        virtual_router_id 51                     #设置虚拟路由表示
        priority 180            #设置优先级，数值越大，优先级越高
        advert_int 5           #设置同步时间间隔
		authentication {                    #设置验证类型和密码
		        auth_type PASS
		        auth_pass 1111
		}
		virtual_ipaddress {                #设置lvs vip
		        192.168.6.202
		}
}
 
virtual_server 192.168.6.202 3306 {
        delay_loop 6          #健康检查时间间隔
        lb_algo rr               #负载均衡调度算法
        lb_kind DR            #负载均衡转发规则
        #persistence_timeout 20  #设置会话保持时间，对bbs等很有用
        protocol TCP                #协议
        real_server 192.168.6.66 3306 {
        weight 3                #设置权重
        TCP_CHECK {
        connect_timeout 3
        nb_get_retry 3
        delay_before_retry 3
        connect_port 3306 
}
}
        real_server 192.168.6.67 3306 {
		        weight 3
		        TCP_CHECK {
		        connect_timeout 3
		        nb_get_retry 3
		        delay_before_retry 3
		        connect_port 3306 
		}
	}
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56

    3.3.2 lvs-backup 配置

    vi /etc/keepalived/keepalived.conf

    • 1

    添加：

    global_defs {
notification_email {
              baidandan@gooagoo.com 
        }
        notification_email_from baidandan@gooagoo.com 
        smtp_server smtp.mxhichina.com 
        smtp_connect_timeout 30
        router_id LVS2
}
vrrp_sync_group test {
group {
        loadbalance
}
}
 
vrrp_instance loadbalance {
        state BACKUP
        interface eth0
        lvs_sync_daemon_inteface eth0
        virtual_router_id 51
        priority 150
        advert_int 5
authentication {
        auth_type PASS
        auth_pass 1111
}
virtual_ipaddress {
        192.168.6.202
}
}
 
virtual_server 192.168.6.202 3306 {
        delay_loop 6
        lb_algo rr
        lb_kind DR
        #persistence_timeout 20
        protocol TCP
        real_server 192.168.6.66 3306 {
        weight 3
        TCP_CHECK {
        connect_timeout 3
        nb_get_retry 3
        delay_before_retry 3
        connect_port 3306 
}
}
        real_server 192.168.6.67 3306 {
        weight 3
        TCP_CHECK {
        connect_timeout 3
        nb_get_retry 3
        delay_before_retry 3
        connect_port 3306 
}
}
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56

    3.3.3 Realserver配置

    在所有sql节点上配置：

    vi /etc/rc.d/init.d/realserver.sh

    • 1

    添加：

    #!/bin/bash
# description: Config realserver lo and apply noarp
 
SNS_VIP=192.168.6.201
 
/etc/rc.d/init.d/functions
 
case "$1" in
start)
       ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
       /sbin/route add -host $SNS_VIP dev lo:0
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
       sysctl -p >/dev/null 2>&1
       echo "RealServer Start OK"
 
       ;;
stop)
       ifconfig lo:0 down
       route del $SNS_VIP >/dev/null 2>&1
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
       echo "RealServer Stoped"
       ;;
*)
       echo "Usage: $0 {start|stop}"
       exit 1
esac
 
exit 0

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34

    赋予执行权限

    chmod +x /etc/rc.d/init.d/functions
chmod +x /etc/rc.d/init.d/realserver.sh

    • 1
    • 2

    #启动

    /etc/rc.d/init.d/realserver.sh start 

    • 1

    启动后使用ifconfig命令能看到类似下面新增的：

    lo:0      Link encap:Local Loopback  

          inet addr:192.168.6.202  Mask:255.255.255.255

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

    • 1
    • 2
    • 3
    • 4
    • 5

    注意：在sql节点一定要启动这个，否则后面无法连接mysql虚拟ip：

     ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.6.202' (110)

    • 1

    .3.4：启动keepalived

    在lvs-master,lvs-backup上启动：

    service keepalived start

    Starting keepalived: [ OK ]

    启动keepalived后，虚拟ip就能ping通了。

    #查看状态

    在lvs-master查看：

    [root@ser6-70 keepalived]# ip addr
1：lo: ,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2：eth0: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
    inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
    inet 192.168.6.202/32 scope global eth0
    inet6 fe80::5054:ff:fe3d:fd30/64 scope link 
       valid_lft forever preferred_lft forever
       

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13

    可以看到虚拟 IP 192.168.6.202.

    [root@ser6-70 keepalived]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.6.202:mysql rr
  -> 192.168.6.67:mysql           Route   3      2          0         
  -> 192.168.6.66:mysql           Route   3      0          0 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7

    – 在 lvs-backup 上查看状态

    [root@ser6-71 keepalived]# ip addr
 1：lo: ,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
 2：eth0: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
    inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
    inet6 fe80::5054:ff:fe3d:fd72/64 scope link 
       valid_lft forever preferred_lft forever

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11

    这里没看到虚拟 ip ，说明这个是从 lvs.

    [root@ser6-71 keepalived]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.6.202:mysql rr
  -> 192.168.6.67:mysql           Route   3      0          0         
  -> 192.168.6.66:mysql           Route   3      0          0      
  

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8

    3.4 建允许远程访问的用户

    在sql节点上创建允许远程访问的用户：

    grant select,update,delete,insert on *.* to 'dandan' identified by 'dandan';

    • 1

    3.5 测试

    3.5.1 测试lvs负载均衡

    #在客户端192.168.6.51连接虚拟ip

    [root@ser6-51 ~]# mysql -u dandan -pdandan -h 192.168.6.202
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3677
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
 
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
 
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| dba                |
| mysql              |
| ndbinfo            |
| performance_schema |
| test               |
+--------------------+
6 rows in set (0.00 sec)
 
mysql> use dba;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Database changed
mysql> show tables;
+---------------+
| Tables_in_dba |
+---------------+
| t2            |
+---------------+
1 row in set (0.01 sec)
 
mysql> select * from t2;
+------+
| id   |
+------+
|    1 |
|    4 |
|    2 |
|    3 |
|    5 |
|    1 |
+------+
6 rows in set (0.01 sec)
 
mysql> delete from t2 where id = 1;
Query OK, 2 rows affected (0.07 sec)
 
mysql> select * from t2;
+------+
| id   |
+------+
|    3 |
|    5 |
|    2 |
|    4 |
+------+
4 rows in set (0.00 sec)
 
mysql> 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68

    在 lvs-master 上查看：

    [root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state       source             virtual            destination
TCP 14:29  ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306

    • 1
    • 2
    • 3
    • 4

    看到192.168.6.51通过虚拟ip，连接到了192.168.6.66.

    进sql节点192.168.6.66，看是否真的删除了数据。

    [root@ser6-66 ~]# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3731
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
 
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
 
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
mysql> use dba;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Database changed
mysql> select * from t2;
+------+
| id   |
+------+
|    3 |
|    5 |
|    2 |
|    4 |
+------+
4 rows in set (0.02 sec)

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29

    真的删除了，说明客户端成功通过虚拟 ip 连接到了 sql 节点。

    #在客户端192.168.6.52上登录vip

    [root@ser6-52 ~]# mysql -u dandan -pdandan -h 192.168.6.202
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3739
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
 
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
 
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
mysql> 

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15

    在 lvs-master 上查看状态：

    [root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state       source             virtual            destination
TCP 12:36  ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306
TCP 14:54  ESTABLISHED 192.168.6.52:52348 192.168.6.202:3306 192.168.6.67:3306

    • 1
    • 2
    • 3
    • 4
    • 5

    显示192.168.6.52连接到了192.168.6.67.说明lvs实现了负载均衡。两个客户端分别连上了不同的sql节点。

    注意：只能在lvs-master上输入ipvsadm -lcn进行查看，在lvs-backup上查看是空的：

    [root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state       source             virtual            destination

    • 1
    • 2
    • 3

    3.5.2 测试keepalived高可用

    现在把lvs-master的keepalived关闭，看是否能实现高可用。

    [root@ser6-70 keepalived]# service keepalived stop
Stopping keepalived:                                       [  OK  ]

    • 1
    • 2

    过几秒钟后，在lvs-backup上进行查看：

    [root@ser6-71 keepalived]# ip addr
1: lo: ,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
    inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
    inet 192.168.6.202/32 scope global eth0
    inet6 fe80::5054:ff:fe3d:fd72/64 scope link 
       valid_lft forever preferred_lft forever
       

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13

    看到虚拟 IP 已经漂移到了 lvs-backup.

    [root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state       source             virtual            destination
TCP 14:25  ESTABLISHED 192.168.6.52:54311 192.168.6.202:3306 192.168.6.67:3306
TCP 14:49  ESTABLISHED 192.168.6.51:29604 192.168.6.202:3306 192.168.6.66:3306

    • 1
    • 2
    • 3
    • 4
    • 5

    客户端也能够正常访问vip节点，这里不再赘述。

    在lvs-master上启动keepalived后，vip会自动漂移到master节点上。

    [root@ser6-70 keepalived]# service keepalived start
Starting keepalived:                                       [  OK  ]

    • 1
    • 2
    [root@ser6-70 keepalived]# ip addr
1: lo: ,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
    inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
    inet 192.168.6.202/32 scope global eth0
    inet6 fe80::5054:ff:fe3d:fd30/64 scope link 
       valid_lft forever preferred_lft forever
       

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
  • 相关阅读:
    html盒子模型
    Openwrt_树莓派B+_Wifi中继
    修复MybatisX1.4.17版本插件误报@Mapkey is required错误
    【项目管理】中途接手的项目应对实用指南
    SpringBoot集成swagger
    GAN论文阅读笔记（9）—— Dual-path Image Inpainting with Auxiliary GAN Inversion
    QT使用时，报错说No suitable kits can be found
    万字长文总结与剖析C语言关键字 -- <<C语言深度解剖>>
    SpringBoot SpringBoot 运维实用篇 3 多环境开发 3.5 多环境开发控制
    【计算机网络】认识协议
  • 原文地址：https://blog.csdn.net/hualinger/article/details/131090886