Rsync+Inotify实现nginx配置文件同步

最近在项目中使用到了nginx做代理转发，前端通过负载均衡转发到两台nginx代理服务器，再由nginx转发给后端的业务主机，架构如下图：

 环境说明：本次共涉及两台nginx代理主机，系统版本为centos7.6
nginx1主机IP：192.168.100.10
nginx2主机IP：192.168.100.13

1、在两台代理主机上安装nginx，上传源码包到代理主机
tar xf nginx-1.20.1.tar.gz
cd nginx-1.20.1
安装依赖
yum -y install gcc*
yum -y install zlib-devel openssl-devel pcre-devel zlib pcre openssl gcc 
编译安装，安装目录为/usr/local/nginx：
./configure --prefix=/usr/local/nginx  --with-http_ssl_module
make && make install

1.1、做软连接

ln -s /usr/local/nginx/sbin/nginx /usr/bin/nginx
修改配置文件，加载conf.d目录下以.conf结尾的配置文件
vim /usr/bin/nginx/conf/nginx.conf
include /usr/local/nginx/conf/conf.d/*.conf;

1.2、将nginx注册为系统服务

vim /usr/lib/systemd/system/nginx.service 添加如下内容
[Unit]
Description=nginx
Documentation=http://nginx.org/en/docs/
After=network.target
 
[Service]
Type=forking
PIDFile=/usr/local/nginx/logs/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t -c /usr/local/nginx/conf/nginx.conf
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/usr/local/nginx/sbin/nginx -s reload
ExecStop=/usr/local/nginx/sbin/nginx -s quit
PrivateTmp=true
 
[Install]
WantedBy=multi-user.target

1.3、启动服务，并设置开机自启：
systemctl start nginx.service
systemctl enable nginx.service

 1.4、安装完成后，在nginx1上，将代理后端业务的配置文件上传到/usr/local/nginx/conf/conf.d/目录下，后期配置文件更新主要在nginx1上进行，nginx2去同步nginx1的配置文件

2、在两台nginx主机上安装rsync，并配置服务：

查询rsync是否已安装，一般主机上默认已经安装了rsync：

rpm -qa | grep -E "rsync|inotify"

如果未安装使用yum进行安装：yum -y install rsync

2.1、在nginx2上配置rsync服务（nginx1作为客户端，不需要启服务）

cp /etc/rsyncd.conf /etc/rsyncd.conf.bak

修改/etc/rsync.conf配置文件，添加如下内容：
uid = root
gid = root
use chroot = yes
address = 192.168.100.13
port 873
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = 192.168.100.0/24
[www]
path = /usr/local/nginx/conf/conf.d
read only = no
dont compress   = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2
auth users = root
secrets file = /etc/rsyncd_users.db

2.2、创建密码文件
vim /etc/rsyncd_users.db
root:$PASSWORD    #$PASSWORD为nginx2的root密码 

chmod  600 /etc/rsyncd_users.db
systemctl start rsyncd.service
systemctl enable rsyncd.service 

3、在两台nginx主机上安装inotify

查询inotify是否已安装，一般未安装
rpm -qa | grep -E "rsync|inotify"

如果主机配置了epel源，可以通过yum安装 inotify，如果未配置可以使用源码安装 inotify
上传源码包到代理主机，编译安装：
tar xf inotify-tools-3.13.tar.gz
cd inotify-tools-3.13
./configure --prefix=/usr/local/inotify
make && make install
配置环境变量:
echo "PATH=/usr/local/inotify/bin:$PATH" >>/etc/profile.d/inotify.sh
source /etc/profile.d/inotify.sh
echo "/usr/local/inotify/lib" >/etc/ld.so.conf.d/inotify.conf
ln -s /usr/local/inotify/include /usr/include/inotify

4、在nginx1上编写inotify脚本，监控/usr/local/nginx/conf/conf.d目录下文件的变化，监控到变化后将文件同步到nginx2

vim backup.sh

#!/bin/bash
inotifywait -rmq --format "%w%f" -e create,delete,moved_to,close_write /usr/local/nginx/conf/conf.d | while read line
do
rsync -az --delete /usr/local/nginx/conf/conf.d/ root@192.168.100.13::www --password-file=/etc/server.pass
done

说明：
inotifywait -rmq --format "%w%f" -e create,delete,moved_to,close_write /usr/local/nginx/conf/conf.d   #监控本机/usr/local/nginx/conf/conf.d目录的变化
rsync -az --delete /usr/local/nginx/conf/conf.d/ root@192.168.100.13::www  #将/usr/local/nginx/conf/conf.dconf.d目录下的文件同步到nginx2 /usr/local/nginx/conf/conf.d目录下 
--password-file=/etc/server.pass  #指定密码文件路径，需要创建
 

4.1、创建密码文件：
vim /etc/server.pass
$PASSWORD    #$PASSWORD为nginx2的root密码

chmod 600 /etc/server.pass 

启动脚本放到后台：nohup bash backup.sh &

5、在nginx2上编写inotify脚本，监控/usr/local/nginx/conf/conf.d目录下文件的变化，当配置文件发生改变时，重新加载nginx配置

vim nginx_reload.sh
#!/bin/bash
inotifywait -rmq --format "%w%f" -e create,delete,moved_to,close_write /usr/local/nginx/conf/conf.d | while read line
do
   nginx -s reload
done

启动脚本放到后台：nohup bash nginx_reload.sh &