Centos7下搭建Nginx Keepalived Cluster

ゝ一纸荒年。 2021-11-17 09:00 305阅读 0赞

# 操作环境 #

CentOS Linux release 7.4.1708 (Core)

nginx version: nginx/1.12.2

Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2

Nginx Master Node：10.10.200.3

Nginx Backup Node：10.10.200.2

VIP:10.10.200.100

--------------------

# 操作步骤 #

## 安装nginx ##

因为主要讲述keepalived配置步骤，nginx就用最简单模式安装

#yum -y install nginx

为了方面测试分别修改2台nginx server中的index.html文件如下：

Nginx Server：10.10.200.3

<h1>Nginx</h1>
    <h1>10.10.200.3</h1>

Nginx Server：10.10.200.2

<h1>Nginx</h1>
    <h1>10.10.200.2</h1>

这样我们可以通过curl获取信息如下：

[root@nginx-backup sbin]# curl -l http://10.10.200.3
    <h1>Nginx</h1>
    <h1>10.10.200.3</h1>
    [root@nginx-backup sbin]# curl -l http://10.10.200.2
    <h1>Nginx</h1>
    <h1>10.10.200.2</h1>

## 安装配置Keepalived ##

#yum -y install ipset keepalived

注意一定要安装ipset，否则后面再keepalived的过程中，会报错！！！

在Nginx Master&Backup Node上编辑nginx监控脚本，我们将脚本放在/usr/local/keepalived/sbin目录下

[root@nginx-backup sbin]# vi check_nginx_alive.sh 
    #!/bin/bash
    counter=$(ps -C nginx --no-heading|wc -l)
    if [ "${counter}" = "0" ]; then
        /usr/local/nginx/sbin/nginx
        sleep 2
        counter=$(ps -C nginx --no-heading|wc -l)
        if [ "${counter}" = "0" ]; then
            systemctl stop keepalived.service
                    exit 1
        fi
    fi
    exit 0

添加可执行权限

# chmod a+x check_nginx_alive.sh

Nginx Master Node（10.10.200.3）配置文件如下：

! Configuration File for keepalived
           vrrp_script check_nginx_alive {
               script "/usr/local/keepalived/sbin/check_nginx_alive.sh"
               interval 2
               weight -10
           }
           
           global_defs {
              notification_email {
               administrator@qq.com
             }
             script_user root  //记得添加
             enable_script_security  //记得添加
             notification_email_from Alexandre.Cassen@firewall.loc
             smtp_server smtp.exmail.qq.com
             smtp_connect_timeout 30
             router_id LVS_DEVEL_1  //唯一ID，要与Backup Node不同
             vrrp_skip_check_adv_addr
             vrrp_strict
             vrrp_garp_interval 0
             vrrp_gna_interval 0
          }
          
          vrrp_instance VI_1 { 
              state MASTER  //注意在Master中该参数为Master，在Backup中该参数为Backup
              interface em2   //VIP使用的网络端口
              virtual_router_id 51   //该数值要与BackUp Node相同
              priority 102    //Master Node的数值要大于Backup Node 
              advert_int 1
              authentication {   //Master Node与Backup Node该项设置要相同
                  auth_type PASS
                  auth_pass 1111
              }
              virtual_ipaddress {
                  10.10.200.100/24    //VIP Address
              }
              track_script {
                  check_nginx_alive
              }
          }

Nginx Backup Node（10.10.200.2）配置文件如下：

配置完成后，分别在2台Nginx Server上启动keepalived和Nginx。

在Master Node上通过 ip addr 可以查看到VIP：

3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
        link/ether 84:8f:69:da:60:7b brd ff:ff:ff:ff:ff:ff
        inet 10.10.200.3/24 brd 10.10.200.255 scope global em2
           valid_lft forever preferred_lft forever
        inet 10.10.200.100/24 scope global secondary em2
           valid_lft forever preferred_lft forever
        inet6 fe80::f1d5:a5ab:3d0d:ef6/64 scope link 
           valid_lft forever preferred_lft forever

也可以ping通VIP：

[root@kvm-server ~]# ping 10.10.200.100
    PING 10.10.200.100 (10.10.200.100) 56(84) bytes of data.
    64 bytes from 10.10.200.100: icmp_seq=1 ttl=64 time=0.052 ms
    64 bytes from 10.10.200.100: icmp_seq=2 ttl=64 time=0.040 ms
    64 bytes from 10.10.200.100: icmp_seq=3 ttl=64 time=0.040 ms
    64 bytes from 10.10.200.100: icmp_seq=4 ttl=64 time=0.049 ms

通过curl 查看10.10.200.100的页面，此时通过10.10.200.100访问的数据时Nginx Master Node（10.10.200.3）的页面数据。

[root@kvm-server ~]# curl -l http://10.10.200.100
    <h1>Nginx</h1>
    <h1>10.10.200.3</h1>

下面测试下，停掉Nginx Master Node的nginx服务，keepalived是否会将nginx服务切换至Nginx Backup Node（10.10.200.2）上面。

#/etc/init.d/nginx stop

此时查看Nginx BackUp Node（10.10.200.2）上面的日志信息，VIP已经切换过来

#tail -f /var/log/message
    Jul 30 15:04:55 nginx-backup Keepalived_vrrp[23293]: VRRP_Instance(VI_1) forcing a new MASTER election
    Jul 30 15:04:56 nginx-backup Keepalived_vrrp[23293]: VRRP_Instance(VI_1) Transition to MASTER STATE
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: VRRP_Instance(VI_1) Entering MASTER STATE
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: VRRP_Instance(VI_1) setting protocol iptable drop rule
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: VRRP_Instance(VI_1) setting protocol VIPs.
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on em2 for 10.10.200.100
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:04:57 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:05:02 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:05:02 nginx-backup Keepalived_vrrp[23293]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on em2 for 10.10.200.100
    Jul 30 15:05:02 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:05:02 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:05:02 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100
    Jul 30 15:05:02 nginx-backup Keepalived_vrrp[23293]: Sending gratuitous ARP on em2 for 10.10.200.100

通过ip addr查看

3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
        link/ether 84:8f:69:da:5e:68 brd ff:ff:ff:ff:ff:ff
        inet 10.10.200.2/24 brd 10.10.200.255 scope global em2
           valid_lft forever preferred_lft forever
        inet 10.10.200.100/24 scope global secondary em2
           valid_lft forever preferred_lft forever
        inet6 fe80::d7ca:173b:3e56:65f5/64 scope link 
           valid_lft forever preferred_lft forever

通过 curl 获取10.10.200.100页面数据，此时的页面数据应该为Nginx BackUp Node（10.10.200.2）的数据

[root@nginx-backup sbin]# curl -l http://10.10.200.100
    <h1>Nginx</h1>
    <h1>10.10.200.2</h1>

经过上面的操作，说明keepalived能正常对nginx cluster的master&backup node进行切换。