Redis高可用架构 (redis主从+sentinel)

快来打我* 2022-05-10 08:20 212阅读 0赞

Redis集群本身已经做到了高可用和横向扩展，但是，实际情况一些小的业务没必要上集群，单个实例就可以满足业务需求，那么我们就要想办法如何保证单个实例的高可用。

keepalived是主备模式，意味着总有一台浪费着。

下面我介绍一下redis主从+sentinel(哨兵)漂移VIP的方案

# 一：实验环境 #

\#redis

10.192.203.102:6400 主库

10.192.203.107:6400 从库

10.192.203.206 VIP

\#sentinel

10.192.203.102:26400 sentinel 本地节点

10.192.203.107:26400 sentinel 本地节点

10.192.203.201:26400 sentinel 仲裁节点

# 二：实验步骤 #

## 1：安装redis ##

分别在10.192.203.102，10.192.203.107，10.192.203.201上安装。

\#下载源码，解压缩后编译源码。

1.  `wget http://download.redis.io/releases/redis-2.8.3.tar.gz`
2.  `tar xzf redis-2.8.3.tar.gz`
3.  `cd redis-2.8.3`
4.  `make`

\#编译完成后，将redis.conf和Src目录下的redis-server、redis-benchmark、redis-cli文件拷贝到一个目录下。

1.  `mkdir /usr/local/redis`
2.  `mkdir -p /data/redis/redis_sentinels`
3.  `mkdir /data/redis/6400`
4.  `cp redis.conf /usr/local/redis`
5.  `cd src`
6.  `cp redis-server /usr/local/redis`
7.  `cp redis-benchmark /usr/local/redis`
8.  `cp redis-cli /usr/local/redis`
9.  `cd /usr/local/redis`

\#撰写redis配置文件

(10.192.203.102，10.192.203.107)

vi /etc/redis\_6400.conf

1.  `daemonize yes`
2.  `pidfile "/var/run/redis_6400.pid"`
3.  `port 6400`
4.  `bind 0.0.0.0`
5.  `timeout 0`
6.  `tcp-keepalive 60`
7.  `loglevel notice`
8.  `logfile "/data/redis/redis_6400.log"`
9.  `maxmemory 8gb`
10. `maxmemory-policy allkeys-lru`
11. `databases 16`
12. `save 900 1`
13. `save 300 10`
14. `save 60 10000`
15. `stop-writes-on-bgsave-error yes`
16. `rdbcompression yes`
17. `rdbchecksum yes`
18. `dbfilename "dump.rdb"`
19. `dir "/data/redis/6400"`
20. `slave-serve-stale-data yes`
21. `slave-read-only yes`
22. `repl-disable-tcp-nodelay no`
23. `slave-priority 100`
24. `appendonly no`
25. `appendfilename "appendonly.aof"`
26. `appendfsync everysec`
27. `no-appendfsync-on-rewrite no`
28. `auto-aof-rewrite-percentage 100`
29. `auto-aof-rewrite-min-size 64mb`
30. `lua-time-limit 5000`
31. `slowlog-log-slower-than 10000`
32. `slowlog-max-len 128`
33. `notify-keyspace-events ""`
34. `hash-max-ziplist-entries 512`
35. `hash-max-ziplist-value 64`
36. `list-max-ziplist-entries 512`
37. `list-max-ziplist-value 64`
38. `set-max-intset-entries 512`
39. `zset-max-ziplist-entries 128`

## 2：撰写sentinel配置文件 ##

（在10.192.203.102，10.192.203.107，10.192.203.201上）

vi /etc/redis-sentinel6400.conf

1.  `daemonize yes`
2.  `port 26400`
3.  `dir "/data/redis/redis_sentinels"`
4.  `pidfile "/var/run/sentinel6400.pid"`
5.  `logfile "/data/redis/redis_sentinels/sentinel6400.log"`
6.  `sentinel monitor master6400 10.192.203.102 6400 2`
7.  `sentinel down-after-milliseconds master6400 30000`
8.  `sentinel failover-timeout master6400 180000`
9.  `sentinel client-reconfig-script master6400 /opt/notify_master6400.sh`

## 3：撰写漂移VIP的脚本 ##

（10.192.203.102，10.192.203.107）

vi /opt/notify\_master6400.sh

1.  `#!/bin/bash`
2.  `MASTER_IP=$6`
3.  `LOCAL_IP='10.192.203.102' #从库修改为10.192.203.107`
4.  `VIP='10.192.203.206'`
5.  `NETMASK='24'`
6.  `INTERFACE='eth0'`
7.  `if [ ${MASTER_IP} = ${LOCAL_IP} ]; then`
8.  `/sbin/ip addr add ${VIP}/${NETMASK} dev ${INTERFACE}`
9.  `/sbin/arping -q -c 3 -A ${VIP} -I ${INTERFACE}`
10. `exit 0`
11. `else`
12. `/sbin/ip addr del ${VIP}/${NETMASK} dev ${INTERFACE}`
13. `exit 0`
14. `fi`
15. `exit 1`

chmod +x /opt/notify\_master6400.sh

这里大概说一下这个脚本的工作原理，sentinel在做failover的过程中会传出6个参数，分别是<master-name>、 <role>、 <state>、 <from-ip>、 <from-port>、 <to-ip> 、<to-port>，其中第6个参数from-ip也就是新的master的ip，对应脚本中的MASTER\_IP，下面的if判断大家应该都很了然了，如果MASTER\_IP=LOCAL\_IP，那就绑定VIP，反之删除VIP。

## 4：启动redis服务 ##

（10.192.203.102，10.192.203.107）

/usr/local/redis/redis-server /etc/redis\_6400.conf

## 5：开放防火墙端口 ##

如果防火墙是开启状态的话，需要开放相应的端口

在三台服务器上开放端口6400,26400

vi /etc/sysconfig/iptables

添加：

\-A INPUT -m state --state NEW -m tcp -p tcp--dport 6400 -j ACCEPT

\-A INPUT -m state --state NEW -m tcp -p tcp--dport 26400 -j ACCEPT

\#重启防火墙

service iptables restart

## 6：初始化主从 ##

（10.192.203.107）

/usr/local/redis/redis-cli -p 6400 slaveof 10.192.203.102 6400

## 7：绑定VIP到主库 ##

（10.192.203.102）

/sbin/ip addr add 10.192.203.206/24 dev eth0

可以使用ip addr命令进行验证，如：

1.  `[root@PC redis]# ip addr`
2.  `1: lo: <LOOPBACK,UP,LOWER_UP> mtu16436 qdisc noqueue state UNKNOWN`
3.  `link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00`
4.  `inet 127.0.0.1/8 scope host lo`
5.  `inet6 ::1/128 scope host`
6.  `valid_lft forever preferred_lft forever`
7.  `2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP>mtu 1500 qdisc pfifo_fast state UP qlen 1000`
8.  `link/ether 08:00:27:04:05:16 brd ff:ff:ff:ff:ff:ff`
9.  `inet 10.192.203.102/24 brd 10.192.203.255 scope global eth0`
10. `inet 10.192.203.206/24 scope global secondary eth0`
11. `inet6 fe80::a00:27ff:fe04:516/64 scope link tentative dadfailed`
12. `valid_lft forever preferred_lft forever`

## 8：启动sentinel服务 ##

(10.192.203.102，10.192.203.107，10.192.203.201)

/usr/local/redis/redis-server /etc/redis-sentinel6400.conf --sentinel

至此，整个高可用方案已经搭建完成。

## 9：检查replication与Sentinel是否配置成功 ##

\#检查复制信息,确保有slave连接。

` `

1.  `/usr/local/redis/redis-cli -h 10.192.203.102 -p 6400 info Replication`
2.  `# Replication`
3.  `role:master`
4.  `connected_slaves:1`
5.  `slave0:ip=10.192.203.107,port=6400,state=online,offset=127,lag=0`
6.  `master_repl_offset:127`
7.  `repl_backlog_active:1`
8.  `repl_backlog_size:1048576`
9.  `repl_backlog_first_byte_offset:2`
10. `repl_backlog_histlen:126`

\#检查Sentinel信息，确保status=ok

1.  `[root@PC redis]# /usr/local/redis/redis-cli-h 10.192.203.102 -p 26400 info Sentinel`
2.  `# Sentinel`
3.  `sentinel_masters:1`
4.  `sentinel_tilt:0`
5.  `sentinel_running_scripts:0`
6.  `sentinel_scripts_queue_length:0`
7.  `master0:name=master6400,status=ok,address=10.192.203.102:6400,slaves=1,sentinels=3`

## 10：验证是否实现了高可用 ##

### 1：停止主库 ###

/usr/local/redis/redis-cli -h 10.192.203.102  -p 6400 shutdown

### 2：看从库是否提升为主库 ###

\[root@PC ~\]\# /usr/local/redis/redis-cli-h  10.192.203.107 -p 6400 info  Replication

1.  `# Replication`
2.  `role:master`
3.  `connected_slaves:0`
4.  `master_repl_offset:0`
5.  `repl_backlog_active:0`
6.  `repl_backlog_size:1048576`
7.  `repl_backlog_first_byte_offset:0`
8.  `repl_backlog_histlen:0`

### 3：验证VIP是否漂移 ###

1.  `[root@PC ~]# ip addr`
2.  `1: lo: <LOOPBACK,UP,LOWER_UP> mtu16436 qdisc noqueue state UNKNOWN`
3.  `link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00`
4.  `inet 127.0.0.1/8 scope host lo`
5.  `inet6 ::1/128 scope host`
6.  `valid_lft forever preferred_lft forever`
7.  `2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000`
8.  `link/ether 08:00:27:04:05:16 brd ff:ff:ff:ff:ff:ff`
9.  `inet 10.192.203.107/24 brd 10.192.203.255 scope global eth0`
10. `inet 10.192.203.206/24 scope global secondary eth0`
11. `inet6 fe80::a00:27ff:fe04:516/64 scope link tentative dadfailed`
12. `valid_lft forever preferred_lft forever`

### 4：查看Sentinel监控状态 ###

说明漂移成功。

## 11：配置PATH ##

修改.bash\_profile文件，在PATH末尾添加：

:/usr/local/redis

source .bash\_profile文件生效

\--本文主要参考自“[屌丝运维男][Link 1]”博客,并做了些许改动。

[Link 1]: http://navyaijm.blog.51cto.com/4647068/1745569/