准备

CentOS：7

Redis：4.0.11

测试在3台虚拟机上进行，ip 分别为：

172.16.105.128
172.16.105.129
172.16.105.130

主从复制

1.分别在三台虚拟机上安装 redis，单机安装部署参考(CentOS 7安装Redis)

2.在从服务器( 172.16.105.129 、 172.16.105.130 )配置文件 redis.conf 中添加主从配置

# slaveof 主服务器IP 端口
slaveof 172.16.105.128 6379

3.启动 172.16.105.128 、 172.16.105.129 、 172.16.105.130 三个节点服务。

4.在主从节点上登录客户端，使用命令 info replication 可以看到相关节点信息。

主节点

127.0.0.1:6379> info replication
# Replication
role:master # 角色
connected_slaves:2 # 连接从服务器数
slave0:ip=172.16.105.129,port=6379,state=online,offset=84,lag=0 # 从服务器列表信息
slave1:ip=172.16.105.130,port=6379,state=online,offset=84,lag=1
master_replid:c6451f091badfc49b9c001a27e6aae7086db08bb
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:84 # 增量复制offset
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:84

从节点

127.0.0.1:6379> info replication
# Replication
role:slave
master_host:172.16.105.128
master_port:6379
master_link_status:up
master_last_io_seconds_ago:2
master_sync_in_progress:0
slave_repl_offset:406
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:c6451f091badfc49b9c001a27e6aae7086db08bb
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:406
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:406

5.在主服务器上执行 set hello world 命令，可以看到相应同步操作执行。

Slave 172.16.105.129:6379 asks for synchronization
2587:M 09 Nov 03:51:13.590 * Partial resynchronization not accepted: Replication ID mismatch (Slave asked for '4c65ba0873afc39788457866070142cb456fc581', my replication IDs are 'c0199fe05490d9fd6c3b49d78a2ab63571f37abf' and '0000000000000000000000000000000000000000')
2587:M 09 Nov 03:51:13.590 * Starting BGSAVE for SYNC with target: disk
2587:M 09 Nov 03:51:13.590 * Background saving started by pid 2595
2595:C 09 Nov 03:51:13.592 * DB saved on disk
2595:C 09 Nov 03:51:13.592 * RDB: 0 MB of memory used by copy-on-write
2587:M 09 Nov 03:51:13.656 * Background saving terminated with success
2587:M 09 Nov 03:51:13.657 * Synchronization with slave 172.16.105.129:6379 succeeded
2587:M 09 Nov 03:51:40.071 * Slave 172.16.105.130:6379 asks for synchronization
2587:M 09 Nov 03:51:40.071 * Partial resynchronization not accepted: Replication ID mismatch (Slave asked for '4c65ba0873afc39788457866070142cb456fc581', my replication IDs are 'c6451f091badfc49b9c001a27e6aae7086db08bb' and '0000000000000000000000000000000000000000')
2587:M 09 Nov 03:51:40.071 * Starting BGSAVE for SYNC with target: disk
2587:M 09 Nov 03:51:40.071 * Background saving started by pid 2596
2596:C 09 Nov 03:51:40.073 * DB saved on disk
2596:C 09 Nov 03:51:40.074 * RDB: 0 MB of memory used by copy-on-write
2587:M 09 Nov 03:51:40.138 * Background saving terminated with success
2587:M 09 Nov 03:51:40.139 * Synchronization with slave 172.16.105.130:6379 succeeded

5.在从服务器客户端执行 get hello 命令，可以返回在主服务器上设置的信息。

127.0.0.1:6379> get hello
"world"

哨兵机制

1.备份哨兵配置文件

[root@localhost redis-4.0.11]# cp sentinel.conf sentinel.conf.bak

2.设置哨兵配置文件

port 26379
# sentinel monitor 自定义master名字 master节点IP 端口 最低通过票数(具体需要几个哨兵节点统一才行)
sentinel monitor mymaster 172.16.105.128 6379 1
sentinel down-after-milliseconds mymaster 5000 # 设置mymaster超过指定时间午相应就认为sdown
sentinel failover-timeout mymaster 15000 # 设置mymaster超过指定时间没活过来，则启动failover，从剩余slave中选举master

3.在主服务器上启动哨兵监控，可以看到相关信息

方式

./redis-sentinel ../sentinel.conf
./redis-server ../sentinel.conf --sentinel

2632:X 09 Nov 04:27:36.922 # Sentinel ID is af1657d3b6dce9213a422d474dd52464ee287484
2632:X 09 Nov 04:27:36.922 # +monitor master mymaster 172.16.105.128 6379 quorum 1
2632:X 09 Nov 04:27:36.923 +slave slave 172.16.105.129:6379 172.16.105.129 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:27:36.924 +slave slave 172.16.105.130:6379 172.16.105.130 6379 @ mymaster 172.16.105.128 6379

4.停止 master 上的服务，查看哨兵服务日志信息

2632:X 09 Nov 04:30:03.657 # +sdown master mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:03.657 # +odown master mymaster 172.16.105.128 6379 #quorum 1/1
2632:X 09 Nov 04:30:03.657 # +new-epoch 1
2632:X 09 Nov 04:30:03.657 # +try-failover master mymaster 172.16.105.128 6379 # 哨兵开始进行故障恢复
2632:X 09 Nov 04:30:03.659 # +vote-for-leader af1657d3b6dce9213a422d474dd52464ee287484 1
2632:X 09 Nov 04:30:03.659 # +elected-leader master mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:03.659 # +failover-state-select-slave master mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:03.731 # +selected-slave slave 172.16.105.130:6379 172.16.105.130 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:03.731 * +failover-state-send-slaveof-noone slave 172.16.105.130:6379 172.16.105.130 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:03.832 * +failover-state-wait-promotion slave 172.16.105.130:6379 172.16.105.130 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:04.712 # +promoted-slave slave 172.16.105.130:6379 172.16.105.130 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:04.712 # +failover-state-reconf-slaves master mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:04.767 * +slave-reconf-sent slave 172.16.105.129:6379 172.16.105.129 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:05.771 * +slave-reconf-inprog slave 172.16.105.129:6379 172.16.105.129 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:05.771 * +slave-reconf-done slave 172.16.105.129:6379 172.16.105.129 6379 @ mymaster 172.16.105.128 6379
2632:X 09 Nov 04:30:05.841 # +failover-end master mymaster 172.16.105.128 6379 # 哨兵完成故障恢复
2632:X 09 Nov 04:30:05.841 # +switch-master mymaster 172.16.105.128 6379 172.16.105.130 6379
2632:X 09 Nov 04:30:05.842 * +slave slave 172.16.105.129:6379 172.16.105.129 6379 @ mymaster 172.16.105.130 6379
2632:X 09 Nov 04:30:05.842 * +slave slave 172.16.105.128:6379 172.16.105.128 6379 @ mymaster 172.16.105.130 6379
2632:X 09 Nov 04:30:10.874 # +sdown slave 172.16.105.128:6379 172.16.105.128 6379 @ mymaster 172.16.105.130 6379

Redis Cluster

集群搭建

1.分别在3台虚拟机上各安装配置1主1从，共6个节点，便于区分采用不同端口

172.16.105.128 节点端口：7001、7002
172.16.105.129 节点端口：7003、7004
172.16.105.130 节点端口：7005、7006

2.依次修改各实例配置文件 redis.conf

port 7001 # 根据上面修改为指定端口
bind 0.0.0.0 # 设置不限制访问来源IP
pidfile /var/run/redis_7001.pid # pidfile文件分别对应7001,7002,7003,7004,7005,7006
cluster-enabled yes # 开启集群模式
cluster-config-file nodes.conf # 设置集群配置文件
cluster-node-timeout 5000 # 集群节点超时时间
appendonly yes

3.分别启动全部6个实例，因为集群不存在配置文件 nodes.conf ，所以会为每个实例分配一个唯一的节点id

[root@localhost redis-7001]# ./src/redis-server ./redis.conf &
7684:M 11 Nov 02:29:59.975 * No cluster configuration found, I'm 00bee510e285e1951863d15e646a5fcccd10e28f

4.安装 ruby 环境（ redis-trib 依赖于 redis gem ， gem依赖于 ruby ）

[root@localhost redis-7001]# yum install ruby
[root@localhost redis-7001]# yum install rubygems

5.gem 安装 redis

[root@localhost redis-7001]# gem install redis
Fetching: redis-4.0.3.gem (100%)
ERROR:  Error installing redis:
    redis requires Ruby version >= 2.2.2.

CentOS 7 yum库 ruby 版本只支持到 2.0.0 ，而 gem 安装 redis 需要 2.2.2 版本以上。

解决方法：采用rvm更新ruby

安装 rvm

[root@localhost redis-7001]# gpg2 —keyserver hkp://keys.gnupg.net —recv-keys D39DC0E3
[root@localhost redis-7001]# curl -L get.rvm.io | bash -s stable

[root@localhost redis-7001]# find / -name rvm -print
/usr/local/rvm
/usr/local/rvm/src/rvm
/usr/local/rvm/src/rvm/bin/rvm
/usr/local/rvm/src/rvm/lib/rvm
/usr/local/rvm/src/rvm/scripts/rvm
/usr/local/rvm/bin/rvm
/usr/local/rvm/lib/rvm
/usr/local/rvm/scripts/rvm

[root@localhost redis-7001]# source /usr/local/rvm/scripts/rvm
查看 rvm 库已经 ruby 版本列表

[root@localhost redis-7001]# rvm list known

MRI Rubies
[ruby-]1.8.6[-p420]
[ruby-]1.8.7[-head] # security released on head
[ruby-]1.9.1[-p431]
[ruby-]1.9.2[-p330]
[ruby-]1.9.3[-p551]
[ruby-]2.0.0[-p648]
[ruby-]2.1[.10]
[ruby-]2.2[.10]
[ruby-]2.3[.7]
[ruby-]2.4[.4]
[ruby-]2.5[.1]
[ruby-]2.6[.0-preview2]
ruby-head
选择一个较高版本(2.5.1)进行安装

[root@localhost redis-7001]# rvm install 2.5.1
使用一个 ruby 版本

[root@localhost redis-7001]# rvm use 2.5.1
Using /usr/local/rvm/gems/ruby-2.5.1
[root@localhost redis-7001]# ruby —version
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]

可选操作

rvm use 2.5.1 --default # 设置ruby默认版本
rvm remove 2.0.0 # 卸载指定版本

再次执行 gem 安装 redis ，成功安装

[root@localhost redis-7001]# gem install redis
Fetching: redis-4.0.3.gem (100%)
Successfully installed redis-4.0.3
Parsing documentation for redis-4.0.3
Installing ri documentation for redis-4.0.3
Done installing documentation for redis after 0 seconds
1 gem installed

6.使用 redis-trib.rb 创建集群（一个 master 主节点拥有一个 slave 从节点）

replicas ：设置每个 master 主节点拥有 slave 从节点个数

[root@localhost redis-7001]# ./src/redis-trib.rb create —replicas 1 172.16.105.128:7001 \172.16.105.128:7002 172.16.105.129:7003 172.16.105.129:7004 172.16.105.130:7005 172.16.105.130:7006

Creating cluster
Performing hash slots allocation on 6 nodes…
Using 3 masters:
172.16.105.128:7001
172.16.105.129:7003
172.16.105.130:7005
Adding replica 172.16.105.129:7004 to 172.16.105.128:7001
Adding replica 172.16.105.130:7006 to 172.16.105.129:7003
Adding replica 172.16.105.128:7002 to 172.16.105.130:7005
M: 00bee510e285e1951863d15e646a5fcccd10e28f 172.16.105.128:7001
slots:0-5460 (5461 slots) master
S: c85ebc85fee43048b8135ac5b73c3337ef6b9dd0 172.16.105.128:7002
replicates 93ba46762a030b853ba3a81634fc79e92e17d6d8
M: 162cf6816ffdd75f22af63359a1da8a2eb7725a2 172.16.105.129:7003
slots:5461-10922 (5462 slots) master
S: e9ebec2d60087531517e08ae102d66e3d8db39eb 172.16.105.129:7004
replicates 00bee510e285e1951863d15e646a5fcccd10e28f
M: 93ba46762a030b853ba3a81634fc79e92e17d6d8 172.16.105.130:7005
slots:10923-16383 (5461 slots) master
S: 633b4c938e3773f90abc11261452d6638f31c7ed 172.16.105.130:7006
replicates 162cf6816ffdd75f22af63359a1da8a2eb7725a2
Can I set the above configuration? (type ‘yes’ to accept): yes
Nodes configuration updated
Assign a different config epoch to each node
7743:M 11 Nov 04:07:33.471 # configEpoch set to 1 via CLUSTER SET-CONFIG-EPOCH
7748:M 11 Nov 04:07:33.471 # configEpoch set to 2 via CLUSTER SET-CONFIG-EPOCH
Sending CLUSTER MEET messages to join the cluster
7743:M 11 Nov 04:07:33.486 # IP address for this node updated to 172.16.105.128
7748:M 11 Nov 04:07:33.650 # IP address for this node updated to 172.16.105.128
Waiting for the cluster to join…
7748:S 11 Nov 04:07:37.498 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
7748:S 11 Nov 04:07:37.498 # Cluster state changed: ok
Performing Cluster Check (using node 172.16.105.128:7001)
M: 00bee510e285e1951863d15e646a5fcccd10e28f 172.16.105.128:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 162cf6816ffdd75f22af63359a1da8a2eb7725a2 172.16.105.129:7003
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 93ba46762a030b853ba3a81634fc79e92e17d6d8 172.16.105.130:7005
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 633b4c938e3773f90abc11261452d6638f31c7ed 172.16.105.130:7006
slots: (0 slots) slave
replicates 162cf6816ffdd75f22af63359a1da8a2eb7725a2
S: c85ebc85fee43048b8135ac5b73c3337ef6b9dd0 172.16.105.128:7002
slots: (0 slots) slave
replicates 93ba46762a030b853ba3a81634fc79e92e17d6d8
S: e9ebec2d60087531517e08ae102d66e3d8db39eb 172.16.105.129:7004
slots: (0 slots) slave
replicates 00bee510e285e1951863d15e646a5fcccd10e28f
[OK] All nodes agree about slots configuration.
Check for open slots…
Check slots coverage…
[OK] All 16384 slots covered.

集群创建完成后，可以看到分配的三个 master 节点和三个 slave 节点，以及对应的槽位区间。

集群测试

客户端连接集群需要指定 -c 参数

[root@localhost redis-7001]# ./src/redis-cli -c -p 7001

在 7001 主节点设置 set name clusterTest ，根据 key 值计算槽位分布在节点 7003(5461-10922)

[root@localhost redis-7001]# ./src/redis-cli -c -p 7001
127.0.0.1:7001> set name clusterTest
-> Redirected to slot [5798] located at 172.16.105.129:7003
OK
172.16.105.129:7003> get name
"clusterTest"
172.16.105.129:7003>

在节点 7006 获取 name 值，自动跳转到节点 7003 获取值。

[root@localhost redis-7006]# ./src/redis-cli -c -p 7006
127.0.0.1:7006> get name
-> Redirected to slot [5798] located at 172.16.105.129:7003
"clusterTest"
172.16.105.129:7003>

集群节点选举

杀掉节点 7005 ，查看集群变化

[root@localhost redis-7006]# ps -ef | grep redis
root       7768   1211  0 03:17 pts/0    00:00:07 ./src/redis-server 0.0.0.0:7005 [cluster]
root       7773   1211  0 03:18 pts/0    00:00:07 ./src/redis-server 0.0.0.0:7006 [cluster]
root       7848   7831  0 04:50 pts/1    00:00:00 grep --color=auto redis
[root@localhost redis-7006]# kill 7768

可以看到集群连接不上节点 7005 ，节点 7002 角色由 slave 转换为 master 。

[root@localhost redis-7001]# ./src/redis-trib.rb check 172.16.105.130:7005
[ERR] Sorry, can't connect to node 172.16.105.130:7005
[root@localhost redis-7001]# ./src/redis-trib.rb check 172.16.105.130:7006
>>> Performing Cluster Check (using node 172.16.105.130:7006)
S: 633b4c938e3773f90abc11261452d6638f31c7ed 172.16.105.130:7006
   slots: (0 slots) slave
   replicates 162cf6816ffdd75f22af63359a1da8a2eb7725a2
S: e9ebec2d60087531517e08ae102d66e3d8db39eb 172.16.105.129:7004
   slots: (0 slots) slave
   replicates 00bee510e285e1951863d15e646a5fcccd10e28f
M: 00bee510e285e1951863d15e646a5fcccd10e28f 172.16.105.128:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: c85ebc85fee43048b8135ac5b73c3337ef6b9dd0 172.16.105.128:7002
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
M: 162cf6816ffdd75f22af63359a1da8a2eb7725a2 172.16.105.129:7003
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.