Redis 5.0.5 Cluster 集群搭建

梦里梦外; 2023-05-30 15:53 3阅读 0赞

**参考资料：https://redis.io/download**

## 1、下载部署 ##

wget  -b http://download.redis.io/releases/redis-5.0.5.tar.gz
    tar xzf redis-5.0.5.tar.gz
    cd redis-5.0.5    
    make && make install 编译安装

## 2、集群创建 ##

Redis 集群最小配置为 3主 3 从，先进行简单配置文件设置

cd  /data/redis
    mkdir  conf logs 7000 7001

修改配置文件，主要修改以下参数，详细配置文件参考附件

cluster-enabled yes
    port 7000
    cluster-config-file nodes-7000.conf
    pidfile /data/redis/redis_7000.pid
    dbfilename 7000-dump.rdb
    dir /data/redis/7000
    logfile /data/redis/logs/redis-7000.log
    include ./redis-conf

使用命令快速创建7001文件 sed ‘s/7000/7001/g’ 7000.conf > 7001.conf  
启动单机服务

./redis-server /data/redis/conf/7000.conf
    140688:C 08 Nov 2019 16:03:44.932 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    140688:C 08 Nov 2019 16:03:44.932 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=140688, just started
    140688:C 08 Nov 2019 16:03:44.932 # Configuration loaded

p此时在/data/redis/6379 文件夹下将看到如下文件

-rw-r--r-- 1 root root   114 Nov  8 15:37 nodes-7000.conf
    -rw-r--r-- 1 root root     6 Nov  8 15:37 redis_7000.pid
    -rw-r--r-- 1 root root 10275 Nov  8 15:36 redis.conf

参照如上配置分别启动7000 7001 7002 7003 7004 7005六个节点  
使用命令进行集群创建

./redis-cli --cluster create 192.168.70.4:7000 192.168.70.4:7001 192.168.41.14:7002 192.168.41.14:7003 192.168.41.18:7004  192.168.41.18:7005  --cluster-replicas 1

–cluster-replicas 1 命令的意思： 一主一从配置，六个节点就是 三主三从

[root@bjxg-elk2-kafka-02 src]# ./redis-cli --cluster create 192.168.70.4:7000 192.168.70.4:7001 192.168.41.14:7002 192.168.41.14:7003 192.168.41.18:7004 192.168.41.18:7005 --cluster-replicas 1
    >>> Performing hash slots allocation on 6 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    Adding replica 192.168.41.14:7003 to 192.168.70.4:7000
    Adding replica 192.168.41.18:7005 to 192.168.41.14:7002
    Adding replica 192.168.70.4:7001 to 192.168.41.18:7004
    M: 0f96b6e007c27255d90f50737073c34741e7aa32 192.168.70.4:7000
       slots:[0-5460] (5461 slots) master
    S: 4b0d525711e0704266e78acb15c3053c94dc9dbd 192.168.70.4:7001
       replicates fbadfccf8e5228d35afa93a17a7e8aef70a3da43
    M: fccbefd52d4bb9ea348a9601d83844a81d3d59ee 192.168.41.14:7002
       slots:[5461-10922] (5462 slots) master
    S: ae2cb8a7821d16fe3770f71ffcc47522d72a24d2 192.168.41.14:7003
       replicates 0f96b6e007c27255d90f50737073c34741e7aa32
    M: fbadfccf8e5228d35afa93a17a7e8aef70a3da43 192.168.41.18:7004
       slots:[10923-16383] (5461 slots) master
    S: a3b9b3edd2a6c69666d2ca7964a1186b2da063a4 192.168.41.18:7005
       replicates fccbefd52d4bb9ea348a9601d83844a81d3d59ee
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    ...
    >>> Performing Cluster Check (using node 192.168.70.4:7000)
    M: 0f96b6e007c27255d90f50737073c34741e7aa32 192.168.70.4:7000
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    S: a3b9b3edd2a6c69666d2ca7964a1186b2da063a4 192.168.41.18:7005
       slots: (0 slots) slave
       replicates fccbefd52d4bb9ea348a9601d83844a81d3d59ee
    S: 4b0d525711e0704266e78acb15c3053c94dc9dbd 192.168.70.4:7001
       slots: (0 slots) slave
       replicates fbadfccf8e5228d35afa93a17a7e8aef70a3da43
    M: fccbefd52d4bb9ea348a9601d83844a81d3d59ee 192.168.41.14:7002
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    S: ae2cb8a7821d16fe3770f71ffcc47522d72a24d2 192.168.41.14:7003
       slots: (0 slots) slave
       replicates 0f96b6e007c27255d90f50737073c34741e7aa32
    M: fbadfccf8e5228d35afa93a17a7e8aef70a3da43 192.168.41.18:7004
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.
    检查集群状态
    redis-cli --cluster check 192.168.41.14:7002
    [root@bjxg-elk2-kafka-02 src]# ./redis-cli --cluster check 192.168.41.14:7002
    192.168.41.14:7002 (fccbefd5...) -> 0 keys | 5462 slots | 1 slaves.
    192.168.41.18:7004 (fbadfccf...) -> 0 keys | 5461 slots | 1 slaves.
    192.168.70.4:7000 (0f96b6e0...) -> 0 keys | 5461 slots | 1 slaves.
    [OK] 0 keys in 3 masters.
    0.00 keys per slot on average.
    >>> Performing Cluster Check (using node 192.168.41.14:7002)
    M: fccbefd52d4bb9ea348a9601d83844a81d3d59ee 192.168.41.14:7002
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    M: fbadfccf8e5228d35afa93a17a7e8aef70a3da43 192.168.41.18:7004
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
    S: 4b0d525711e0704266e78acb15c3053c94dc9dbd 192.168.70.4:7001
       slots: (0 slots) slave
       replicates fbadfccf8e5228d35afa93a17a7e8aef70a3da43
    S: a3b9b3edd2a6c69666d2ca7964a1186b2da063a4 192.168.41.18:7005
       slots: (0 slots) slave
       replicates fccbefd52d4bb9ea348a9601d83844a81d3d59ee
    S: ae2cb8a7821d16fe3770f71ffcc47522d72a24d2 192.168.41.14:7003
       slots: (0 slots) slave
       replicates 0f96b6e007c27255d90f50737073c34741e7aa32
    M: 0f96b6e007c27255d90f50737073c34741e7aa32 192.168.70.4:7000
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

**测试集群状态**

redis-cli -c -h  192.168.41.14 -p 7002
    [root@bjxg-elk2-kafka-02 src]# ./redis-cli -c -h 192.168.41.14 -p 7002
    192.168.41.14:7002> cluster info
    cluster_state:ok
    cluster_slots_assigned:16384
    cluster_slots_ok:16384
    cluster_slots_pfail:0
    cluster_slots_fail:0
    cluster_known_nodes:6
    cluster_size:3
    cluster_current_epoch:6
    cluster_my_epoch:3
    cluster_stats_messages_ping_sent:213
    cluster_stats_messages_pong_sent:231
    cluster_stats_messages_meet_sent:5
    cluster_stats_messages_sent:449
    cluster_stats_messages_ping_received:231
    cluster_stats_messages_pong_received:218
    cluster_stats_messages_received:449

**查看帮助命令**

./redis-cli --help

**测试性能**

redis-benchmark -h192.168.41.14 -p 7002 -c 100 -n 100000 
    100个并发连接，100000个请求，检测host为localhost 端口为7002的redis服务器性能

## 3、遇到的问题 ##

> 1、WARNING overcommit\_memory is set to 0! Background save may fail  
> under low memory condition. To fix this issue add  
> ‘vm.overcommit\_memory = 1’ to /etc/sysctl.conf and then reboot or run  
> the command ‘sysctl vm.overcommit\_memory=1’ for this to take effect.

**解决**：  
如果“/proc/sys/vm/overcommit\_memory”的值为0，则会表示开启了OOM。可以设置为1关闭OOM  
需要在文件/etc/sysctl.conf中增加一行 vm.overcommit\_memory = 1，然后执行命令“sysctl -p”以生效。

> 2、WARNING you have Transparent Huge Pages (THP) support enabled in  
> your kernel. This will create latency and memory usage issues with  
> Redis. To fix this issue run the command ‘echo never >  
> /sys/kernel/mm/transparent\_hugepage/enabled’ as root, and add it to  
> your /etc/rc.local in order to retain the setting after a reboot.  
> Redis must be restarted after THP is disabled.

**解决：**  
你使用的是透明大页，可能导致redis延迟和内存使用问题。执行echo never > /sys/kernel/mm/transparent\_hugepage/enabled修复该问题。  
**永久解决方法：**  
将其写入/etc/rc.local文件中  
echo never > /sys/kernel/mm/redhat\_transparent\_hugepage/enabled

**3、TCP监听队列大小**  
即TCP listen的backlog大小，“/proc/sys/net/core/somaxconn”的默认值一般较小如128，需要修改大一点，比如改成32767。立即生效还可以使用命令：sysctl -w net.core.somaxconn=32767。  
要想永久生效，需要在文件/etc/sysctl.conf中增加一行：net.core.somaxconn = 32767，然后执行命令“sysctl -p”以生效。  
Redis配置项tcp-backlog的值不能超过somaxconn的大小。