redis系列——入门内容(一) 男娘i 2022-05-08 01:16 156阅读 0赞 # 一、基本介绍 # ## 1、NoSql介绍 ## 为了解决高并发、高可用、高可扩展,大数据存储等一系列问题而产生的数据库解决方案,就是NoSql。NoSql,叫非关系型数据库,它的全名Not only sql。它不能替代关系型数据库,只能作为关系型数据库的一个良好补充。关系型数据库中的表都是存储一些结构化的数据,每条记录的字段的组成都一样,即使不是每条记录都需要所有的字段,但数据库会为每条数据分配所有的字段。而非关系型数据库以键值对(key-value)存储,它的结构不固定,每一条记录可以有不一样的键,每条记录可以根据需要增加一些自己的键值对,这样就不会局限于固定的结构,可以减少一些时间和空间的开销。 其中Nosql数据库分类如下: * 键值(Key-Value)存储数据库 相关产品: Tokyo Cabinet/Tyrant、Redis、Voldemort、Berkeley DB 典型应用: 内容缓存,主要用于处理大量数据的高访问负载。 数据模型: 一系列键值对 优势: 快速查询 劣势: 存储的数据缺少结构化 * 列存储数据库 相关产品:Cassandra, HBase, Riak 典型应用:分布式的文件系统 数据模型:以列簇式存储,将同一列数据存在一起 优势:查找速度快,可扩展性强,更容易进行分布式扩展 劣势:功能相对局限 * 文档型数据库 相关产品:CouchDB、MongoDB 典型应用:Web应用(与Key-Value类似,Value是结构化的) 数据模型: 一系列键值对 优势:数据结构要求不严格 劣势: 查询性能不高,而且缺乏统一的查询语法 * 图形(Graph)数据库 相关数据库:Neo4J、InfoGrid、Infinite Graph 典型应用:社交网络 数据模型:图结构 优势:利用图结构相关算法。 劣势:需要对整个图做计算才能得出结果,不容易做分布式的集群方案。 ## 2、Redis介绍 ## ### 1.基本说明 ### Redis是用C语言开发的一个开源的高性能键值对(key-value)数据库。它通过提供多种键值数据类型来适应不同场景下的存储需求,目前为止Redis支持的键值数据类型如下: * 字符串类型【String】 * 散列类型【map】 * 列表类型【list】 * 集合类型【set】 * 有序集合类型【sortedset】 ### 2.历史发展 ### 2008年,意大利的一家创业公司Merzia推出了一款基于MySQL的网站实时统计系统LLOOGG,然而没过多久该公司的创始人 Salvatore Sanfilippo便 对MySQL的性能感到失望,于是他决定亲自为LLOOGG量身定做一个数据库,并于2009年开发完成,这个数据库就是Redis。 不过Salvatore Sanfilippo并不满足只将Redis用于LLOOGG这一款产品,而是希望更多的人使用它,于是在同一年Salvatore Sanfilippo将Redis开源发布,并开始和Redis的另一名主要的代码贡献者Pieter Noordhuis一起继续着Redis的开发,直到今天。 Salvatore Sanfilippo自己也没有想到,短短的几年时间,Redis就拥有了庞大的用户群体。Hacker News在2012年发布了一份数据库的使用情况调查,结果显示有近12%的公司在使用Redis。国内如新浪微博、街旁网、知乎网,国外如GitHub、Stack Overflow、Flickr等都是Redis的用户。 VMware公司从2010年开始赞助Redis的开发, Salvatore Sanfilippo和Pieter Noordhuis也分别在3月和5月加入VMware,全职开发Redis。 ### 3.应用场景 ### * 缓存(数据查询、短连接、新闻内容、商品内容等等)。(最多使用) * 分布式集群架构中的session分离。 * 聊天室的在线好友列表。 * 任务队列。(秒杀、抢购、12306等等) * 应用排行榜。 * 网站访问统计。 * 数据过期处理(可以精确到毫秒) # 二、Redis的安装配置 # ## 1、redis安装 ## Redis下载地址为:[http://download.redis.io/releases/redis-3.0.0.tar.gz][http_download.redis.io_releases_redis-3.0.0.tar.gz] ;官网地址:[http://redis.io/][http_redis.io] 。安装前我们先要配置编译环境: sudo yum install gcc-c++ 下载源码: wget http://download.redis.io/releases/redis-4.0.9.tar.gz 解压源码: tar -zxvf redis-4.0.9.tar.gz 进入到解压目录: cd redis-4.0.9 执行make编译Redis: make MALLOC=libc 注意:make命令执行完成编译后,会在src目录下生成6个可执行文件,分别是redis-server、redis-cli、redis-benchmark、redis-check-aof、redis-check-rdb、redis-sentinel。 安装Redis: make install PREFIX=/usr/local/InstallationSoftware/redis ## 2、 redis的启动 ## ### 1.前端启动 ### 前端启动的命令为(首先进入bin目录): ![20190919131134788.png][] 启动界面如下 : ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIyMTcyMTMz_size_16_color_FFFFFF_t_70][] 前端启动的关闭: * 强制关闭:Ctrl+c * 正常关闭:./redis-cli shutdown 前端启动的问题:一旦客户端关闭,则redis服务也停掉。 ### 2.后端启动 ### 首先需要将redis解压之后的源码包中的redis.conf文件拷贝到我们安装的bin目录下,然后使用vim修改redis.conf文件。将daemonize no改为daemonize yes ![20190919131317580.png][] 然后使用命令后端启动redis ![20190919131334240.png][] 看是否启动成功 ![20190919131344664.png][] 关闭后端启动的方式为: 强制关闭:kill -9 7944 正常关闭:./redis-cli shutdown 注意:在项目中,建议使用正常关闭。因为redis作为缓存来使用的话,将数据存储到内存中,如果使用正常关闭,则会将内存数据持久化到本地之后,再关闭。如果是强制关闭,则不会进行持久化操作,可能会造成部分数据的丢失。 ### 3.脚本启动 ### 如果是做一些实验的话,用redis-server启动一下redis是可以的,但在生产环境上是没有意义的。我们要把redis作为一个系统的daemon进程去运行的,每次系统启动,redis进程一起启动。在redis解压目录中的utils目录下,有个redis\_init\_script脚本。将redis\_init\_script脚本拷贝到linux的/etc/init.d目录中(改下路径),将redis\_init\_script重命名为redis\_6379,6379是我们希望这个redis实例监听的端口号。然后修改redis\_6379脚本的第6行的REDISPORT,设置为相同的端口号(默认就是6379),同时修改EXEC和CLIEXEC为自己的路径。创建两个目录:/etc/redis(存放redis的配置文件),/var/redis/6379(存放redis的持久化文件)。修改redis配置文件(默认在根目录下,redis.conf),拷贝到/etc/redis目录中,修改名称为6379.conf。修改conf中的部分配置为生产环境 daemonize yes #让redis以daemon进程运行 pidfile /var/run/redis_6379.pid #设置redis的pid文件位置 port 6379 #设置redis的监听端口号 dir /var/redis/6379 #设置持久化文件的存储位置 bind 192.168.234.131 #绑定的ip 启动redis,执行 cd /etc/init.d chmod 777 redis_6379 ./redis_6379 start 确认redis进程是否启动,ps -ef | grep redis ![20190919131705114.png][] 下面让redis跟随系统启动自动启动,在redis\_6379脚本中,最上面,加入两行注释 # chkconfig: 2345 90 10 # description: Redis is a persistent key-value database 然后执行,即可 chkconfig redis_6379 on # 三、 redis.conf 的配置信息 # <table> <tbody> <tr> <td style="vertical-align:middle;"><strong>参数</strong></td> <td style="vertical-align:middle;"><strong>说明</strong></td> </tr> <tr> <td style="vertical-align:middle;">daemonize</td> <td style="vertical-align:middle;">如果需要在后台运行,把该项改为yes</td> </tr> <tr> <td style="vertical-align:middle;">pidfile</td> <td style="vertical-align:middle;">配置多个pid的地址 默认在/var/run/redis.pid</td> </tr> <tr> <td style="vertical-align:middle;">bind</td> <td style="vertical-align:middle;">绑定ip,设置后只接受来自该ip的请求</td> </tr> <tr> <td style="vertical-align:middle;">port</td> <td style="vertical-align:middle;">监听端口,默认是6379</td> </tr> <tr> <td style="vertical-align:middle;">loglevel</td> <td style="vertical-align:middle;">分为4个等级:debug verbose notice warning</td> </tr> <tr> <td style="vertical-align:middle;">logfile</td> <td style="vertical-align:middle;">用于配置log文件地址</td> </tr> <tr> <td style="vertical-align:middle;">databases</td> <td style="vertical-align:middle;">设置数据库个数,默认使用的数据库为0</td> </tr> <tr> <td style="vertical-align:middle;">save</td> <td style="vertical-align:middle;">设置redis进行数据库镜像的频率。</td> </tr> <tr> <td style="vertical-align:middle;">rdbcompression</td> <td style="vertical-align:middle;">在进行镜像备份时,是否进行压缩</td> </tr> <tr> <td>dbfilename</td> <td>镜像备份文件的文件名</td> </tr> <tr> <td>Dir</td> <td>数据库镜像备份的文件放置路径</td> </tr> <tr> <td>Slaveof</td> <td>设置数据库为其他数据库的从数据库</td> </tr> <tr> <td>Masterauth</td> <td>主数据库连接需要的密码验证</td> </tr> <tr> <td>Requriepass</td> <td>设置 登陆时需要使用密码</td> </tr> <tr> <td>Maxclients</td> <td>限制同时使用的客户数量</td> </tr> <tr> <td>Maxmemory</td> <td>设置redis能够使用的最大内存</td> </tr> <tr> <td>Appendonly</td> <td>开启append only模式</td> </tr> <tr> <td>Appendfsync</td> <td>设置对appendonly.aof文件同步的频率(对数据进行备份的第二种方式)</td> </tr> <tr> <td>vm-enabled</td> <td>是否开启虚拟内存支持 (vm开头的参数都是配置虚拟内存的)</td> </tr> <tr> <td>vm-swap-file</td> <td>设置虚拟内存的交换文件路径</td> </tr> <tr> <td>vm-max-memory</td> <td>设置redis使用的最大物理内存大小</td> </tr> <tr> <td>vm-page-size</td> <td>设置虚拟内存的页大小</td> </tr> <tr> <td>vm-pages</td> <td>设置交换文件的总的page数量</td> </tr> <tr> <td>vm-max-threads</td> <td>设置VM IO同时使用的线程数量</td> </tr> <tr> <td>Glueoutputbuf</td> <td>把小的输出缓存存放在一起</td> </tr> <tr> <td>hash-max-zipmap-entries</td> <td>设置hash的临界值</td> </tr> <tr> <td>Activerehashing</td> <td>重新hash</td> </tr> </tbody> </table> # 四、对QPS进行压测 # 如果我们想要对自己刚刚搭建好的redis做一个基准的压测,测一下redis的性能和QPS(query per second)。redis自己提供的redis-benchmark压测工具,是最快捷最方便的,当然啦,这个工具比较简单,用一些简单的操作和场景去压测。该工具在redis的源码包src中, ./redis-benchmark -h 192.168.31.187 -c <clients> Number of parallel connections (default 50) -n <requests> Total number of requests (default 100000) -d <size> Data size of SET/GET value in bytes (default 2) 这个要根据自己的高峰期的访问量来调整一些参数,比如在高峰期,瞬时最大用户量会达到10万+,-c 100000,-n 10000000,-d 50。我这里就用默认配置测一下自己的虚拟机结果如下: [root@redis_01 src]# ./redis-benchmark -h 192.168.234.129 ====== PING_INLINE ====== 100000 requests completed in 1.83 seconds 50 parallel clients 3 bytes payload keep alive: 1 89.64% <= 1 milliseconds 97.91% <= 2 milliseconds 99.35% <= 3 milliseconds 99.88% <= 4 milliseconds 100.00% <= 4 milliseconds 54704.60 requests per second ====== PING_BULK ====== 100000 requests completed in 1.80 seconds 50 parallel clients 3 bytes payload keep alive: 1 89.67% <= 1 milliseconds 98.20% <= 2 milliseconds 99.60% <= 3 milliseconds 99.99% <= 4 milliseconds 100.00% <= 4 milliseconds 55493.89 requests per second ====== SET ====== 100000 requests completed in 2.39 seconds 50 parallel clients 3 bytes payload keep alive: 1 80.40% <= 1 milliseconds 94.78% <= 2 milliseconds 98.47% <= 3 milliseconds 99.62% <= 4 milliseconds 99.80% <= 12 milliseconds 99.81% <= 13 milliseconds 99.85% <= 14 milliseconds 99.90% <= 15 milliseconds 99.95% <= 16 milliseconds 100.00% <= 16 milliseconds 41858.52 requests per second ====== GET ====== 100000 requests completed in 1.89 seconds 50 parallel clients 3 bytes payload keep alive: 1 88.78% <= 1 milliseconds 97.86% <= 2 milliseconds 99.26% <= 3 milliseconds 99.81% <= 4 milliseconds 99.97% <= 7 milliseconds 100.00% <= 7 milliseconds 53022.27 requests per second ====== INCR ====== 100000 requests completed in 2.37 seconds 50 parallel clients 3 bytes payload keep alive: 1 81.38% <= 1 milliseconds 94.57% <= 2 milliseconds 98.35% <= 3 milliseconds 99.38% <= 4 milliseconds 99.62% <= 5 milliseconds 99.65% <= 6 milliseconds 99.70% <= 7 milliseconds 99.80% <= 8 milliseconds 99.80% <= 9 milliseconds 99.83% <= 10 milliseconds 99.86% <= 11 milliseconds 99.90% <= 13 milliseconds 99.96% <= 15 milliseconds 100.00% <= 15 milliseconds 42176.30 requests per second ====== LPUSH ====== 100000 requests completed in 2.52 seconds 50 parallel clients 3 bytes payload keep alive: 1 78.92% <= 1 milliseconds 94.05% <= 2 milliseconds 98.00% <= 3 milliseconds 99.27% <= 4 milliseconds 99.45% <= 5 milliseconds 99.45% <= 6 milliseconds 99.47% <= 7 milliseconds 99.55% <= 8 milliseconds 99.60% <= 11 milliseconds 99.64% <= 12 milliseconds 99.70% <= 13 milliseconds 99.80% <= 14 milliseconds 99.88% <= 15 milliseconds 99.92% <= 16 milliseconds 99.95% <= 17 milliseconds 99.97% <= 18 milliseconds 100.00% <= 19 milliseconds 39714.06 requests per second ====== RPUSH ====== 100000 requests completed in 2.45 seconds 50 parallel clients 3 bytes payload keep alive: 1 81.01% <= 1 milliseconds 94.65% <= 2 milliseconds 98.07% <= 3 milliseconds 99.23% <= 4 milliseconds 99.48% <= 5 milliseconds 99.52% <= 6 milliseconds 99.64% <= 7 milliseconds 99.70% <= 13 milliseconds 99.85% <= 14 milliseconds 99.90% <= 15 milliseconds 99.95% <= 16 milliseconds 99.95% <= 30 milliseconds 100.00% <= 30 milliseconds 40766.41 requests per second ====== LPOP ====== 100000 requests completed in 2.33 seconds 50 parallel clients 3 bytes payload keep alive: 1 83.04% <= 1 milliseconds 95.55% <= 2 milliseconds 98.42% <= 3 milliseconds 99.37% <= 4 milliseconds 99.52% <= 5 milliseconds 99.65% <= 6 milliseconds 99.65% <= 10 milliseconds 99.70% <= 12 milliseconds 99.72% <= 13 milliseconds 99.80% <= 14 milliseconds 99.83% <= 15 milliseconds 99.85% <= 16 milliseconds 99.90% <= 20 milliseconds 99.90% <= 21 milliseconds 99.94% <= 22 milliseconds 99.95% <= 26 milliseconds 99.99% <= 27 milliseconds 100.00% <= 27 milliseconds 42863.27 requests per second ====== RPOP ====== 100000 requests completed in 2.28 seconds 50 parallel clients 3 bytes payload keep alive: 1 83.38% <= 1 milliseconds 95.56% <= 2 milliseconds 98.41% <= 3 milliseconds 99.46% <= 4 milliseconds 99.70% <= 5 milliseconds 99.75% <= 13 milliseconds 99.76% <= 14 milliseconds 99.90% <= 17 milliseconds 99.92% <= 18 milliseconds 100.00% <= 18 milliseconds 43840.42 requests per second ====== SADD ====== 100000 requests completed in 1.92 seconds 50 parallel clients 3 bytes payload keep alive: 1 89.08% <= 1 milliseconds 97.33% <= 2 milliseconds 99.10% <= 3 milliseconds 99.73% <= 4 milliseconds 99.90% <= 5 milliseconds 100.00% <= 6 milliseconds 100.00% <= 6 milliseconds 52110.47 requests per second ====== SPOP ====== 100000 requests completed in 1.80 seconds 50 parallel clients 3 bytes payload keep alive: 1 90.44% <= 1 milliseconds 98.10% <= 2 milliseconds 99.53% <= 3 milliseconds 99.95% <= 7 milliseconds 100.00% <= 8 milliseconds 100.00% <= 8 milliseconds 55463.12 requests per second ====== LPUSH (needed to benchmark LRANGE) ====== 100000 requests completed in 2.42 seconds 50 parallel clients 3 bytes payload keep alive: 1 80.83% <= 1 milliseconds 95.31% <= 2 milliseconds 98.37% <= 3 milliseconds 99.51% <= 4 milliseconds 99.60% <= 6 milliseconds 99.70% <= 7 milliseconds 99.70% <= 13 milliseconds 99.84% <= 14 milliseconds 99.85% <= 15 milliseconds 99.89% <= 16 milliseconds 99.90% <= 18 milliseconds 99.95% <= 33 milliseconds 100.00% <= 34 milliseconds 100.00% <= 34 milliseconds 41305.25 requests per second ====== LRANGE_100 (first 100 elements) ====== 100000 requests completed in 3.06 seconds 50 parallel clients 3 bytes payload keep alive: 1 55.41% <= 1 milliseconds 93.58% <= 2 milliseconds 98.68% <= 3 milliseconds 99.79% <= 4 milliseconds 99.96% <= 5 milliseconds 100.00% <= 5 milliseconds 32658.39 requests per second ====== LRANGE_300 (first 300 elements) ====== 100000 requests completed in 6.17 seconds 50 parallel clients 3 bytes payload keep alive: 1 3.37% <= 1 milliseconds 52.56% <= 2 milliseconds 85.62% <= 3 milliseconds 96.04% <= 4 milliseconds 99.49% <= 5 milliseconds 99.92% <= 6 milliseconds 99.95% <= 8 milliseconds 99.97% <= 9 milliseconds 100.00% <= 10 milliseconds 16215.34 requests per second ====== LRANGE_500 (first 450 elements) ====== 100000 requests completed in 8.35 seconds 50 parallel clients 3 bytes payload keep alive: 1 0.00% <= 1 milliseconds 25.11% <= 2 milliseconds 61.67% <= 3 milliseconds 84.97% <= 4 milliseconds 95.57% <= 5 milliseconds 99.47% <= 6 milliseconds 99.94% <= 7 milliseconds 99.99% <= 8 milliseconds 100.00% <= 8 milliseconds 11973.18 requests per second ====== LRANGE_600 (first 600 elements) ====== 100000 requests completed in 10.65 seconds 50 parallel clients 3 bytes payload keep alive: 1 0.00% <= 1 milliseconds 10.27% <= 2 milliseconds 36.59% <= 3 milliseconds 65.09% <= 4 milliseconds 83.84% <= 5 milliseconds 94.71% <= 6 milliseconds 98.99% <= 7 milliseconds 99.85% <= 8 milliseconds 99.97% <= 9 milliseconds 99.99% <= 10 milliseconds 100.00% <= 10 milliseconds 9392.32 requests per second ====== MSET (10 keys) ====== 100000 requests completed in 4.98 seconds 50 parallel clients 3 bytes payload keep alive: 1 15.10% <= 1 milliseconds 77.37% <= 2 milliseconds 90.90% <= 3 milliseconds 95.76% <= 4 milliseconds 96.87% <= 5 milliseconds 97.11% <= 6 milliseconds 97.15% <= 7 milliseconds 97.20% <= 8 milliseconds 97.24% <= 9 milliseconds 97.30% <= 10 milliseconds 97.56% <= 11 milliseconds 97.68% <= 12 milliseconds 97.70% <= 13 milliseconds 97.79% <= 14 milliseconds 98.05% <= 15 milliseconds 98.63% <= 16 milliseconds 99.01% <= 17 milliseconds 99.35% <= 18 milliseconds 99.41% <= 19 milliseconds 99.45% <= 24 milliseconds 99.47% <= 25 milliseconds 99.56% <= 26 milliseconds 99.57% <= 27 milliseconds 99.67% <= 28 milliseconds 99.70% <= 62 milliseconds 99.74% <= 63 milliseconds 99.75% <= 75 milliseconds 99.80% <= 90 milliseconds 99.85% <= 92 milliseconds 99.87% <= 93 milliseconds 99.90% <= 103 milliseconds 99.95% <= 114 milliseconds 100.00% <= 114 milliseconds 20084.35 requests per second 我们可以看到redis提供的高并发,至少到上万没问题,基本上几万差不多。当然跟生产环境还有区别的,生产环境上,大量的网络请求的调用,网络本身就有开销,redis的吞吐量就不一定那么高了。从上面压测结果上,我们也能发现**QPS的两个杀手:** * 复杂操作,比如特别多的lrange等操作; * value很大,比如做商品详情页的cache,可能是需要把大串数据,拼接在一起,作为一个json串,大小可能都几k。 [http_download.redis.io_releases_redis-3.0.0.tar.gz]: http://download.redis.io/releases/redis-3.0.0.tar.gz [http_redis.io]: http://redis.io/ [20190919131134788.png]: /images/20220508/c50e739302f549fdb3ebdf9cd23ba58a.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIyMTcyMTMz_size_16_color_FFFFFF_t_70]: /images/20220508/9958d98df07f4ee6a4e841de3381793b.png [20190919131317580.png]: /images/20220508/a5b09f5122bb4c3485ae8a8a30db75bf.png [20190919131334240.png]: /images/20220508/7831389a404542eeb82a15ce9c95db01.png [20190919131344664.png]: /images/20220508/53a5e65c0b1643ebb446e15b2509aaf0.png [20190919131705114.png]: /images/20220508/a4a2860b28e44caea77020a926343f8d.png
还没有评论,来说两句吧...