实现 KeepAlive + Haproxy + Nginx 七层负载均衡 + 动静分离 ﹏ヽ暗。殇╰゛Y 2022-10-19 01:59 199阅读 0赞 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70][] * 试验环境: * 系统统一使用 CentOS 7.6 * 192.168.2.6 KeepAlived + Haproxy * 192.168.2.10 KeepAlived + Haproxy * 192.168.2.4 Nginx 静态服务器1 * 192.168.2.7 Nginx 静态服务器2 * 192.168.2.11 Nginx + php 动态服务器 * 192.168.2.12 Nginx + php 动态服务器2 * Firewalld 、iptables 以及 SElinux 均为关闭状态 * * **首先配置 Keepalived + Haproxy** * **在 192.168.2.6 和 2.10 上配置好互为主备** * 安装过程请参考:[https://blog.csdn.net/gaofei0428/article/details/104454894?spm=1001.2014.3001.5501][https_blog.csdn.net_gaofei0428_article_details_104454894_spm_1001.2014.3001.5501] * * 添加(VIP)虚拟网卡 * ifconfig ens33:0 192.168.2.168 netmask 255.255.255.255 up * * ### **下面只列出配置文件和运行状态** ### * ### **192.168.2.6** ### * ### vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived ### global\_defs \{ \#notification\_email \{ \# acassen@firewall.loc \# failover@firewall.loc \# sysadmin@firewall.loc \#\} \#notification\_email\_from Alexandre.Cassen@firewall.loc \#smtp\_server 192.168.200.1 \#smtp\_connect\_timeout 30 router\_id LVS\_DEVEL vrrp\_skip\_check\_adv\_addr \#vrrp\_strict vrrp\_garp\_interval 0.001 vrrp\_gna\_interval 0 \} vrrp\_instance DR\_1 \{ state **MASTER** interface ens33 virtual\_router\_id **51** lvs\_sync\_daemon\_inteface ens33 **priority 100** advert\_int 1 authentication \{ auth\_type PASS auth\_pass 1111 \} virtual\_ipaddress \{ **192.168.2.168** \} \} vrrp\_instance DR\_2 \{ state **BACKUP** interface ens33 virtual\_router\_id **53** lvs\_sync\_daemon\_inteface ens33 ** priority 100** advert\_int 1 authentication \{ auth\_type PASS auth\_pass 1111 \} virtual\_ipaddress \{ **192.168.2.169** \} \} * ### **192.168.2.10** ### * **cat /etc/keepalived/keepalived.conf** ! Configuration File for keepalived global\_defs \{ \#notification\_email \{ \# acassen@firewall.loc \# failover@firewall.loc \# sysadmin@firewall.loc \#\} \#notification\_email\_from Alexandre.Cassen@firewall.loc \#smtp\_server 192.168.200.1 \#smtp\_connect\_timeout 30 router\_id **LVS\_DEVEL** vrrp\_skip\_check\_adv\_addr \#vrrp\_strict vrrp\_garp\_interval 0.001 vrrp\_gna\_interval 0 \} vrrp\_instance **DR\_1** \{ state **BACKUP** interface ens33 virtual\_router\_id **51** lvs\_sync\_daemon\_inteface ens33 ** priority 90** advert\_int 1 authentication \{ auth\_type PASS auth\_pass 1111 \} virtual\_ipaddress \{ **192.168.2.168** \} \} vrrp\_instance **DR\_2** \{ state **MASTER** interface ens33 virtual\_router\_id **53** lvs\_sync\_daemon\_inteface ens33 **priority 110** advert\_int 1 authentication \{ auth\_type PASS auth\_pass 1111 \} virtual\_ipaddress \{ **192.168.2.169** \} \} -------------------- * 下面开始安装配置 Haproxy * 官网 : https://www.haproxy.com (访问比较慢) * github : [https://github.com/haproxy/haproxy/releases/][https_github.com_haproxy_haproxy_releases] * ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 1][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 2][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 3][] * **首先要安装 lua5.4.3** * wget [https://www.lua.org/ftp/lua-5.4.3.tar.gz][https_www.lua.org_ftp_lua-5.4.3.tar.gz] * tar xf [lua-5.4.3.tar.gz][https_www.lua.org_ftp_lua-5.4.3.tar.gz] * cd lua-5.4.3 * make linux make install INSTALL\_TOP=/usr/local/lu ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 4][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 5][] * **下面开始安装 haproxy** * * 安装依赖 * yum install vim iotop bc gcc gcc-c++ glibc glibc-devel pcre pcre-devel openssl openssl-devel zip unzip zlib-devel net-tools lrzsz tree ntpdate telnet lsof tcpdump wget libevent libevent-devel bc systemd-devel bash-completion traceroute -y yum install libtermcap-devel ncurses-devel libevent-devel readline-devel -y * * 下载 haproxy ,这里做实验使用最新测试版本,生产环境请选择稳定版本 (2.4) * wget https://github.com/haproxy/haproxy/archive/refs/tags/v2.5-dev1.tar.gz * tar xf v2.5-dev1.tar.gz * * **make -j $(nproc) TARGET=linux-glibc USE\_OPENSSL=1 USE\_ZLIB=1 USE\_LUA=1 LUA\_LIB=/usr/local/lua/lib/ LUA\_INC=/usr/local/lua/include/ USE\_PCRE=1 USE\_SYSTEMD=1** * * **make install PREFIX=/usr/local/haproxy** ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 6][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 7][] ![20210709105343575.png][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 8][] * ** cp /usr/local/haproxy/sbin/haproxy /usr/sbin/ 复制 haproxy 运行脚本** * * **cp ./examples/haproxy.init /etc/init.d/haproxy 复制 haproxy 启动脚本** * **修改如下** * \#!/bin/sh \# chkconfig: - 85 15 \# description: HA-Proxy server \# processname: haproxy \# config: /usr/local/haproxy/etc/haproxy.cfg \# pidfile: /usr/local/haproxy/run/haproxy.pid \# Source function library. if \[ -f /etc/init.d/functions \]; then . /etc/init.d/functions elif \[ -f /etc/rc.d/init.d/functions \] ; then . /etc/rc.d/init.d/functions else exit 0 fi \# Source networking configuration. \#. /etc/sysconfig/network \# Check that networking is up. \#\[ "$NETWORKING" = "no" \] && exit 0 **\# This is our service name** **BASENAME=\`haproxy\`** **BIN=/usr/sbin/haproxy** **CFG=/usr/local/haproxy/etc/haproxy.cfg** **\[ -f $CFG \] || exit 1** **PIDFILE=/usr/local/haproxy/run/haproxy.pid** **LOCKFILE=/usr/local/haproxy/run/haproxy** RETVAL=0 start() \{ quiet\_check if \[ $? -ne 0 \]; then echo "Errors found in configuration file, check it with '$BASENAME check'." return 1 fi echo -n "Starting $BASENAME: " daemon $BIN -D -f $CFG -p $PIDFILE RETVAL=$? echo \[ $RETVAL -eq 0 \] && touch $LOCKFILE return $RETVAL \} stop() \{ echo -n "Shutting down $BASENAME: " killproc $BASENAME -USR1 RETVAL=$? echo \[ $RETVAL -eq 0 \] && rm -f $LOCKFILE \[ $RETVAL -eq 0 \] && rm -f $PIDFILE return $RETVAL \} restart() \{ quiet\_check if \[ $? -ne 0 \]; then echo "Errors found in configuration file, check it with '$BASENAME check'." return 1 fi stop start \} reload() \{ if ! \[ -s $PIDFILE \]; then return 0 fi quiet\_check if \[ $? -ne 0 \]; then echo "Errors found in configuration file, check it with '$BASENAME check'." return 1 fi $BIN -D -f $CFG -p $PIDFILE -sf $(cat $PIDFILE) \} check() \{ $BIN -c -q -V -f $CFG \} quiet\_check() \{ $BIN -c -q -f $CFG \} rhstatus() \{ status $BASENAME \} condrestart() \{ \[ -e $LOCKFILE \] && restart || : \} \# See how we were called. case "$1" in start) start ;; stop) stop ;; restart) restart ;; reload) reload ;; condrestart) condrestart ;; status) rhstatus ;; check) check ;; \*) echo $"Usage: $BASENAME \{start|stop|restart|reload|condrestart|status|check\}" exit 1 esac exit $? ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 9][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 10][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 11][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 12][] * **chmod 755 /etc/init.d/haproxy** 创建运行用户 useradd haproxy -Ms /sbin/nologin mkdir /usr/local/haproxy/etc vim /usr/local/haproxy/etc/haproxy.cfg \#新创建该文件 global log 127.0.0.1 local0 \#log 127.0.0.1 local1 notice \#log loghost local0 info maxconn 4096 chroot /usr/local/haproxy uid 99 gid 99 daemon \#nbproc 1 pidfile /usr/local/haproxy/run/haproxy.pid defaults log global log 127.0.0.1 local3 mode http option httplog option httpclose option dontlognull option forwardfor option redispatch retries 2 maxconn 2000 balance roundrobin stats uri /haproxy-stats timeout connect 5000 timeout client 50000 timeout server 50000 mode http option httpchk GET /index.php frontend http bind \*:80 default\_backend http\_back backend http\_back server Ngxin\_html1 192.168.2.4:80 weight 3 check server Nginx\_html2 192.168.2.7:80 weight 3 check server Nginx\_php1 192.168.2.11:80 weight 3 check server Nginx\_php2 192.168.2.12:80 weight 3 check \#server node1 192.168.179.131:8081 check inter 2000 rise 3 fall 3 weight 30 \# inter 2000 健康检查时间间隔 2 秒 \# rise 3 检测多少次才认为是正常的 \# fall 3 失败多少次才认为是不可用的 \# weight 30 权重 * 关于负载均衡算法 \#source 根据请求源 IP \#static-rr 根据权重 \#leastconn 最少连接者先处理 \#uri 根据请求的 uri \#url\_param 根据请求的 url 参数 \#rdp-cookie 据据 cookie(name)来锁定并哈希每一次请求 \#hdr(name) 根据 HTTP 请求头来锁定每一次 HTTP 请求 \#roundrobin 轮询方式 * mkdir -p /usr/local/haproxy/run 创建 PID 文件存放目录及赋予权限 chown -R nobody. /usr/local/haproxy/ ![20210709110319409.png][] * 配置日志收集 vim /etc/rsyslog.conf $ModLoad imuxsock \# provides support for local system logging (e.g. via logger command) $ModLoad imjournal \# provides access to the systemd journal **$ModLoad imudp $UDPServerRun 514** $WorkDirectory /var/lib/rsyslog $ActionFileDefaultTemplate RSYSLOG\_TraditionalFileFormat $IncludeConfig /etc/rsyslog.d/\*.conf $OmitLocalLogging on $IMJournalStateFile imjournal.state \*.info;mail.none;authpriv.none;cron.none /var/log/messages authpriv.\* /var/log/secure mail.\* -/var/log/maillog cron.\* /var/log/cron \*.emerg :omusrmsg:\* uucp,news.crit /var/log/spooler local7.\* /var/log/boot.log **local3.\* /var/log/haproxy.log local0.\* /var/log/haproxy.log** ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 13][] * 修改完配置文件一定要重启服务或者 reload ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 14][] * 依次启动 keepalive 和 haproxy * systemctl restart keepalived.service /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/etc/haproxy.cfg * * 停止 haproxy * killall haproxy 或者 pkill haproxy * 没有 killall 命令?安装 yum -y install psmisc * * 设置开机启动 * chkconfig --add haproxy chkconfig haproxy on chkconfig --list haproxy ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 15][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 16][] * 然后在另外一台 keepalive 上做同样配置 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 17][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 18][] * 打卡浏览器 访问 VIP ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 19][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 20][] * rr 论循成功 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 21][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 22][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 23][] * 测试漂移 * 首先停止 192.168.2.6 的 keepalive ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 24][] * 观察另外一台 192.168.2.10 的运行情况 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 25][] * 成功漂移,测试浏览网页 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 26][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 27][] * 停止 192.168.2.10 后 依次打开 192.168.2.6 和 2.10 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 28][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 29][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 30][] * 查看 HAproxy 的监控页面 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 31][] * ### **下面配置 nginx 动静分离** ### * 修改配置文件 * vim /usr/local/haproxy/etc/haproxy.cfg global log 127.0.0.1 local0 \#log 127.0.0.1 local1 notice \#log loghost local0 info maxconn 4096 chroot /usr/local/haproxy uid 99 gid 99 daemon \#nbproc 1 pidfile /usr/local/haproxy/run/haproxy.pid defaults log global log 127.0.0.1 local3 mode http option httplog option httpclose option dontlognull option forwardfor option redispatch retries 2 maxconn 2000 balance roundrobin stats uri /haproxy-stats timeout connect 5000 timeout client 50000 timeout server 50000 mode http **frontend http bind \*:80 acl url\_static path\_beg -i /upload/ acl url\_static path\_end -i .jpg .jpeg .gif .png .css .js acl url\_php path\_end -i .php use\_backend imgserver if url\_static use\_backend php if url\_php default\_backend static** backend static \#cookie srv insert nocache balance roundrobin option httpchk GET /index.html server web1 192.168.2.4:80 cookie web1 weight 3 check inter 2000 rise 2 fall 3 server web2 192.168.2.7:80 cookie web1 weight 3 check inter 2000 rise 2 fall 3 backend php balance roundrobin option httpchk GET /index.php server php1 192.168.2.11:80 check inter 2000 fall 3 backend imgserver balance roundrobin server img1 192.168.2.12:80 check inter 2000 fall 3 * 重启服务 * pkill haproxy * /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/etc/haproxy.cfg ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 32][] * 访问静态页面 html 就分配到 2.4 和 2.7 * 访问 动态页面 php 就分配到 2.11 * 访问图片 jpg png等就分配到 2.12 ![20210709135048539.png][] ![20210709135058279.png][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 33][] ![20210709135137260.png][] * 测试 keepalive ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 34][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 35][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 36][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 37][] -------------------- * Zabbix 监控 Haproxy * 根据官方文档提示,在 ** /usr/local/haproxy/etc/haproxy.cfg** 末尾添加 **frontend stats bind \*:8404 stats enable stats uri /stats stats refresh 10s** ![20210710105724284.png][] * 重启 haproxy 服务,然后在 zabbix web 管理界面添加 * 选择 haproxy 主机,左键-------配置-----模板----- * 搜索 [HAProxy by HTTP][] -----更新 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 38][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 39][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 40][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 41][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 42][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 43][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 44][] -------------------- * # Zabbix 监控 KeepAlived # * 编辑 shell 脚本 * mkdir /etc/zabbix/scripts * vim /etc/zabbix/scripts/check\_keepalived.sh \#!/bin/bash if \[\[ \`ip address show dev ens33 |grep -E "192.168.2.168/32|192.168.2.169/32"|wc -l\` == "0" \]\]; then echo "0"; else echo "1"; fi * chmod +x /etc/zabbix/scripts/check\_keepalived.sh ![20210710202541337.png][] * 在 配置文件末尾添加 * tail -n2 /etc/zabbix/zabbix\_agentd.conf UnsafeUserParameters=1 UserParameter=check\_keepalived,/etc/zabbix/scripts/check\_keepalived.sh ![20210710202712908.png][] * 重启 zabbix-agent 服务 * 然后在 web 界面添加 监控项 * 配置 ----主机----监控项----创建监控项 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 45][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 46][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 47][] * 创建触发器 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 48][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 49][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 50][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 51][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 52][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 53][] * 这里的问题和恢复表达式的值,就是基于 shell 脚本的判断值 * 添加 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 54][] * 测试 依次停止 2台服务器的 keepalive ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 55][] * 查看 报表-----动作日志 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 56][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 57][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 58][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 59][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 60][] * 停止另外一台 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 61][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 62][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 63][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 64][] * 依次恢复 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 65][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 66][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 67][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 68][] -------------------- * 静态页面缓存 * 在配置文件中添加以下内容,开启缓存 * * cache section cache 声明cache区域,分配一个共享缓存名称。 total-max-size 定义使用内存大小,这个块将会被分为1kb的小块,它的最大值为4095 max-object-size 定义缓存对象的大小,不能大于total-max-size的一般。如果没有定义,将会等于256个cache size。当objects的大小大于该值后将不会被缓存 max-age 定义缓存过期时间,默认60s, * ## proxy section ## * http-request cache-use \[\{if | unless\} \] 试着去发送中缓存的对象。 * http-response cache-store \[\{if | unless\} \] 将http-response缓存起来, * * http-request cache-use foobar http-response cache-store foobar cache foobar total-max-size 4 max-age 240 ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 69][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 70][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 71][] ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 72][] [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70]: /images/20221005/7ec6f5d4f58644e7b92e56c1ce953629.png [https_blog.csdn.net_gaofei0428_article_details_104454894_spm_1001.2014.3001.5501]: https://blog.csdn.net/gaofei0428/article/details/104454894?spm=1001.2014.3001.5501 [https_github.com_haproxy_haproxy_releases]: https://github.com/haproxy/haproxy/releases/ [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 1]: /images/20221005/18032605711f451e9d26dd4776d4838a.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 2]: /images/20221005/af9884e208694160b3342e52721ac7b6.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 3]: /images/20221005/11156bbfd9424db2896c016ce8219f22.png [https_www.lua.org_ftp_lua-5.4.3.tar.gz]: https://www.lua.org/ftp/lua-5.4.3.tar.gz [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 4]: /images/20221005/64053e7421f542b2b448eb9b2534fcca.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 5]: /images/20221005/1ea4e75f9f524a17967d592cfe4cab07.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 6]: /images/20221005/c429d34961af4a68a0a8cf18f0ef5498.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 7]: /images/20221005/a8edfef5afd94b26b3d31d70f021f039.png [20210709105343575.png]: /images/20221005/5791551c6be3439fb48e92ebd7d85e0c.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 8]: /images/20221005/6b979c8a9ff14e819c1072f9d8ce2408.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 9]: /images/20221005/3917997a5ae34bcfbecda8a21e1794c5.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 10]: /images/20221005/abf886a54ae649b3b4536cb370883eff.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 11]: /images/20221005/80c8fcaeda7b4c308c78455ac0da3ba3.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 12]: /images/20221005/f18114da2d0141b79b6afc004f4d9d11.png [20210709110319409.png]: /images/20221005/2758f49c365d4b84b41cf2e0ff209022.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 13]: /images/20221005/8b22e1b96b44437285fcc68f1c2cbb19.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 14]: /images/20221005/73eb301ed51b4f058879b799d434e11b.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 15]: /images/20221005/c532f9fadb964c3a96cd1cc15a7da0ee.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 16]: /images/20221005/194a29c5f5df4549896dcf86fc5e4009.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 17]: /images/20221005/2351d8fe7ee84ae29efcd9029943222e.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 18]: /images/20221005/66d5d222aec940e9bc729424fdab77c3.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 19]: /images/20221005/5ded1901f69e41469aa7e46e6f3da704.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 20]: /images/20221005/da7b346197a94080a26df47e23bc4c34.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 21]: /images/20221005/960fe1df220f41a0980a304925b12128.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 22]: /images/20221005/f042119444e9463a8c7d390335b33981.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 23]: /images/20221005/40b7f57158aa4d82bb4f009b1daebde2.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 24]: /images/20221005/3633b970fdc7428bba2320308f8fb4b0.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 25]: /images/20221005/953977d34f03474db2453817dcbdb03f.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 26]: /images/20221005/abe6daecbefd424b8d48b6bd104470e9.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 27]: /images/20221005/fa1e96065845469fb8eed907829664a3.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 28]: /images/20221005/b4ca466d9bf14f19b18ae5a809163948.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 29]: /images/20221005/60803a03d7b6436a9921b8faff06e350.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 30]: /images/20221005/1992ae6641c44df4b8743ead6aa15cd5.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 31]: /images/20221005/562e2cc28eca4b1e8d710f1a7afaafce.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 32]: /images/20221005/4b63eed123884b6881c34ab691fdb8ba.png [20210709135048539.png]: /images/20221005/27ab1e08e1574364967f646ec1add225.png [20210709135058279.png]: /images/20221005/ccc55753043848d394b0723c43149f09.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 33]: /images/20221005/ce721c3d339b4983b000c59e9138d334.png [20210709135137260.png]: /images/20221005/acb63a3dc5504db5afbd518c37ccc758.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 34]: /images/20221005/71d4f83b912f4ceab4688a76ac736fdf.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 35]: /images/20221005/53c0e092ccbe46c7a2876ae87499f4c3.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 36]: /images/20221005/89fec214f0974918a493281834ad6882.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 37]: /images/20221005/ec2e046c019b4eb3b0d6247ccaaa8553.png [20210710105724284.png]: /images/20221005/b1624643a458455baa80cb6e17f0b2de.png [HAProxy by HTTP]: http://192.168.2.3/zabbix/templates.php?form=update&templateid=10309 [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 38]: /images/20221005/1036fd946f71463492468571daf950d6.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 39]: /images/20221005/933b511c0a8941b592fbd60f838c4a62.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 40]: /images/20221005/1530260c05f24dae8274cc6c5c818cb4.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 41]: /images/20221005/cf8a7e05d96c4563805bf2b64d93ea71.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 42]: /images/20221005/ebeaa9d3cb374600a847ec2da83e88c2.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 43]: /images/20221005/b616ad3a16a8443ebe4994392dd33a48.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 44]: /images/20221005/013f6d2951ac4220998981c2f1c17b67.png [20210710202541337.png]: /images/20221005/da95224ae5d3408d8ef84faf77c466d3.png [20210710202712908.png]: /images/20221005/19a4a7337c684d94b2372c02818020d5.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 45]: /images/20221005/4014503e99f64980846e3a59d541bb13.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 46]: /images/20221005/9d29db5653554c419c8dae227c9f5723.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 47]: /images/20221005/05a547e1a8f541859def094c65b398e4.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 48]: /images/20221005/b272328919cf441bb46e3052cdbd1389.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 49]: /images/20221005/21e7aa8eae544c9195ebb0ff419af77e.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 50]: /images/20221005/1bc61689d7a14a8d8a7eb5206633f8e5.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 51]: /images/20221005/a3bab296198044178e80321ba205db02.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 52]: /images/20221005/a67f8cec17dd44959fc0e6bc3ccf5e08.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 53]: /images/20221005/434999c83de442b08c148492e5320cbd.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 54]: /images/20221005/a7e3b16fb93e436795fac9cc16223763.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 55]: /images/20221005/bad59cdda6b84febbc8fd8253737b2f6.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 56]: /images/20221005/78d65ac61a564eb1b622542787d1b7e4.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 57]: /images/20221005/62adca2e442146f8bebb8931e45123e6.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 58]: /images/20221005/04aa5c3160c44581a12a3033a6bc3790.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 59]: /images/20221005/93067f68aa8c4370b375fc3659bda1aa.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 60]: /images/20221005/e18316acaba04626afde6a0e638642af.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 61]: /images/20221005/94e712d8792c486fbf4ffa41e5a96a12.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 62]: /images/20221005/3c7c0474f3da4a78ae0255a957e1587d.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 63]: /images/20221005/9f297aa07950427e8df7f819f901785e.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 64]: /images/20221005/5edd2d48e194443aa670ace619f734aa.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 65]: /images/20221005/0b14ba69fa9c4ab285ec4e3303b5d188.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 66]: /images/20221005/c9abda3e7e314ecda23c3bceed4b5430.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 67]: /images/20221005/eeea82efe9c54b019c04db83218c7a6c.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 68]: /images/20221005/d7a6519e9dfb47ac8e786ff6e179874c.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 69]: /images/20221005/3bfd21473dac4605b40945d313e7bd53.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 70]: /images/20221005/25f3b1f835a14838baffd2110ac4bee0.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 71]: /images/20221005/4baa83832d83425681482821dc8695ed.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2dhb2ZlaTA0Mjg_size_16_color_FFFFFF_t_70 72]: /images/20221005/48bb6c681946451db7b0b5cc11d391b3.png
还没有评论,来说两句吧...