Corosync+Pacemaker+NFS+Mysql高可用集群部署

一时失言乱红尘 2022-05-17 12:53 255阅读 0赞

**Corosync+Pacemaker+NFS+Mysql****高可用集群部署（****使用资源管理工具crmsh配置）**

**框架：crmsh(Corosync+pacemaker)+nfs+mysql**

集群节点1:192.168.88.132 cen7.field.com

集群节点2:192.168.88.133 node2.field.com

集群节点3:192.168.88.135 node3.field.com

vip: 192.168.88.188  资源代理：ocf:heartbeat:IPaddr

nfs服务器：node3.field.com  资源代理：ocf:heartbeat:Filesystem

mysql服务器：cen7.field.com node1.field.com  资源代理：lsb:mysqld

**配置集群的前提**

**（1****）、时间同步；**

**（2****）、基于当前正在使用的主机名互相访问；**

**（3****）、是否会用到仲裁设备；**

**参考 ：[《使用资源管理工具pcs配置Corosync+pacemaker》][pcs_Corosync_pacemaker]**

**一、配置****NFS****服务器：共享目录作为****mysql****的****datadir**

该步骤中：主要通过创建逻辑卷并设置开机自动挂载到nfs共享目录，用于存放mysql数据库文件，其所有者必须是mysql用户和mysql组。

**1****、创建逻辑卷**

**1****）、fdisk****创建物理卷分区**

\[root@node3 ~\]\# fdisk  /dev/sda

欢迎使用 fdisk (util-linux 2.23.2)。

更改将停留在内存中，直到您决定将更改写入磁盘。

使用写入命令前请三思。

命令(输入 m 获取帮助)：p

磁盘 /dev/sda：32.2 GB, 32212254720 字节，62914560 个扇区

Units = 扇区 of 1 \* 512 = 512 bytes

扇区大小(逻辑/物理)：512 字节 / 512 字节

I/O 大小(最小/最佳)：512 字节 / 512 字节

磁盘标签类型：dos

磁盘标识符：0x00057c95

设备 Boot      Start         End      Blocks   Id  System

/dev/sda1   \*        2048     1075199      536576   83  Linux

/dev/sda2         1075200     3174399     1049600   82  Linux swap / Solaris

/dev/sda3         3174400    36728831    16777216   83  Linux

命令(输入 m 获取帮助)：n

Partition type:

p   primary (3 primary, 0 extended, 1 free)

e   extended

Select (default e): p

已选择分区 4

起始 扇区 (36728832-62914559，默认为 36728832)：

将使用默认值 36728832

Last 扇区, +扇区 or +size\{K,M,G\} (36728832-62914559，默认为 62914559)：+2G

分区 4 已设置为 Linux 类型，大小设为 2 GiB

命令(输入 m 获取帮助)：t

分区号 (1-4，默认 4)：4

Hex 代码(输入 L 列出所有代码)：8e  -->说明：用作物理卷的分区id为8e

已将分区“Linux”的类型更改为“Linux LVM”

命令(输入 m 获取帮助)：w

The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: 设备或资源忙.

The kernel still uses the old table. The new table will be used at

the next reboot or after you run partprobe(8) or kpartx(8)

正在同步磁盘。

**2****）、使用partprobe****让内核识别新建的分区**

\[root@node3 ~\]\# partprobe /dev/sda

**3****）、查看新建立的分区**

\[root@node3 ~\]\# fdisk -l

磁盘 /dev/sda：32.2 GB, 32212254720 字节，62914560 个扇区

Units = 扇区 of 1 \* 512 = 512 bytes

扇区大小(逻辑/物理)：512 字节 / 512 字节

I/O 大小(最小/最佳)：512 字节 / 512 字节

磁盘标签类型：dos

磁盘标识符：0x00057c95

设备 Boot      Start         End      Blocks   Id  System

/dev/sda1   \*        2048     1075199      536576   83  Linux

/dev/sda2         1075200     3174399     1049600   82  Linux swap / Solaris

/dev/sda3         3174400    36728831    16777216   83  Linux

/dev/sda4        36728832    40923135     2097152   8e  Linux LVM  -->标记为LVM

**4****）、创建物理卷**

\[root@node3 ~\]\# pvcreate /dev/sda4

Physical volume "/dev/sda4" successfully created.

**5****）、创建卷组 **

\[root@node3 ~\]\# vgcreate myvg /dev/sda4

Volume group "myvg" successfully created

**6****）、创建大小为500M****的逻辑卷 **

\[root@node3 ~\]\# lvcreate -L 500M -n lv\_mydata myvg

Logical volume "lv\_mydata" created.

**7****）、格式化逻辑卷**

\[root@node3 ~\]\# mke2fs -j /dev/myvg/lv\_mydata

mke2fs 1.42.9 (28-Dec-2013)

文件系统标签=

OS type: Linux

块大小=1024 (log=0)

分块大小=1024 (log=0)

Stride=0 blocks, Stripe width=0 blocks

128016 inodes, 512000 blocks

25600 blocks (5.00%) reserved for the super user

第一个数据块=1

Maximum filesystem blocks=67633152

63 block groups

8192 blocks per group, 8192 fragments per group

2032 inodes per group

Superblock backups stored on blocks:

8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Allocating group tables: 完成

正在写入inode表: 完成

Creating journal (8192 blocks): 完成

Writing superblocks and filesystem accounting information: 完成

**2****、挂载共享存储**

**1****）、创建挂载的目录/mydata****作为nfs****共享存储介质**

\[root@node3 ~\]\# mkdir /mydata

**2****）、设置开机自动挂载**

\[root@node3 ~\]\# vim /etc/fstab

/dev/myvg/lv\_mydata     /mydata    ext3    defaults    0 0

**3****）、挂载测试挂载成功与否**

\[root@node3 ~\]\# mount -a

\[root@node3 ~\]\# ls /mydata/

lost+found

**3****、启动****nfs****服务并配置共享存储**

**1****）、启动nfs****服务**

\[root@node3 ~\]\# systemctl start nfs.service

\[root@node3 ~\]\# systemctl status nfs.service

● nfs-server.service - NFS server and services

Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled)

Active: active (exited) since 六 2018-08-04 03:43:47 CST; 6s ago

Process: 1442 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)

Process: 1438 ExecStartPre=/bin/sh -c /bin/kill -HUP \`cat /run/gssproxy.pid\` (code=exited, status=0/SUCCESS)

Process: 1436 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)

Main PID: 1442 (code=exited, status=0/SUCCESS)

CGroup: /system.slice/nfs-server.service

8月 04 03:43:47 node3.field.com systemd\[1\]: Starting NFS server and services...

8月 04 03:43:47 node3.field.com systemd\[1\]: Started NFS server and services.

**2****）、添加共享存储文件，用于存放mysql****数据的目录**

\[root@node3 ~\]\# vim /etc/exports

\#/www/hadocs 192.168.88.0/24(rw)

/mydata 192.168.88.0/24(rw,no\_root\_squash)

\#该网段的主机可读写，不需root用户

**3****）、编辑测试页**

\[root@node3 ~\]\#  echo "<h1>Test page on NFS Service</h1>" >/mydata/index.html

**4****）、重启nfs****服务并修改共享目录属组属主为mysql****，注意：此步骤为必须操作**

\[root@node3 ~\]\# systemctl restart nfs.service

\[root@node3 ~\]\# chown -R mysql.mysql /mydata

chown: 无效的用户: "mysql.mysql"

\[root@node3 ~\]\# groupadd -g 306  -r mysql

\[root@node3 ~\]\# useradd -g mysql -u 306 -s /sbin/nologin mysql

\[root@node3 ~\]\# id mysql

uid=306(mysql) gid=306(mysql) 组=306(mysql)

\[root@node3 ~\]\# chown -R mysql.mysql /mydata

**二、编译、安装、配置、测试****Mariadb**

该步骤主要分两大步：cen7节点安装和node2节点安

**（一）、****cen7****节点上的编译、安装、配置、测试****Mariadb**

**1****、安装前配置**

**1****）、创建mysql****用户、组和挂载目录**

\[root@cen7 ~\]\# groupadd -g 306 -r mysql

\[root@cen7 ~\]\# useradd -u 306 -g mysql -r mysql

\[root@cen7 ~\]\# mkdir /mydata

**2****）、挂载NFS****文件系统并确认**

\[root@cen7 ~\]\# mount -t nfs 192.168.88.135:/mydata /mydata

\[root@cen7 ~\]\# mount |grep nfs

sunrpc on /var/lib/nfs/rpc\_pipefs type rpc\_pipefs (rw,relatime)

192.168.88.135:/mydata on /mydata type nfs4 (rw,relatime,vers=4.1,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.88.132,local\_lock=none,addr=192.168.88.135)

\[root@cen7 ~\]\# ls /mydata

index.html  lost+found

**3****）、以mysql****用户身份创建data****目录，作为初始化时的数据存放目录（datadir****）**

注意：mysql初始化只需在集群节点中的一个进行一次，后续集群会自动挂载配置，另一台机mariadb不需初始化，所以不用以mysql用户创建data目录

\[root@cen7 ~\]\# su - mysql

su: 警告：无法更改到 /home/mysql 目录: 没有那个文件或目录

\-bash-4.2$ cd /mydata

\-bash-4.2$ mkdir data

\-bash-4.2$ exit

登出

**2****、源码编译安装****Mariadb**

说明：此处采用10.0以上版本可能会因为c++版本低编译无法通过，笔者采用mariadb-5.5.61.tar.gz安装包

\[root@cen7 ~\]\# ls

anaconda-ks.cfg                     mariadb-5.5.61.tar.gz    python-parallax-1.0.0a1-7.1.noarch.rpm

crmsh-3.0.0-6.2.noarch.rpm          ~None                      python-pssh-2.3.1-7.3.noarch.rpm

crmsh-scripts-3.0.0-6.2.noarch.rpm  pssh-2.3.1-7.3.noarch.rpm

**1****）、编译安装mariadb**

\[root@cen7 ~\]\# tar xf mariadb-5.5.61.tar.gz -C /usr/local

\[root@cen7 ~\]\# cd /usr/local/

\[root@cen7 local\]\# ln -sv mariadb-5.5.61 mysql

"mysql" -> "mariadb-5.5.61"

\[root@cen7 local\]\# cd mysql

注意：修改文件属主属组为mysql

\[root@cen7 mysql\]\# chown -R mysql:mysql .

\[root@cen7 mysql\]\# cmake . -DCMAKE\_INSTALL\_PREFIX=/usr/local/mysql -DMYSQL\_DATADIR=/mydata/data/ -DSYSCONFDIR=/etc  -DWITH\_INNOBASE\_STORAGE\_ENGINE=1 -DWITH\_ARCHIVE\_STORAGE\_ENGINE=1 -DWITH\_BLACKHOLE\_STORAGE\_ENGINE=1 -DWITH\_READLINE=1 -DWITH\_SSL=system -DWITH\_ZLIB=system -DWITH\_LIBWRAP=0 -DMYSQL\_UNIX\_ADDR=/tmp/mysql.sock -DDEFAULT\_CHARSET=utf8 -DDEFAULT\_COLLATION=utf8\_general\_ci

\[root@cen7 mysql\]\# make && make install

\[root@cen7 mysql\]\# chown -R mysql:mysql  .

**2****）、初始化mysql****数据库：只需在该节点进行初始化，后续会自动挂载。**

\[root@cen7 mysql\]\# scripts/mysql\_install\_db --user=mysql --datadir=/mydata/data

Installing MariaDB/MySQL system tables in '/mydata/data' ...

180803  6:02:53 \[Note\] ./bin/mysqld (mysqld 5.5.61-MariaDB) starting as process 60647 ...

Filling help tables...

180803  6:02:55 \[Note\] ./bin/mysqld (mysqld 5.5.61-MariaDB) starting as process 60656 ...

To start mysqld at boot time you have to copy

support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !

To do so, start the server, then issue the following commands:

'./bin/mysqladmin' -u root password 'new-password'

'./bin/mysqladmin' -u root -h cen7.field.com password 'new-password'

Alternatively you can run:

'./bin/mysql\_secure\_installation'

which will also give you the option of removing the test

databases and anonymous user created by default.  This is

strongly recommended for production servers.

See the MariaDB Knowledgebase at http://mariadb.com/kb or the

MySQL manual for more instructions.

You can start the MariaDB daemon with:

cd '.' ; ./bin/mysqld\_safe --datadir='/mydata/data'

You can test the MariaDB daemon with mysql-test-run.pl

cd './mysql-test' ; perl mysql-test-run.pl

Please report any problems at http://mariadb.org/jira

The latest information about MariaDB is available at http://mariadb.org/.

You can find additional information about the MySQL part at:

http://dev.mysql.com

Consider joining MariaDB's strong and vibrant community:

https://mariadb.org/get-involved/

\[root@cen7 mysql\]\# chown -R root  .

**3****）、复制my-large.cnf****为mysql****主配置文件并修改**

\[root@cen7 mysql\]\#  cp support-files/my-large.cnf  /etc/my.cnf

cp：是否覆盖"/etc/my.cnf"？ y

\[root@cen7 mysql\]\# vim /etc/my.cnf

\#thread\_concurrency=CPU\*2

thread\_concurrency = 6

\#指定mysql数据文件的存放位置：

datadir = /mydata/data

**4****）、配置使用service****管理mysqld****服务并关闭开机自启动，用CRM****控制mysql****启停**

\[root@cen7 mysql\]\# cp support-files/mysql.server  /etc/rc.d/init.d/mysqld

\[root@cen7 mysql\]\# chkconfig --add mysqld

\[root@cen7 mysql\]\# chkconfig mysqld off

\[root@cen7 mysql\]\# service mysqld start

env: /etc/init.d/mysqld: 权限不够

\[root@cen7 mysql\]\# chmod +x /etc/rc.d/init.d/mysqld

**3****、启动****mysql****服务并确认状态**

\[root@cen7 mysql\]\# service mysqld start

Starting MariaDB.180803 06:08:41 mysqld\_safe Logging to '/mydata/data/cen7.field.com.err'.

180803 06:08:41 mysqld\_safe Starting mysqld daemon with databases from /mydata/data

. SUCCESS!

\[root@cen7 mysql\]\# ps -ef | grep 3306

mysql    61180 60874  1 06:08 pts/0    00:00:00 /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/mydata/data --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=/mydata/data/cen7.field.com.err --pid-file=/mydata/data/cen7.field.com.pid --socket=/tmp/mysql.sock --port=3306

root     61233  3454  2 06:09 pts/0    00:00:00 grep --color=auto 3306

\[root@cen7 mysql\]\# service mysqld stop

Shutting down MariaDB..... SUCCESS!

**4****、针对用户操作习惯的更改**

使用源码包编译安装mariadb使用方法与用户习惯有所出入，通过创建链接，配置环境变量可以使mysql符合用户操作习惯，此步可以不进行。

**1****）、输出mysql****的man****手册至man****命令的查找路径**

\[root@cen7 mysql\]\# vim /etc/man.config

MANPATH  /usr/local/mysql/man

**2****）、将mysql****的头文件链接至系统头文件路径/usr/include****：**

\[root@cen7 mysql\]\# ln -sv /usr/local/mysql/include  /usr/include/mysql

"/usr/include/mysql" -> "/usr/local/mysql/include"

**3****）、输出mysql****的库文件给系统库查找路径**

\[root@cen7 mysql\]\# echo '/usr/local/mysql/lib' > /etc/ld.so.conf.d/mysql.conf

**4****）、让系统重新载入系统库**

\[root@cen7 mysql\]\# ldconfig

**5****）、编辑/etc/profile****添加mysql****环境，让系统所有用户可以直接使用mysql****的相关命令:**

\[root@cen7 mysql\]\# vim /etc/profile

MYSQL\_HOME=/usr/local/mysql

PATH=$PATH:$MYSQL\_HOME/bin

**6****）、使配置生效**

\[root@cen7 mysql\]\# . /etc/profile

**5****、验证****mysql****是否安装成功：启动****mysql****，连接****mysql****客户端。**

\[root@cen7 mysql\]\# service mysqld start

Starting MariaDB.180803 06:12:02 mysqld\_safe Logging to '/mydata/data/cen7.field.com.err'.

180803 06:12:02 mysqld\_safe Starting mysqld daemon with databases from /mydata/data

.. SUCCESS!

\[root@cen7 mysql\]\# mysql

Welcome to the MariaDB monitor.  Commands end with ; or \\g.

Your MariaDB connection id is 2

Server version: 5.5.61-MariaDB Source distribution

Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.

MariaDB \[(none)\]> quit

Bye

**6****、复制配置文件到****node2****节点：****mysql****配置文件各节点相同**

\[root@cen7 corosync\]\# scp /etc/my.cnf node2:/etc/my.cnf

\[root@cen7 corosync\]\# service mysqld stop

Shutting down MariaDB.... SUCCESS!

**7****、卸载共享目录：最后必须卸载共享目录**

\[root@cen7 corosync\]\# umount /mydata

**（二）、****node2.field.com****上的配置****Mariadb**

因node2节点上mariadb安装配置同cen7节点，只是不需要再次初始化数据库，此处不再针对操作做出说明。

**以下是操作步骤及代码：**

\[root@node2 ~\]\# tar xf mariadb-5.5.61.tar.gz -C /usr/local/

\[root@node2 ~\]\# groupadd -g 306 -r mysql

\[root@node2 ~\]\# useradd -u 306 -g mysql -r mysql

\[root@node2 ~\]\# mkdir /mydata

\[root@node2 ~\]\# cd /usr/local/

\[root@node2 local\]\# ln -sv mariadb-5.5.61 mysql

"mysql" -> "mariadb-5.5.61"

\[root@node2 local\]\# cd mysql

\[root@node2 mysql\]\# chown -R mysql:mysql  .

\[root@node2 mysql\]\# yum install cmake openssl-devel zlib-devel ncurses-devel bision gcc gcc-c++ -y

\[root@node2 mysql\]\# cmake . -DCMAKE\_INSTALL\_PREFIX=/usr/local/mysql -DMYSQL\_DATADIR=/mydata/data/ -DSYSCONFDIR=/etc  -DWITH\_INNOBASE\_STORAGE\_ENGINE=1 -DWITH\_ARCHIVE\_STORAGE\_ENGINE=1 -DWITH\_BLACKHOLE\_STORAGE\_ENGINE=1 -DWITH\_READLINE=1 -DWITH\_SSL=system -DWITH\_ZLIB=system -DWITH\_LIBWRAP=0 -DMYSQL\_UNIX\_ADDR=/tmp/mysql.sock -DDEFAULT\_CHARSET=utf8 -DDEFAULT\_COLLATION=utf8\_general\_ci

\[root@node2 mysql\]\# make && make install

\[root@node2 mysql\]\# chown -R root:mysql  .

\[root@node2 mysql\]\#  cp support-files/mysql.server  /etc/rc.d/init.d/mysqld

\[root@node2 mysql\]\# chkconfig --add mysqld

\[root@node2 mysql\]\# chkconfig mysqld off

\[root@node2 mysql\]\# vim /etc/man.config

\[root@node2 mysql\]\# ln -sv /usr/local/mysql/include  /usr/include/mysql

"/usr/include/mysql" -> "/usr/local/mysql/include"

\[root@node2 mysql\]\# echo '/usr/local/mysql/lib' > /etc/ld.so.conf.d/mysql.conf

\[root@node2 mysql\]\# ldconfig

\[root@node2 mysql\]\# vim /etc/profile

MYSQL\_HOME=/usr/local/mysql

PATH=$PATH:$MYSQL\_HOME/bin

\[root@node2 mysql\]\# vim /etc/profile

\[root@node2 mysql\]\# . /etc/profile

\[root@node2 mysql\]\# systemctl start mysqld.service

Failed to start mysqld.service: Unit not found.

\[root@node2 mysql\]\# systemctl start mysql.service

Failed to start mysql.service: Unit not found.

\[root@node2 mysql\]\# chmod +x /etc/rc.d/init.d/mysqld

\[root@node2 mysql\]\# service mysqld start

Starting MariaDB.180803 06:49:01 mysqld\_safe Logging to '/mydata/data/node2.field.com.err'.

180803 06:49:01 mysqld\_safe Starting mysqld daemon with databases from /mydata/data

/usr/local/mysql/bin/mysqld\_safe\_helper: Can't create/write to file '/mydata/data/node2.field.com.err' (Errcode: 2)

ERROR!

\#此报错是因为未挂载mysql数据存放目录

**挂载/mydata****目录，验证Mariadb****是否安装成功**

\[root@node2 corosync\]\# mount -t nfs 192.168.88.135:/mydata /mydata

\[root@node2 corosync\]\# service mysqld start

Starting MariaDB.180803 07:04:16 mysqld\_safe Logging to '/mydata/data/node2.field.com.err'.

180803 07:04:16 mysqld\_safe Starting mysqld daemon with databases from /mydata/data

SUCCESS!

\[root@node2 corosync\]\# mysql

Welcome to the MariaDB monitor.  Commands end with ; or \\g.

Your MariaDB connection id is 2

Server version: 5.5.61-MariaDB Source distribution

Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.

MariaDB \[(none)\]>

MariaDB \[(none)\]> quit

Bye

\#可以看到：挂载/mydata目录后，Mariadb客户端接入成功。

注意：测试成功后关闭mariadb并卸载共享目录

\[root@node2 corosync\]\# service mysqld stop

\[root@node2 corosync\]\# umount /mydata

**三、使用****crmsh****配置****Corosync+Pacemaker**

**1****、将集群节点加入****/etc/ansible/hosts****中，便于管理**

\#或使用systemctl start|stop|restart xxxx

\[root@cen7 corosync\]\# vim /etc/ansible/hosts

\[hamysql\]

192.168.88.132

192.168.88.133

**2****、启动****corosync****和****pacemaker****，删除此前配置**

\[root@cen7 mysql\]\# ansible hamysql -m service -a 'name=corosync state=started'

\[root@cen7 mysql\]\# ansible hamysql -m service -a 'name=pacemaker state=started'

\[root@cen7 mysql\]\# crm status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 04:33:13 2018

Last change: Sat Aug  4 01:03:41 2018 by root via cibadmin on cen7.field.com

2 nodes configured

3 resources configured

Online: \[ cen7.field.com node2.field.com \]

Full list of resources:

Resource Group: mycluster

mysqlip  (ocf::heartbeat:IPaddr):  Started node2.field.com

mynfs     (ocf::heartbeat:Filesystem):   Stopped

mysqld   (lsb:mysqld):    Stopped

使用crm交互模式：删除此前定义的资源

\[root@cen7 mysql\]\# crm

crm(live)\# resource

crm(live)resource\# help

\#删除资源之前需要先停止资源，各命令用法可以使用“help \[command\]”查看

crm(live)resource\# help stop

crm(live)resource\# show

Resource Group: mycluster

mysqlip  (ocf::heartbeat:IPaddr):  Started

mynfs     (ocf::heartbeat:Filesystem):   Stopped

mysqld   (lsb:mysqld):    Stopped

crm(live)resource\# stop mysqlip

crm(live)resource\# show

Resource Group: mycluster

mysqlip  (ocf::heartbeat:IPaddr):  Stopped (disabled)

mynfs     (ocf::heartbeat:Filesystem):   Stopped

mysqld   (lsb:mysqld):    Stopped

crm(live)resource\# cd

crm(live)\# configure

crm(live)configure\# help delete

crm(live)configure\# delete mycluster

crm(live)configure\# delete mysqlip

crm(live)configure\# delete mynfs

crm(live)configure\# delete mysqld

crm(live)configure\# verify

ERROR: Warnings found during check: config may not be valid

\#注意：如果有无意义的配置会报如上error，使用show查看配置，如此处的cluster-name。

\#直接使用edit编辑删除无效配置:wq保存退出即可。

crm(live)configure\# show

node 1: cen7.field.com \\

attributes standby=off

node 2: node2.field.com \\

attributes standby=off

property cib-bootstrap-options: \\

have-watchdog=false \\

dc-version=1.1.18-11.el7\_5.3-2b07d5c5a9 \\

cluster-infrastructure=corosync \\

cluster-name=mycluster \\

stonith-enabled=false \\

default-resource-stickiness=0 \\

no-quorum-policy=ignore

crm(live)configure\# edit

crm(live)configure\# verify

\#可以看到，删除无效配置后verify不会报ERROR

crm(live)configure\# show

node 1: cen7.field.com \\

attributes standby=off

node 2: node2.field.com \\

attributes standby=off

property cib-bootstrap-options: \\

have-watchdog=false \\

dc-version=1.1.18-11.el7\_5.3-2b07d5c5a9 \\

cluster-infrastructure=corosync \\

stonith-enabled=false \\

no-quorum-policy=ignore

crm(live)configure\# commit

crm(live)configure\# quit

bye

**3****、编辑配置文件****corosync.conf**

\[root@cen7 corosync\]\# cp corosync.conf corosync.conf.bak080222

\[root@cen7 corosync\]\# cat corosync.conf.bak080215 >corosync.conf

\[root@cen7 corosync\]\# vim corosync.conf

\[root@cen7 corosync\]\# grep -v '^\[\[:space:\]\]\*\#' corosync.conf

totem \{

version: 2

crypto\_cipher: aes128

crypto\_hash: sha1

secauth: on

interface \{

ringnumber: 0

bindnetaddr: 192.168.88.0

mcastaddr: 239.188.1.188

mcastport: 5405

ttl: 1

nodelist \{

node \{

ring0\_addr: 192.168.88.132

nodeid: 1

node \{

ring0\_addr: 192.168.88.133

nodeid: 2

logging \{

fileline: off

to\_stderr: no

to\_logfile: yes

logfile: /var/log/cluster/corosync.log

to\_syslog: no

debug: off

timestamp: on

logger\_subsys \{

subsys: QUORUM

debug: off

quorum \{

provider: corosync\_votequorum

**4****、生成节点间通信时用到的认证密钥文件，并复制到****node2****节点**

\[root@cen7 corosync\]\# corosync-keygen

Corosync Cluster Engine Authentication key generator.

Gathering 1024 bits for key from /dev/random.

Press keys on your keyboard to generate entropy.

Press keys on your keyboard to generate entropy (bits = 920).

Press keys on your keyboard to generate entropy (bits = 1000).

Writing corosync key to /etc/corosync/authkey.

\[root@cen7 corosync\]\# ls

authkey  corosync.conf  corosync.conf.bak080215  corosync.conf.bak080222  corosync.conf.example  corosync.conf.example.udpu  corosync.xml.example  uidgid.d

\[root@cen7 corosync\]\#  scp -p authkey corosync.conf node2:/etc/corosync/

authkey                                                                                                                                                      100%  128    44.1KB/s   00:00

corosync.conf

**5****、启动****corosync****和****pacemaker****，并确认状态是否正常****                                                                                        **

\[root@cen7 corosync\]\# ansible hamysql -m service -a 'name=corosync state=started enabled=yes'

\[root@cen7 corosync\]\# ansible hamysql -m service -a 'name=pacemaker state=started enabled=yes'

**1****）、查看是否能够正常发出初始化成员节点通知**

\[root@cen7 corosync\]\# grep  TOTEM  /var/log/messages|tail

Aug  3 06:25:24 cen7 corosync\[62060\]: \[TOTEM \] A new membership (192.168.88.132:348) was formed. Members

Aug  3 06:25:27 cen7 corosync\[62060\]: \[TOTEM \] A new membership (192.168.88.132:352) was formed. Members

Aug  3 06:25:29 cen7 corosync\[62060\]: \[TOTEM \] A new membership (192.168.88.132:356) was formed. Members joined: 3

Aug  3 06:26:16 cen7 corosync\[62060\]: \[TOTEM \] A processor failed, forming new configuration.

Aug  3 06:26:16 cen7 corosync\[62060\]: \[TOTEM \] A new membership (192.168.88.132:368) was formed. Members joined: 3 2 left: 3 2

Aug  3 06:26:16 cen7 corosync\[62060\]: \[TOTEM \] Failed to receive the leave message. failed: 3 2

Aug  3 06:28:21 cen7 corosync\[62060\]: \[TOTEM \] A new membership (192.168.88.132:372) was formed. Members

Aug  3 06:30:14 cen7 corosync\[62060\]: \[TOTEM \] A processor failed, forming new configuration.

Aug  3 06:30:14 cen7 corosync\[62060\]: \[TOTEM \] A new membership (192.168.88.132:376) was formed. Members

Aug  3 06:34:54 cen7 corosync\[62060\]: \[TOTEM \] A new membership (192.168.88.132:380) was formed. Members left: 2

**2****）、查看corosync****引擎状态**

\[root@cen7 corosync\]\# grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/messages|tail -n 3

Aug  4 04:16:16 cen7 systemd: Starting Corosync Cluster Engine...

Aug  4 04:16:23 cen7 corosync: Starting Corosync Cluster Engine (corosync): \[  确定  \]

Aug  4 04:16:23 cen7 systemd: Started Corosync Cluster Engine.

**3****）、确认启动过程无ERROR****产生**

\[root@cen7 corosync\]\# grep ERROR: /var/log/messages | grep -v unpack\_resources

**6****、查看集群节点状态**

**1****）、查看cen7****节点状态：正常**

\[root@cen7 corosync\]\# systemctl status corosync.service

● corosync.service - Corosync Cluster Engine

Loaded: loaded (/usr/lib/systemd/system/corosync.service; enabled; vendor preset: disabled)

Active: active (running) since 六 2018-08-04 04:16:23 CST; 41min ago

Docs: man:corosync

man:corosync.conf

man:corosync\_overview

Process: 765 ExecStart=/usr/share/corosync/corosync start (code=exited, status=0/SUCCESS)

Main PID: 795 (corosync)

CGroup: /system.slice/corosync.service

└─795 corosync

8月 04 04:16:16 cen7.field.com systemd\[1\]: Starting Corosync Cluster Engine...

8月 04 04:16:23 cen7.field.com corosync\[765\]: Starting Corosync Cluster Engine (corosync): \[  确定  \]

8月 04 04:16:23 cen7.field.com systemd\[1\]: Started Corosync Cluster Engine.

\[root@cen7 corosync\]\# corosync-cfgtool -s

Printing ring status.

Local node ID 1

RING ID 0

id      = 192.168.88.132    -->说明：此处如果为127.0.0.1则说明启动错误

\#编辑/etc/hosts文件，注销127.0.0.1行，只保留ip和域名后重启Corosync即可

status       = ring 0 active with no faults

**2****）、关闭stonith**

\[root@cen7 corosync\]\# crm configure property stonith-enabled=false

使用crm\_verify检查配置有没问题

\[root@cen7 corosync\]\# crm\_verify -L -V

**3****）、查看node2****节点状态：正常**

\[root@node2 ~\]\# service corosync status

Redirecting to /bin/systemctl status corosync.service

● corosync.service - Corosync Cluster Engine

Loaded: loaded (/usr/lib/systemd/system/corosync.service; enabled; vendor preset: disabled)

Active: active (running) since 六 2018-08-04 04:16:23 CST; 34min ago

Docs: man:corosync

man:corosync.conf

man:corosync\_overview

Process: 839 ExecStart=/usr/share/corosync/corosync start (code=exited, status=0/SUCCESS)

Main PID: 858 (corosync)

CGroup: /system.slice/corosync.service

└─858 corosync

8月 04 04:16:17 node2.field.com systemd\[1\]: Starting Corosync Cluster Engine...

8月 04 04:16:23 node2.field.com corosync\[839\]: Starting Corosync Cluster Engine (corosync): \[  确定  \]

8月 04 04:16:23 node2.field.com systemd\[1\]: Started Corosync Cluster Engine.

\[root@node2 ~\]\# corosync-cfgtool -s

Printing ring status.

Local node ID 2

RING ID 0

id      = 192.168.88.133

status       = ring 0 active with no faults

**4****）、查看集群状态信息**

\[root@cen7 mysql\]\# crm status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 04:45:27 2018

Last change: Sat Aug  4 04:45:18 2018 by root via cibadmin on cen7.field.com

2 nodes configured

0 resources configured

Online: \[ cen7.field.com node2.field.com \]

No resources

\#可以看到集群节点的启动状态：已经Online

**四、使用****crmsh****定义资源，配置****Corosync+Pacemaker+nfs+Mysql****高可用集群**

说明：笔者使用crm交互式模式定义

**1、定义****vip****资源****ocf:heartbeat:IPaddr**

**部分资源定义命令说明：**

**primitive****定义资源：**

**monitor****：定义资源监控选项：interval****（检测间隔时间）timeout****（监控超时时间）**

\[root@cen7 corosync\]\# crm

crm(live)\# configure

\#以下命令表示定义vip资源mysqlip:虚拟IP地址“192.168.88.188”，检测间隔时间为30s，超时时长为20s

crm(live)configure\# primitive mysqlip ocf:heartbeat:IPaddr params ip="192.168.88.188" op monitor interval=30s timeout=20s

crm(live)configure\# verify

\#verify检查语法

crm(live)configure\# commit

\#commit提交配置

crm(live)configure\# show

node 1: cen7.field.com \\

attributes standby=off

node 2: node2.field.com \\

attributes standby=off

primitive mysqlip IPaddr \\

params ip=192.168.88.188 \\

op monitor interval=30s timeout=20s

property cib-bootstrap-options: \\

have-watchdog=false \\

dc-version=1.1.18-11.el7\_5.3-2b07d5c5a9 \\

cluster-infrastructure=corosync \\

stonith-enabled=false \\

no-quorum-policy=ignore

crm(live)configure\# cd

crm(live)\# status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 05:07:38 2018

Last change: Sat Aug  4 05:04:36 2018 by root via cibadmin on cen7.field.com

2 nodes configured

1 resource configured

Online: \[ cen7.field.com node2.field.com \]

Full list of resources:

mysqlip  (ocf::heartbeat:IPaddr):  Started cen7.field.com

\#可以看到：资源mysqlip已经启动在cen7上

crm(live)\# quit

bye

\[root@cen7 corosync\]\# ip addr list | grep ens

2: ens32: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast state UP group default qlen 1000

inet 192.168.88.132/24 brd 192.168.88.255 scope global noprefixroute ens32

inet 192.168.88.188/24 brd 192.168.88.255 scope global secondary ens32

\#可以看到：vip已经启动在cen7上

**2、定义****nfs****文件系统资源：文件系统资源属于****ocf**

**部分命令及参数说明：**

**param****资源管理参数：文件系统资源中，必须指明的参数有device****（设备）、directory****（挂载目录）、fstype****（文件系统类型）**

**start****启动资源选项：interval****（间隔时间）、timeout****（超时时间）**

**stop****停止资源选项：interval****（间隔时间）、timeout****（超时时间）**

**monitor****资源监控选项：interval****（间隔时间）timeout****（监控超时时间）**

\[root@cen7 corosync\]\# crm

crm(live)\# configure

\#以下命令表示创建nfs文件系统资源：

\#将"192.168.88.135:/mydata"nfs共享目录挂载到/mydata目录下，启动超时时间60s，关闭超时时间60s，监控检测间隔时间为20s，超时时长为40s

crm(live)configure\# primitive mynfs ocf:heartbeat:Filesystem params device="192.168.88.135:/mydata" directory="/mydata" fstype="nfs" op start timeout=60s op stop timeout=60s op monitor interval=20s timeout=40s

crm(live)configure\# verify

crm(live)configure\# commit

crm(live)configure\# cd

crm(live)\# status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 05:10:30 2018

Last change: Sat Aug  4 05:10:26 2018 by root via cibadmin on cen7.field.com

2 nodes configured

2 resources configured

Online: \[ cen7.field.com node2.field.com \]

Full list of resources:

mysqlip  (ocf::heartbeat:IPaddr):  Started cen7.field.com

mynfs     (ocf::heartbeat:Filesystem):   Started node2.field.com

可以发现：资源mynfs已经定义并成功启动在node2节点上

\#未配置约束时资源会顺序启动在不同节点

crm(live)\# quit

bye

**3****、定义****mysqld****资源：编译安装的****mysql****属于****lsb****资源代理**

\#以下命令表示定义mysqld资源，检测间隔时间为30s，超时时长为30s

crm(live)configure\# primitive mysql ocf:heartbeat:mysql op monitor interval=30s timeout=30s

crm(live)configure\# verify

crm(live)configure\# commit

crm(live)configure\# cd

crm(live)\# status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 05:25:07 2018

Last change: Sat Aug  4 05:24:57 2018 by root via cibadmin on cen7.field.com

2 nodes configured

3 resources configured

Online: \[ cen7.field.com node2.field.com \]

Full list of resources:

mysqlip  (ocf::heartbeat:IPaddr):  Started cen7.field.com

mynfs     (ocf::heartbeat:Filesystem):   Started node2.field.com

mysqlsrv         (ocf::heartbeat:mysql):   Stopped

Failed Actions:

\* mysqlsrv\_start\_0 on node2.field.com 'not installed' (5): call=37, status=complete, exitreason='Setup problem: couldn't find command: /usr/bin/safe\_mysqld',

last-rc-change='Sat Aug  4 05:24:59 2018', queued=0ms, exec=63ms

\* mysqlsrv\_start\_0 on cen7.field.com 'not installed' (5): call=37, status=complete, exitreason='Setup problem: couldn't find command: /usr/bin/safe\_mysqld',

last-rc-change='Sat Aug  4 05:24:58 2018', queued=0ms, exec=64ms

crm(live)\# configure

crm(live)configure\# edit

\#因为是编译安装的mysql，不能用ocf设置，删除mysql资源定义并:wq保存退出

**改用lsb****下的mysqld****定义mysql****资源**

crm(live)configure\# primitive mysqld lsb:mysqld

crm(live)configure\# verify

crm(live)configure\# commit

crm(live)configure\# cd

crm(live)\# status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 05:26:50 2018

Last change: Sat Aug  4 05:26:44 2018 by root via cibadmin on cen7.field.com

2 nodes configured

3 resources configured

Online: \[ cen7.field.com node2.field.com \]

Full list of resources:

mysqlip  (ocf::heartbeat:IPaddr):  Started cen7.field.com

mynfs     (ocf::heartbeat:Filesystem):   Started node2.field.com

mysqld   (lsb:mysqld):    Starting node2.field.com

Failed Actions:

\* mysqld\_start\_0 on cen7.field.com 'unknown error' (1): call=44, status=complete, exitreason='',

last-rc-change='Sat Aug  4 05:26:45 2018', queued=0ms, exec=1376ms

\#此错误是因为没找到datadir中的初始化文件，重启Corosync可以解决

**4****、定义约束：资源必须启动在同一个节点**

**约束说明：**

**colocation  \#****排列约束，指定哪些资源捆绑一起，在同一节点上运行**

**order       \#****指定排列约束中的资源启动顺序，该顺序和colocation****顺序相反**

**location    \#****位置约束，指定资源首选在哪些节点上运行**

**定义约束的顺序：排列约束-->****顺序约束-->****位置约束**

**“inf:****”表示黏性值无穷大，定义inf:****的两个资源始终在同一节点运行。**

**“-inf:****”表示黏性值负无穷，定义-inf:****的两个资源始终不在同一节点运行。**

\#以下配置表示mysqlip和(mynfs mysqld)间的黏性值无穷大，即三个资源始终在同一节点上运行。

crm(live)configure\# colocation mysqlip\_with\_mynfs\_and\_mysqld inf: mysqlip ( mynfs mysqld)

crm(live)configure\# verify

crm(live)configure\# commit

\#mandatory默认值，表示强制约束，强制约束mynfs和mysqld的状态必须相同且均为start

crm(live)configure\# order mysqld-after-mynfs mandatory: mynfs mysqld:start

crm(live)configure\# verify

crm(live)configure\# commit

\#强制约束mysqld资源必须在mysqlip资源启动后再启动

crm(live)configure\# order mysqld-after-mysqlip  mandatory: mysqlip mysqld

crm(live)configure\# verify

crm(live)configure\# commit

crm(live)configure\# show

node 1: cen7.field.com \\

attributes standby=off

node 2: node2.field.com \\

attributes standby=off

primitive mynfs Filesystem \\

params device="192.168.88.135:/mydata" directory="/mydata" fstype=nfs \\

op start timeout=60s interval=0 \\

op stop timeout=60s interval=0 \\

op monitor interval=20s timeout=40s

primitive mysqld lsb:mysqld

primitive mysqlip IPaddr \\

params ip=192.168.88.188 \\

op monitor interval=30s timeout=20s

order mysqld-after-mynfs Mandatory: mynfs mysqld:start

order mysqld-after-mysqlip Mandatory: mysqlip mysqld

colocation mysqlip\_with\_mynfs\_and\_mysqld inf: mysqlip ( mynfs mysqld )

property cib-bootstrap-options: \\

have-watchdog=false \\

dc-version=1.1.18-11.el7\_5.3-2b07d5c5a9 \\

cluster-infrastructure=corosync \\

stonith-enabled=false \\

no-quorum-policy=ignore

crm(live)configure\# quit

bye

**五、测试高可用集群配置成功与否，并检测高可用性**

**1****、重启****corosync****和****pacemaker**

\[root@cen7 corosync\]\# ansible hamysql -m service -a 'name=corosync state=restarted'

\[root@cen7 corosync\]\# ansible hamysql -m service -a 'name=pacemaker state=restarted'

**2****、查看集群状态信息**

\[root@cen7 corosync\]\# crm status

Stack: corosync

Current DC: node2.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 05:34:30 2018

Last change: Sat Aug  4 05:30:59 2018 by root via cibadmin on cen7.field.com

2 nodes configured

3 resources configured

Online: \[ cen7.field.com node2.field.com \]

Full list of resources:

mysqlip  (ocf::heartbeat:IPaddr):  Started node2.field.com

mynfs     (ocf::heartbeat:Filesystem):   Started node2.field.com

mysqld   (lsb:mysqld):    Started node2.field.com

**3****、在****node2****节点测试**

**1****）、连接mysql****客户端：成功**

\[root@node2 ~\]\# mysql

Welcome to the MariaDB monitor.  Commands end with ; or \\g.

Your MariaDB connection id is 2

Server version: 5.5.61-MariaDB Source distribution

Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.

MariaDB \[(none)\]> quit

Bye

**2****）、VIP****流转：成功**

\[root@node2 ~\]\# ip addr list | grep ens

2: ens33: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast state UP group default qlen 1000

inet 192.168.88.133/24 brd 192.168.88.255 scope global noprefixroute dynamic ens33

inet 192.168.88.188/24 brd 192.168.88.255 scope global secondary ens33

**4****、集群高可用性测试**

**1****）、置node2.field.com****状态为备用状态，检查能否自动切换**

\[root@cen7 corosync\]\# crm

crm(live)\# node

crm(live)node\# standby node2.field.com

crm(live)node\# cd

crm(live)\# status

Stack: corosync

Current DC: node2.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 05:38:52 2018

Last change: Sat Aug  4 05:37:51 2018 by root via crm\_attribute on cen7.field.com

2 nodes configured

3 resources configured

Node node2.field.com: standby

Online: \[ cen7.field.com \]

Full list of resources:

mysqlip  (ocf::heartbeat:IPaddr):  Started cen7.field.com

mynfs     (ocf::heartbeat:Filesystem):   Started cen7.field.com

mysqld   (lsb:mysqld):    Started cen7.field.com

crm(live)\# quit

bye

**2****）、连接mysql****测试：成功**

\[root@cen7 corosync\]\# mysql

Welcome to the MariaDB monitor.  Commands end with ; or \\g.

Your MariaDB connection id is 2

Server version: 5.5.61-MariaDB Source distribution

Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.

MariaDB \[(none)\]> show databases;

\+--------------------+

| Database           |

\+--------------------+

| information\_schema |

| mysql              |

| performance\_schema |

| test               |

\+--------------------+

4 rows in set (0.21 sec)

MariaDB \[(none)\]> quit

Bye

**3****）、查看VIP****是否流转成功：成功**

\[root@cen7 corosync\]\# ip addr list | grep ens

2: ens32: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast state UP group default qlen 1000

inet 192.168.88.132/24 brd 192.168.88.255 scope global noprefixroute ens32

inet 192.168.88.188/24 brd 192.168.88.255 scope global secondary ens32

**附：****crmsh****整体配置**** **

\[root@cen7 corosync\]\# crm

crm(live)\# node

crm(live)node\# online node2.field.com

crm(live)node\# cd

crm(live)\# configure

crm(live)configure\# group mycluster  mysqlip  mynfs mysqld

INFO: modified order:mysqld-after-mysqlip from mysqlip to mycluster

INFO: modified colocation:mysqlip\_with\_mynfs\_and\_mysqld from mysqlip to mycluster

INFO: modified order:mysqld-after-mynfs from mynfs to mycluster

INFO: modified colocation:mysqlip\_with\_mynfs\_and\_mysqld from mynfs to mycluster

INFO: modified order:mysqld-after-mynfs from mysqld to mycluster

INFO: modified order:mysqld-after-mysqlip from mysqld to mycluster

INFO: modified colocation:mysqlip\_with\_mynfs\_and\_mysqld from mysqld to mycluster

crm(live)configure\# verify

crm(live)configure\# commit

crm(live)configure\# show

node 1: cen7.field.com \\

attributes standby=off

node 2: node2.field.com \\

attributes standby=off

primitive mynfs Filesystem \\

params device="192.168.88.135:/mydata" directory="/mydata" fstype=nfs \\

op start timeout=60s interval=0 \\

op stop timeout=60s interval=0 \\

op monitor interval=20s timeout=40s

primitive mysqld lsb:mysqld

primitive mysqlip IPaddr \\

params ip=192.168.88.188 \\

op monitor interval=30s timeout=20s

group mycluster mysqlip mynfs mysqld

property cib-bootstrap-options: \\

have-watchdog=false \\

dc-version=1.1.18-11.el7\_5.3-2b07d5c5a9 \\

cluster-infrastructure=corosync \\

stonith-enabled=false \\

no-quorum-policy=ignore

crm(live)configure\# cd

crm(live)\# status

Stack: corosync

Current DC: node2.field.com (version 1.1.18-11.el7\_5.3-2b07d5c5a9) - partition with quorum

Last updated: Sat Aug  4 05:42:51 2018

Last change: Sat Aug  4 05:42:45 2018 by root via cibadmin on cen7.field.com

2 nodes configured

3 resources configured

Online: \[ cen7.field.com node2.field.com \]

Full list of resources:

Resource Group: mycluster

mysqlip  (ocf::heartbeat:IPaddr):  Started cen7.field.com

mynfs     (ocf::heartbeat:Filesystem):   Started cen7.field.com

mysqld   (lsb:mysqld):    Started cen7.field.com

crm(live)\# quit

bye

\[root@cen7 corosync\]\#

[pcs_Corosync_pacemaker]: https://blog.csdn.net/Field_Yang/article/details/81434710

Corosync+Pacemaker+NFS+Mysql高可用集群部署

发表评论取消回复

还没有评论，来说两句吧...

相关阅读

相关 Kubernetes高可用集群二进制部署（二）ETCD集群部署

相关 Kubernetes中部署MySQL高可用集群

相关高可用集群

相关 rabbitmq高可用集群部署

相关 MySQL高可用负载均衡集群部署

相关 ansible-playbook部署Nginx高可用集群

相关容器化部署Cassandra高可用集群

相关利用saltstack部署keepalived高可用集群

相关 hadoop-HA高可用集群部署

相关 RabbitMQ 集群高可用部署详细介绍