消息中间件:RocketMq部署

悠悠 2022-03-31 11:18 233阅读 0赞

# 分布式消息中间件rocketmq #

## 架构图 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3o3ODU0OTUyOTU_size_16_color_FFFFFF_t_70][]

消息中间件和RPC最大区别：

Broker Cluster存储

首先来提问下：

1.  我们为什么要用消息中间件？
2.  什么是消息中间件，有哪些中间件了？

3、为什么是rocketmq

大家可以先尝试部署下rocketmq环境，并且我会在课上把这些问题告诉大家。

# 1、环境 #

192.168.0.31

192.168.0.32

部署模式：2M-2S-SYNC(两主两从同步写)（机器不够用可以不用部署这么多2M-nosalve）

相关安装包存储路径：/root/svr/rocketmq

注意：部署的时候jdk最好用JDK8。

通过本地打包部署的话，要注意了windows和linux系统的转义符不一样，需要对脚本进行更改。

# 2、部署 #

## 2.1下载 ##

Apache:

[https://www.apache.org/dyn/closer.cgi?path=rocketmq/4.2.0/rocketmq-all-4.2.0-bin-release.zip][https_www.apache.org_dyn_closer.cgi_path_rocketmq_4.2.0_rocketmq-all-4.2.0-bin-release.zip]

Github：

[https://github.com/apache/rocketmq/][https_github.com_apache_rocketmq]

## 2.2解压： ##

unzip rocketmq-all-4.2.0-bin-release.zip -d /root/svr/rocketmq

## 2.3配置host(给nameServer用)： ##

命令：vim /etc/hosts

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">192.168.0.12 rocketmq1</p> <p style="margin-left:0cm;">192.168.0.13 rocketmq2</p> </td> 
  </tr> 
 </tbody> 
</table>

配置环境变量：vim /etc/profile

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;"><span style="color:#333333;">export ROCKETMQ_HOME=/root/svr/rocketmq</span></p> <p style="margin-left:0cm;"><span style="color:#333333;">export PATH=$PATH::$ROCKETMQ_HOME/bin</span></p> </td> 
  </tr> 
 </tbody> 
</table>

执行：source /etc/profile

**2.4****配置集群参数**：

命令（master）：vim /root/svr/rocketmq/conf/2m-2s-sync/broker-a.properties

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">brokerClusterName=tl-rocketmq-cluster</p> <p style="margin-left:0cm;">brokerName=broker-a</p> <p style="margin-left:0cm;">brokerId=0</p> <p style="margin-left:0cm;">namesrvAddr=rocketmq1:9876;rocketmq2:9876</p> <p style="margin-left:0cm;">defaultTopicQueueNums=4</p> <p style="margin-left:0cm;">autoCreateTopicEnable=true</p> <p style="margin-left:0cm;">autoCreateSubscriptionGroup=true</p> <p style="margin-left:0cm;">listenPort=10911</p> <p style="margin-left:0cm;">deleteWhen=04</p> <p style="margin-left:0cm;">fileReservedTime=120</p> <p style="margin-left:0cm;">mapedFileSizeCommitLog=1073741824</p> <p style="margin-left:0cm;">mapedFileSizeConsumeQueue=300000</p> <p style="margin-left:0cm;">destroyMapedFileIntervalForcibly=120000</p> <p style="margin-left:0cm;">redeleteHangedFileInterval=120000</p> <p style="margin-left:0cm;">diskMaxUsedSpaceRatio=88</p> <p style="margin-left:0cm;">storePathRootDir=/root/svr/rocketmq/data/store</p> <p style="margin-left:0cm;">storePathCommitLog=/root/svr/rocketmq/data/store/commitlog</p> <p style="margin-left:0cm;">maxMessageSize=65536</p> <p style="margin-left:0cm;">flushCommitLogLeastPages=4</p> <p style="margin-left:0cm;">flushConsumeQueueLeastPages=2</p> <p style="margin-left:0cm;">flushCommitLogThoroughInterval=10000</p> <p style="margin-left:0cm;">flushConsumeQueueThoroughInterval=60000</p> <p style="margin-left:0cm;">checkTransactionMessageEnable=false</p> <p style="margin-left:0cm;">sendMessageThreadPoolNums=128</p> <p style="margin-left:0cm;">pullMessageThreadPoolNums=128</p> <p style="margin-left:0cm;">brokerRole=SYNC_MASTER</p> <p style="margin-left:0cm;">flushDiskType=ASYNC_FLUSH</p> </td> 
  </tr> 
 </tbody> 
</table>

命令（slave）：vim /root/svr/rocketmq/conf/2m-2s-sync/broker-b-s.properties

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">brokerClusterName=tl-rocketmq-cluster</p> <p style="margin-left:0cm;"><span style="color:#FF0000;">brokerName=broker-a</span></p> <p style="margin-left:0cm;">#0 表示 Master，&gt;0 表示 Slave</p> <p style="margin-left:0cm;"><span style="color:#FF0000;">brokerId=1 </span></p> <p style="margin-left:0cm;">namesrvAddr=rocketmq1:9876;rocketmq2:9876</p> <p style="margin-left:0cm;">defaultTopicQueueNums=4</p> <p style="margin-left:0cm;">autoCreateTopicEnable=true</p> <p style="margin-left:0cm;">autoCreateSubscriptionGroup=true</p> <p style="margin-left:0cm;"><span style="color:#FF0000;">listenPort=10911</span></p> <p style="margin-left:0cm;">deleteWhen=04</p> <p style="margin-left:0cm;">fileReservedTime=120</p> <p style="margin-left:0cm;">mapedFileSizeCommitLog=1073741824</p> <p style="margin-left:0cm;">mapedFileSizeConsumeQueue=300000</p> <p style="margin-left:0cm;">destroyMapedFileIntervalForcibly=120000</p> <p style="margin-left:0cm;">redeleteHangedFileInterval=120000</p> <p style="margin-left:0cm;">diskMaxUsedSpaceRatio=88</p> <p style="margin-left:0cm;">storePathRootDir=/root/svr/rocketmq/data/store</p> <p style="margin-left:0cm;">storePathCommitLog=/root/svr/rocketmq/data<span style="color:#FF0000;">/</span>store/commitlog</p> <p style="margin-left:0cm;">maxMessageSize=65536</p> <p style="margin-left:0cm;">flushCommitLogLeastPages=4</p> <p style="margin-left:0cm;">flushConsumeQueueLeastPages=2</p> <p style="margin-left:0cm;">flushCommitLogThoroughInterval=10000</p> <p style="margin-left:0cm;">flushConsumeQueueThoroughInterval=60000</p> <p style="margin-left:0cm;">checkTransactionMessageEnable=false</p> <p style="margin-left:0cm;">sendMessageThreadPoolNums=128</p> <p style="margin-left:0cm;">pullMessageThreadPoolNums=128</p> <p style="margin-left:0cm;">#角色</p> <p style="margin-left:0cm;"><span style="color:#FF0000;">brokerRole=SLAVE</span></p> <p style="margin-left:0cm;">flushDiskType=ASYNC_FLUSH</p> </td> 
  </tr> 
 </tbody> 
</table>

自动切换角色 zk redis 切换 不自动切换

## 2.5创建存储&日志文件 ##

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">mkdir /root/svr/rocketmq/data (也要创建store文件夹)</p> <p style="margin-left:0cm;">mkdir /root/svr/rocketmq/data/store/commitlog</p> <p style="margin-left:0cm;">mkdir /root/svr/rocketmq/data/store/consumequeue</p> <p style="margin-left:0cm;">mkdir /root/svr/rocketmq/data/store/index</p> </td> 
  </tr> 
 </tbody> 
</table>

注意：如果一台虚拟机部署多个需要用文件进行区分

## 2.6修改日志配置文件 ##

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">mkdir -p /root/svr/rocketmq/logs</p> <p style="margin-left:0cm;">cd /root/svr/rocketmq/conf &amp;&amp; sed -i 's#${user.home}#/root/svr/rocketmq#g'</p> <p style="margin-left:0cm;">*.xml</p> </td> 
  </tr> 
 </tbody> 
</table>

注意logback.\*.xml配置文件中$\{user.home\}需要替换自己指定的目录

## 2.7改参数 ##

runbroker.sh,runserver.sh启动参数默认对jvm的堆内存设置比较大(不改启动不起来)，如果是虚拟机非线上环境需要改下参数，大小可以根据自己机器来决定

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">默认大小</p> <p style="margin-left:0cm;">-Xms8g -Xmx8g -Xmn4g</p> <p style="margin-left:0cm;">改为：</p> <p style="margin-left:0cm;"><img alt="" height="35" src="https://img-blog.csdnimg.cn/20190103102229468.png" width="780"></p> <p style="margin-left:0cm;">JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"</p> </td> 
  </tr> 
 </tbody> 
</table>

## 2.8复制 ##

**scp****其他服务器**

scp -r rocketmq/ root@192.168.0.13:/root/svr/

## 3、启动服务 ##

##  3.1**启动Nameserver****服务** ##

在cd /root/svr/rocketmq/bin目录下执行命令：

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;"><span style="color:#000000;">nohup sh mqnamesrv &amp;</span></p> </td> 
  </tr> 
 </tbody> 
</table>

## 3.2启动BrokerServer ##

在cd /root/svr/rocketmq/bin目录下执行命令：

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;"><span style="color:#000000;">nohup sh mqbroker -c /root/svr/rocketmq/conf/2m-2s-sync/broker-a.properties&amp;</span></p> </td> 
  </tr> 
 </tbody> 
</table>

## 3.3是否启动成功 ##

输入命令jps或者查看rocketmq/logs下日志是否输出正常。

![20190103102229474.png][]

安装成功。

查看集群监控状态：

sh mqadmin clusterlist -n 192.168.0.12:9876

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3o3ODU0OTUyOTU_size_16_color_FFFFFF_t_70 1][]![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3o3ODU0OTUyOTU_size_16_color_FFFFFF_t_70 2][]

# 监控源码下载地址： #

[https://github.com/apache/rocketmq-externals.git][https_github.com_apache_rocketmq-externals.git]

配置namesrv

rocketmq.config.namesrvAddr=192.168.0.31:9876

## 测试： ##

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">export NAMESRV_ADDR=rocketmq1:9876;rocketmq2:9876</p> <p style="margin-left:0cm;">测试发送端</p> <p style="margin-left:0cm;">&nbsp;&gt; sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer</p> <p style="margin-left:0cm;">测试消费端</p> <p style="margin-left:0cm;">&nbsp;&gt; sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer</p> </td> 
  </tr> 
 </tbody> 
</table>

## 服务停止： ##

在cd /root/svr/rocketmq/bin目录下执行命令或者jps查看进程号 kill -9 pid

<table style="width:568px;"> 
 <tbody> 
  <tr> 
   <td style="vertical-align:top;width:426.1pt;"> <p style="margin-left:0cm;">sh mqshutdown broker</p> <p style="margin-left:0cm;">sh mqshutdown namesrv</p> </td> 
  </tr> 
 </tbody> 
</table>

### 常见错误 ###

Jvm参数没有配置启动错误：

\# There is insufficient memory for the Java Runtime Environment to continue.

\# Native memory allocation (malloc) failed to allocate 8589934592 bytes for committing reserved memory.

\# An error report file with more information is saved as:

\# /root/svr/rocketmq/bin/hs\_err\_pid30288.log

\[root@localhost bin\]\# ll

启动日志报:Please set the JAVA\_HOME variable in your environment, We need java(x64)! !!

需要设置vim /etc/profile环境变量

export JAVA\_HOME=/root/svr/jdk

source /etc/profile

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3o3ODU0OTUyOTU_size_16_color_FFFFFF_t_70]: /images/20220331/7363c8db3f7d4d7fa1cf6e2f995759d4.png
[https_www.apache.org_dyn_closer.cgi_path_rocketmq_4.2.0_rocketmq-all-4.2.0-bin-release.zip]: https://www.apache.org/dyn/closer.cgi?path=rocketmq/4.2.0/rocketmq-all-4.2.0-bin-release.zip
[https_github.com_apache_rocketmq]: https://github.com/apache/rocketmq/
[20190103102229474.png]: /images/20220331/260007e5fa234711aaaad837f53cc8b8.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3o3ODU0OTUyOTU_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20190103102229519.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3o3ODU0OTUyOTU=,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3o3ODU0OTUyOTU_size_16_color_FFFFFF_t_70 2]: /images/20220331/fa55038ffc584f3db15c91c0c13e9eb9.png
[https_github.com_apache_rocketmq-externals.git]: https://github.com/apache/rocketmq-externals.git