记一次 Centos7.6 安装部署基于 zookeeper 3.6.2 集群的 Storm 2.2.0 集群

阳光穿透心脏的1/2处 2022-12-31 01:10 179阅读 0赞

# 一、基本信息 #

1、官网 [http://storm.apache.org/][http_storm.apache.org]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70]  
2、下载地址 [http://storm.apache.org/downloads.html][http_storm.apache.org_downloads.html]

https://mirror.bit.edu.cn/apache/storm/apache-storm-2.2.0/apache-storm-2.2.0.tar.gz 
    https://mirrors.bfsu.edu.cn/apache/storm/apache-storm-2.2.0/apache-storm-2.2.0.tar.gz 
    https://mirrors.tuna.tsinghua.edu.cn/apache/storm/apache-storm-2.2.0/apache-storm-2.2.0.tar.gz

3、集群搭建文档 [http://storm.apache.org/releases/2.2.0/Setting-up-a-Storm-cluster.html][http_storm.apache.org_releases_2.2.0_Setting-up-a-Storm-cluster.html]  
4、github [https://github.com/apache/storm][https_github.com_apache_storm]  
5、w3cschool  
[https://www.w3cschool.cn/storm/][https_www.w3cschool.cn_storm]  
[https://www.w3cschool.cn/apache\_storm/][https_www.w3cschool.cn_apache_storm]

# 二、基础环境准备 #

服务器基本配置可参考  
[https://blog.csdn.net/llwy1428/article/details/111144524][https_blog.csdn.net_llwy1428_article_details_111144524]  
[https://blog.csdn.net/llwy1428/article/details/111464707][https_blog.csdn.net_llwy1428_article_details_111464707]  
安装部署 Zookeeper 3.6.2 集群 [https://blog.csdn.net/llwy1428/article/details/111601567][https_blog.csdn.net_llwy1428_article_details_111601567]

# 三、安装部署 #

1、文件下载  
在 master 节点下载文件

[root@master ~]# wget -P /usr/bigdata/ https://mirror.bit.edu.cn/apache/storm/apache-storm-2.2.0/apache-storm-2.2.0.tar.gz

效果如下  
![在这里插入图片描述][20201227011338245.png]  
2、进入目录 /usr/bigdata/

[root@slave1 ~]# cd /usr/bigdata/

3、解压文件

[root@master bigdata]# tar zxf apache-storm-2.2.0.tar.gz

4、查看文件目录  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 1]  
5、创建目录

[root@master ~]# mkdir /usr/bigdata/apache-storm-2.2.0/work

6、进入 storm 的配置文件目录

[root@master ~]# cd /usr/bigdata/apache-storm-2.2.0/conf

7、编辑文件 storm.yaml

[root@master conf]# vim storm.yaml

8、编辑内容

storm.zookeeper.servers:
    - "slave1"
    - "slave2"
    - "slave3"
    
    nimbus.seeds: ["master"]
    
    storm.local.dir: "/usr/bigdata/apache-storm-2.2.0/work"
    
    supervisor.slots.ports:
        - 6700
        - 6701
        - 6702
        - 6703
      
    drpc.servers: ["master"]

9、编辑文件 storm-env.sh

[root@master conf]# vim storm-env.sh

10、编辑内容

export JAVA_HOME="/usr/lib/jvm/java-1.8.0"
    export STORM_CONF_DIR="/usr/bigdata/apache-storm-2.2.0/conf"

11、把配置好的 storm 目录及文件发送给指定的数据处理节点

[root@master ~]# scp -r /usr/bigdata/apache-storm-2.2.0 root@slave1:/usr/bigdata/
    [root@master ~]# scp -r /usr/bigdata/apache-storm-2.2.0 root@slave2:/usr/bigdata/
    [root@master ~]# scp -r /usr/bigdata/apache-storm-2.2.0 root@slave3:/usr/bigdata/

12、启动各个 zookeeper 节点上的 zkServer 服务

[root@slave1 ~]# /usr/bigdata/apache-zookeeper-3.6.2-bin/bin/zkServer.sh start
    [root@slave2 ~]# /usr/bigdata/apache-zookeeper-3.6.2-bin/bin/zkServer.sh start
    [root@slave3 ~]# /usr/bigdata/apache-zookeeper-3.6.2-bin/bin/zkServer.sh start

13、进入 Storm 的 bin 目录

[root@master ~]# cd /usr/bigdata/apache-storm-2.2.0/bin/

14、启动 nimbus

[root@master bin]# nohup ./storm nimbus > /dev/null 2>&1 &

15、启动 stormui

[root@master bin]# nohup ./storm ui > /dev/stormui 2>&1 &

16、在各个数据处理节点进入 bin 目录，启动 supervisor

[root@slave1 ~]# cd /usr/bigdata/apache-storm-2.2.0/bin
    [root@slave1 bin]# nohup ./storm supervisor > /dev/null 2>&1 &
    
    [root@slave2 ~]# cd /usr/bigdata/apache-storm-2.2.0/bin
    [root@slave2 bin]# nohup ./storm supervisor > /dev/null 2>&1 &
    
    [root@slave3 ~]# cd /usr/bigdata/apache-storm-2.2.0/bin
    [root@slave3 bin]# nohup ./storm supervisor > /dev/null 2>&1 &

17、或在各个节点上直接启动 supervisor

[root@slave1 ~]# nohup /usr/bigdata/apache-storm-2.2.0/bin/storm supervisor > /dev/null 2>&1 &
    [root@slave2 ~]# nohup /usr/bigdata/apache-storm-2.2.0/bin/storm supervisor > /dev/null 2>&1 &
    [root@slave3 ~]# nohup /usr/bigdata/apache-storm-2.2.0/bin/storm supervisor > /dev/null 2>&1 &

18、查看启动结果  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 2]  
19、查看 master 节点上的启动进程（我这个集群上有启动的 mysql 数据库服务和hadoop）  
![在这里插入图片描述][20201227013714492.png]

# 四、浏览器查看 Storm Web UI #

浏览器地址栏录入 192.168.11.21:8080 说明：192.168.11.21 是 master 节点的 IP  
1、集群基本信息  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 3]  
2、各数据处理节点信息  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 4]  
3、集群的各项配置信息  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 5]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 6]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 7]

# 五、报错及解决方案 #

查看 Storm Web UI 失败  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 8]  
报错内容：  
Could not find leader nimbus from seed hosts \[slave1, slave2, slave3\]. Did you specify a valid list of nimbus hosts for config nimbus.seeds?  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 9]  
问题原因：  
由于是重新部署的storm，原先旧的 storm 已在 zookeeper 中注册了信息，只要进入 zookeeper 客户端将 storm 删除：

\[root@slave1 ~\]\# /usr/bigdata/apache-zookeeper-3.6.2-bin/bin/zkCli.shzkCli.sh  
ls /  
递归删除  
deleteall /storm  
重启 storm 集群即可。

# 六、其他 #

[http://www.tianshouzhi.com/api/tutorials/storm][http_www.tianshouzhi.com_api_tutorials_storm]  
[https://ifeve.com/tag/storm/][https_ifeve.com_tag_storm]

至此，安装部署基于 zookeeper 3.6.2 集群的 Storm 2.2.0 集群安装部署完毕，希望能够对您有所帮助！

[http_storm.apache.org]: http://storm.apache.org/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70]: /images/20221120/26e9d25d68aa459aac38c621f2bc1123.png
[http_storm.apache.org_downloads.html]: http://storm.apache.org/downloads.html
[http_storm.apache.org_releases_2.2.0_Setting-up-a-Storm-cluster.html]: http://storm.apache.org/releases/2.2.0/Setting-up-a-Storm-cluster.html
[https_github.com_apache_storm]: https://github.com/apache/storm
[https_www.w3cschool.cn_storm]: https://www.w3cschool.cn/storm/
[https_www.w3cschool.cn_apache_storm]: https://www.w3cschool.cn/apache_storm/
[https_blog.csdn.net_llwy1428_article_details_111144524]: https://blog.csdn.net/llwy1428/article/details/111144524
[https_blog.csdn.net_llwy1428_article_details_111464707]: https://blog.csdn.net/llwy1428/article/details/111464707
[https_blog.csdn.net_llwy1428_article_details_111601567]: https://blog.csdn.net/llwy1428/article/details/111601567
[20201227011338245.png]: /images/20221120/3d406b1193f64ae78dbe642e1829a98a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 1]: /images/20221120/1b5a1b8e50f14cdd8b833f33d9353507.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 2]: /images/20221120/60ecba3822224c8abdd265cdd2270d10.png
[20201227013714492.png]: /images/20221120/01dab8ab418647c38adde4a2163b649b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 3]: /images/20221120/bdca1f4691814997b39eab7c7f2e6349.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 4]: /images/20221120/c7362b75f9fc4b34ab58d04e1f8c2073.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 5]: /images/20221120/714e2916077c4ca1accb5f016bc2e082.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 6]: /images/20221120/e12d3808557d4ede93c446a70697d84b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 7]: /images/20221120/f6e9867f324d4542879608058375fee4.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 8]: /images/20221120/9aed998aac9841b795f1c4d553495bf2.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xsd3kxNDI4_size_16_color_FFFFFF_t_70 9]: /images/20221120/e77f76a21c0b47b2a92a866a7535f82f.png
[http_www.tianshouzhi.com_api_tutorials_storm]: http://www.tianshouzhi.com/api/tutorials/storm
[https_ifeve.com_tag_storm]: https://ifeve.com/tag/storm/