Cloudera Manager中安装部署Zeppelin服务

灰太狼 2022-01-29 07:51 316阅读 0赞

1.  **[制作Zeppelin的Parcel包和csd文件][Zeppelin_Parcel_csd]**
2.  **部署Livy 服务**  
    参考 [Cloudera Manager中安装部署Livy服务][Cloudera Manager_Livy]
3.  **将 Parcel 包和 manifest.json 文件部署到 httpd 服务中**
    
        [root@cdh01 ~]# mkdir -p /var/www/html/parcels/zeppelin
        [root@cdh01 ~]# cd /var/www/html/parcels/zeppelin
        [root@cdh01 zeppelin]# cp ~/github/cloudera/livy_zeppelin_cdh_csd_parcels/ZEPPELIN-0.8.1_build/ZEPPELIN-0.8.1-el7.parcel ./
        [root@cdh01 zeppelin]# cp ~/github/cloudera/livy_zeppelin_cdh_csd_parcels/ZEPPELIN-0.8.1_build/manifest.json ./
        [root@cdh01 zeppelin]# ll
        总用量 968592
        -rw-r--r-- 1 root root       316 5月  25 15:14 manifest.json
        -rw-r--r-- 1 root root 991831138 5月  25 15:14 ZEPPELIN-0.8.1-el7.parcel
    
    浏览器访问是否部署成功  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70]
4.  **将生成的csd文件,复制到cloudera-manager-server服务所在节点的 /opt/cloudera/csd 目录下**
    
        [root@cdh01 ~]# cd /opt/cloudera/csd/
        [root@cdh01 csd]# cp ~/github/cloudera/livy_zeppelin_cdh_csd_parcels/ZEPPELIN-0.8.1.jar ./
5.  **重启cloudera-scm-server服务**
    
        [root@cdh01 ~]# systemccloudera-scm-server restart
6.  **配置远程Parcel存储库**  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 1]  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 2]
7.  **重启Cloudera Management Service**  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 3]
8.  **下载分发激活**
    
    **下载**![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 4] ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 5]
    
    **分配** ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 6]  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 7]  
    **激活**  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 8]  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 9]
9.  **部署Zeppelin**  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 10]  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 11]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 12]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 13]

1.  **遇到的问题**
    
    ZEPPELIN --> 配置 --> 搜索 zeppelin\_java\_options
    
    1.  **问题一**
        
            java.lang.IllegalArgumentException: The variable [${zeppelin_java_options}] does not have a corresponding value
        
        ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 14]  
        **解决办法：**
        
        ZEPPELIN --> 配置 --> 搜索 zeppelin\_java\_options --> 添加参数 -Xms1024m --> 重启ZEPPELIN 服务![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 15]
    2.  **问题二**
        
            Error found before invoking supervisord: 'getpwnam(): name not found: zeppelin'
        
        ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 16]
        
        **解决办法：**
        
        在 Zeppelin Server 服务所在的节点添加 zeppelin 用户和角色：
        
            [root@cdh04 ~]# groupadd zeppelin
            [root@cdh04 ~]# useradd zeppelin -g zeppelin
    3.  **问题三**
        
            mkdir: `file:///var/local/zeppelin/conf': Input/output error
        
        ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 17]
        
        **解决方法**
        
        ZEPPELIN --> 配置 --> 搜索 dir --> 修改配置 --> 重启ZEPPELIN 服务
        
        修改 `zeppelin.config.fs.dir` 和 `zeppelin.notebook.dir` 配置为 hdfs 路径![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 18]

[Zeppelin_Parcel_csd]: https://blog.csdn.net/weixin_43215250/article/details/90406232
[Cloudera Manager_Livy]: https://blog.csdn.net/weixin_43215250/article/details/89157348
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70]: /images/20220129/f351b0ddb9274771a4562125c4fced09.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 1]: /images/20220129/6c9ee3d4be784fb09b2461bd1edde0b1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 2]: /images/20220129/88da4e59939542f2b50657bf130296d0.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 3]: /images/20220129/93c23590d3d44a229043b3fdec4e7d4e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20190526122320221.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 5]: /images/20220129/97e2938b2a954e85a51d52c291c5e985.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 6]: /images/20220129/fae3e7d70b544f018935ec83b530fc4e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 7]: /images/20220129/59dc47e0cd354ccc954c5313ff3b4853.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 8]: /images/20220129/8063e4337ea54229955e6a8685132c83.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 9]: /images/20220129/dfc30ccd1890488f9368c73b41538227.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 10]: /images/20220129/420aaaba1e49488e94752a26693162a1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 11]: /images/20220129/cdfe929e8f024f9195f12a423f6b60b2.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 12]: /images/20220129/dbbcb2ab211e4314a993601ddd73d47c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 13]: /images/20220129/7c7002d1c071406084014c22c3aca442.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 14]: /images/20220129/d89c2e82701b4948a80003467a5901f3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 15]: /images/20220129/4b021bffdd224784a214cfa0a1e43f4c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 16]: /images/20220129/4b0f3eae35a142059feb55a88686710b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 17]: /images/20220129/9c1ab34fff6a415fa62916187021aa56.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzIxNTI1MA_size_16_color_FFFFFF_t_70 18]: /images/20220129/177aba99f29448e2b1c480cdd9937468.png