Ambari2.7.4集成Hue4.6.0

太过爱你忘了你带给我的痛 2022-11-15 05:55 190阅读 0赞

# 一、前言 #

Hue是一个开源的Apache Hadoop UI系统，由Cloudera Desktop演化而来，最后Cloudera公司将其贡献给Apache基金会的Hadoop社区，它是基于Python Web框架Django实现的。

通过使用Hue我们可以在浏览器端的Web控制台上与Hadoop集群进行交互来分析处理数据，例如操作HDFS上的数据，运行MapReduce Job，执行Hive的SQL语句，浏览HBase数据库等等。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70][]

# 二、版本信息 #

**Ambari：2.7.4**

**HDP：3.1.4**

**HUE：4.6.0   下载地址：[https://cdn.gethue.com/downloads/hue-4.6.0.tgz][https_cdn.gethue.com_downloads_hue-4.6.0.tgz]**

**ambari-hue-service 集成插件  下载地址：[https://github.com/lijufeng2016/ambari-hue-service][https_github.com_lijufeng2016_ambari-hue-service]**

*全文参考[https://blog.csdn.net/u013289115/article/details/104757902][https_blog.csdn.net_u013289115_article_details_104757902]，感谢这位大佬提供的插件，但是有些步骤不够详细也比较乱，弄了两天才搞成功。按照自己的思路重新整理了一下。*

# 二、环境准备 #

### 1、hue的master节点上执行，为编译环境做准备 ###

yum install sqlite-devel  libxslt-devel.x86\_64 python-devel openldap-devel asciidoc cyrus-sasl-gssapi  libxml2-devel.x86\_64 gcc gcc-c++ kernel-devel openssl-devel gmp-devel libffi-devel

![20210331104402903.png][]

**注意：一定要全部yum安装成功，不然后面编译hue会失败。**

\#安装新版的nodejs

curl --silent --location https://rpm.nodesource.com/setup\_10.x | bash -

yum install -y nodejs

![20210331104534415.png][]

### 2、所有机器上创建用户和组 ###

useradd -g hue hue

### 3、提前在mysql创建好hue的库并授权 ###

create database hue character set utf8;

CREATE USER 'hue'@'%'IDENTIFIED BY 'Siger\_123';

grant ALL PRIVILEGES ON \*.\* TO 'hue'@'%';

CREATE USER 'hue'@'localhost'IDENTIFIED BY 'Siger\_123';

grant ALL PRIVILEGES on \*.\* to hue@'localhost' identified by 'Siger\_123';

FLUSH PRIVILEGES;

### 4、提前建好hue在hdfs的HOME目录 ###

su hdfs

hadoop fs -mkdir /user/hue

hadoop fs -chown hue:hue /user/hue

# 三、搭建 #

### 1、下载插件源码 ###

在ambari server节点执行

VERSION=\`hdp-select status hadoop-client | sed 's/hadoop-client - \$\[0-9\]\\.\[0-9\]\$.\*/\\1/'\`

rm -rf /var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/HUE

sudo git clone https://github.com/lijufeng2016/ambari-hue-service.git /var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/HUE

![20210331104604941.png][]

### 2、拷贝安装包 ###

mkdir /var/www/html/ambari/HDP/centos7/3.1.4.0-315/hue/

将hue-4.6.0.tgz安装包放在hue目录下

![20210331104749647.png][]

### 3、修改代码和环境配置 ###

*  **（1）package/files/configs.sh****文件**

cd /var/lib/ambari-server/resources/stacks/HDP/3.1/services/HUE/package/files

vim configs.sh

![20210331104817283.png][]

*  **（2）package/scripts/params.py文件**

第32行 download\_url 改成你自己的地址，可以跟hdp的本地仓库放一起

第40行 ambari\_server\_hostname 改成你自己的地址

http://172.8.10.100/ambari/HDP/centos7/3.1.4.0-315/hue/hue-4.6.0.tgz

![20210331104826267.png][]

*  **（3）**修改ignore\_groupsusers\_create \[w1\]

cd /var/lib/ambari-server/resources/scripts

python configs.py -u admin -p admin -n CloudTest -l siger-master.test.com -t 8080 -a get -c cluster-env |grep -i ignore\_groupsusers\_create

python configs.py -u admin -p admin -n CloudTest -l siger-master.test.com -t 8080 -a set -c cluster-env -k ignore\_groupsusers\_create -v true

![20210331104849755.png][]

*  （4）修改编码\[w2\]

\# 在hue的安装节点上：

vim /usr/lib/ambari-agent/lib/resource\_management/core/sudo.py

\# 添加如下代码：

import sys

reload(sys)

sys.setdefaultencoding(‘utf-8’)

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 1][]

### 4、部署安装 ###

\# 重启ambari

ambari-server restart

\# ambari界面操作

界面左侧 >> services >> Add service >> Hue >> NEXT >> 选择Hue Server >> NEXT >> 配置：数据库配置，这里选了mysql：

**HUE DATABASES:**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 2][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 3][]

**HUE USER INFO:**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 4][]

**HUE SERVICE MOOULE:**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 5][]

**\#** **安装成功**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 6][]

### 5、编译hue ###

cd /usr/hdp/3.1.4.0-315/hue/

make apps

![20210331105317603.png][]

### 6、修改配置 ###

*  （1）修改hue的pseudo-distributed.ini 配置 \[w3\]

allowed\_hosts=\*

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 7][]

*  （2）修改hue源码 \[w4\]

vim /usr/hdp/3.1.4.0-315/hue/desktop/core/src/desktop/lib/conf.py

第293行改为：

if raw is None or raw=='':

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 8][]

*  （3）修改host和port \[w5\]

\# 把hiveserver2的host和端口号手动设置一下

**修改hue-hive-site**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 9][]

**修改pseudo-distributed.ini**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 10][]

*  （4）修改use\_sasl \[w6\]

修改pseudo-distributed.ini，在beeswax的配置下面加上 use\_sasl=true

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 11][]

*  （5）修改源码 \[w7\]

新版的hue4.6.0与hdp3.1.4这种双新组合往往自带坑位，网上也找不到任何答案，经过一步步推测排查，首先可以确定的是后端返回字段的时候有问题，与之相关的是hive相关的包，经过漫长的一步一步排查，确定到了哪一行代码。

vim /usr/hdp/3.1.4.0-315/hue/apps/beeswax/src/beeswax/server/hive\_server2\_lib.py

118行和119行的2改成1即可

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 12][]

*  （6）启动thrift \[w8\]

hdp3中，hbase的thrift默认不开启，需要手动在各个Hmaster节点启动，注意，一定要使用hbase用户启动thrift，而不是thrift2！！否则后面还会有问题，上代码

su hbase

/usr/hdp/current/hbase-master/bin/hbase-daemon.sh start thrift

![20210331110202900.png][]

**建议把启动thrift的命令写到启动hbase master的脚本里，这样就不用每次手动起了。**

### 7、启动 ###

\#改完重启，终于看到人样的页面了，创建hue账号，第一次任意设置，我这里设置成admin/admin，

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 13][]

--------------------

\[w1\] 不修改，安装时报错：KEY ERROR -u'hue'

keyError -u'hue'

Error: Error: Unable to run the custom hook script \['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-303.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-303.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL\_TLSv1\_2', ''\]

\[w2\] 不修改，安装时报错：UnicodeEncodeError: 'ascii' codec can't encode character u'\\u201c' in position 3462: ordinal not in range(128)

因为/usr/lib/python2.7/site-packages/resource\_management/core/sudo.py文件在新版本中不适用！需要修改编码

\[w3\] 不修改，安装时报错：Invalid HTTP\_HOST header: ‘siger-master.test.com:8888’. You may need to add u’ siger-master.test.com’ to ALLOWED\_HOSTS.

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 14][]

\[w4\] 不修改，访问hue时会报：加载数据库时出错

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 15][]

\[w5\] 端口号当成字符串输入了，会报错：**%d format: a number is required, not str**

![20210331105848636.png][]

\[w6\] 不修改，会报错：TSocket read 0 bytes (code THRIFTTRANSPORT): TTransportException(‘TSocket read 0 bytes’,)

![20210331105938826.png][]

\[w7\] 不修改，会报错：显示无列

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 16][]

\[w8\]  不启动，查看hbase报错，无法联系到 HBase Thrift 1 服务器：Could not connect to siger-master.test.com:9090

![20210331110218301.png][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70]: /images/20221022/94ca6b89a7f54125925551413c9704a3.png
[https_cdn.gethue.com_downloads_hue-4.6.0.tgz]: https://cdn.gethue.com/downloads/hue-4.6.0.tgz
[https_github.com_lijufeng2016_ambari-hue-service]: https://github.com/lijufeng2016/ambari-hue-service
[https_blog.csdn.net_u013289115_article_details_104757902]: https://blog.csdn.net/u013289115/article/details/104757902
[20210331104402903.png]: /images/20221022/498fcbdd6d784e7b8ac8fd56ce73818d.png
[20210331104534415.png]: https://img-blog.csdnimg.cn/20210331104534415.png
[20210331104604941.png]: https://img-blog.csdnimg.cn/20210331104604941.png
[20210331104749647.png]: https://img-blog.csdnimg.cn/20210331104749647.png
[20210331104817283.png]: https://img-blog.csdnimg.cn/20210331104817283.png
[20210331104826267.png]: https://img-blog.csdnimg.cn/20210331104826267.png
[20210331104849755.png]: https://img-blog.csdnimg.cn/20210331104849755.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20210331105039760.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20210331105120177.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20210331105129835.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20210331105137945.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20210331105144613.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20210331105257263.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[20210331105317603.png]: https://img-blog.csdnimg.cn/20210331105317603.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20210331105411863.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20210331105651404.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 9]: https://img-blog.csdnimg.cn/20210331105807801.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 10]: https://img-blog.csdnimg.cn/20210331105829797.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 11]: https://img-blog.csdnimg.cn/2021033110592665.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 12]: https://img-blog.csdnimg.cn/20210331110031128.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[20210331110202900.png]: https://img-blog.csdnimg.cn/20210331110202900.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 13]: https://img-blog.csdnimg.cn/20210331110341271.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 14]: https://img-blog.csdnimg.cn/20210331105544459.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 15]: https://img-blog.csdnimg.cn/20210331105723903.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[20210331105848636.png]: https://img-blog.csdnimg.cn/20210331105848636.png
[20210331105938826.png]: https://img-blog.csdnimg.cn/20210331105938826.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5_size_16_color_FFFFFF_t_70 16]: https://img-blog.csdnimg.cn/20210331110126692.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzIxMTUzNjE5,size_16,color_FFFFFF,t_70
[20210331110218301.png]: https://img-blog.csdnimg.cn/20210331110218301.png