hive 与 kylin的搭建

忘是亡心i 2022-07-13 04:52 184阅读 0赞

# hive安装 #

## 前言 ##

由于kylin现在最高支持的hive版本为1.2.1，所以只能安装此版本，不能安装hive2.0。使用hive需要事先安装好ant，具体安装过程不进行介绍，还需要一个mysql来保存hive的元数据。

## 编译hive ##

编译hive一定要保证网络通畅，编译过程中会出现个别的包无法下载，导致错误，换个时间或者调整网络环境，从新编译，就可以了

解压源码包。进行编译。

tar -xzvf apache-hive-1.2.1-src.tar.gz cd apache-hive-1.2.1-src

修改pom.xml中的如下项

<table> 
 <tbody> 
  <tr> 
   <td><span style="font-size:14px">hadoop-23.version</span></td> 
  </tr> 
  <tr> 
   <td><span style="font-size:14px">hbase.hadoop2.version</span></td> 
  </tr> 
 </tbody> 
</table>

保存完毕后，执行编译命令

mvn clean package -Phadoop-2,dist -DskipTests

编译完毕后，源代码在packaging/target/apache-hive-1.2.1-bin.tar.gz

## 部署hive ##

解压hive

tar -xzvf apache-hive-1.2.1-bin.tar.gz mv apache-hive-1.2.1-bin /home/hadooper/hive

在hive源码包中，打包web 界面

cd hwi/web zip -rq hive-hwi-1.2.1.war * mv hive-hwi-1.2.1.war /home/hadooper/hive/lib/

复制hive必须的包

cp /home/hadooper/hbase/lib/hbase* /home/hadooper/hive/lib/
    cp /home/hadooper/hadoop/lib/*.jar /home/hadooper/hive/lib/

修改hive目录的权限

chown -R hadooper.hadooper /home/hadooper/hive

创建hive的log目录

mkdir /home/hadooper/data/hive

chown -R hadooper.hadooper /home/hadooper/data/hive

mkdir /home/hadooper/hive/logs

chown -R hadooper.hadooper /home/hadooper/hive/logs

修改配置文件

#将模板复制成正常的配置文件 mv hive-default.xml.template hive-site.xml 修改里边的如下项 hive.exec.local.scratchdir = /home/hadooper/data/hive/hadooper hive.downloaded.resources.dir = /home/hadooper/data/hive/${hive.session.id}_resources hive.metastore.warehouse.dir=hdfs://hhts/hive/warehouse hive.metastore.uris=thrift://172.16.1.90:9083 javax.jdo.option.ConnectionPassword=mysql的密码 javax.jdo.option.ConnectionURL=mysql的链接参数，格式为jdbc:mysql://mysql地址:3306/数据库名称?createDatabaseIfNotExist=true javax.jdo.option.ConnectionUserName=mysql用户名 hive.aux.jars.path=file:///home/hadooper/hive/lib/hive-hbase-handler-1.2.1.jar,file:///home/hadooper/hive/lib/protobuf-java-2.5.0.jar,file:///home/hadooper/hive/lib/hbase-client-1.2.4.jar,file:///home/hadooper/hive/lib/hbase-common-1.2.4.jar,file:///home/hadooper/hive/lib/hbase-hadoop2-compat-1.2.4.jar,file:///home/hadooper/hive/lib/zookeeper-3.4.9.jar,file:///home/hadooper/hive/lib/guava-14.0.1.jar hive.querylog.location=/home/hadooper/data/hive/querylog hive.hwi.war.file=lib/hive-hwi-1.2.1.war hive.zookeeper.quorum=zookeeper节点1,zookeeper节点n hive.zookeeper.client.port=zookeeper端口号 hive.server2.logging.operation.log.location=/home/hadooper/data/hive/operation_logs

## 配置环境变量 ##

在/etc/profile中，配置如下

export ANT_HOME=/usr/local/ant export HIVE_HOME=/home/hadooper/hive export HCAT_HOME=$HIVE_HOME/hcatalog export HIVE_CONF=$HIVE_HOME/conf export hive_dependency=/home/hadooper/hive/conf:/home/hadooper/hive/lib/*:/home/hadooper/hive/hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-1.2.1.jar:/home/hadooper/hive/hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1.jar:/home/hadooper/hive/hcatalog/share/hcatalog/hive-hcatalog-server-extensions-1.2.1.jar:/home/hadooper/hive/hcatalog/share/hcatalog/hive-hcatalog-streaming-1.2.1.jar:/home/hadooper/hive/lib/hive-exec-1.2.1.jar PATH=$PATH:$HIVE_HOME/bin:$HCAT_HOME/bin CLASSPATH=.:$HIVE_HOME/lib

运行如下命令，即时生效

source /etc/profile

同时修改同样的内容到/home/hadooper/.baserc

## 启动hive ##

切换到hadooper用户

su hadooper cd /home/hadooper/hive/bin/

启动hive服务

nohup /home/hadooper/hive/bin/hive --service hiveserver2 > /home/hadooper/hive/logs/hiveserver2.log&

启动metastore服务

nohup /home/hadooper/hive/bin/hive --service metastore -p 9083 > /home/hadooper/hive/logs/metastore.log&

启动web界面服务

nohup /home/hadooper/hive/bin/hive --service hwi > /home/hadooper/hive/logs/hwi.log&