CentOS 7部署Hive 3.1.0

ゝ一世哀愁。 2022-04-17 00:44 322阅读 0赞

# 一、环境 #

<table> 
 <thead> 
  <tr> 
   <th>软件</th> 
   <th>版本</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>Hadoop</td> 
   <td>3.1.1</td> 
  </tr> 
  <tr> 
   <td>Hive</td> 
   <td>3.1.0</td> 
  </tr> 
  <tr> 
   <td>MySQL</td> 
   <td>8.0.11</td> 
  </tr> 
  <tr> 
   <td>JDK</td> 
   <td>1.8</td> 
  </tr> 
 </tbody> 
</table>

# 二、安装Hive #

从[http://hive.apache.org/downloads.html][http_hive.apache.org_downloads.html]下载Hive 3.1.0。在安装Hive前，要先搭建Hadoop集群，Hadoop集群搭建可以参考[CentOS 7搭建Apache Hadoop 3.1.1集群][CentOS 7_Apache Hadoop 3.1.1]。

Hive只要在一台Hadoop集群的服务器上安装即可，安装到/usr/local/hive目录下。执行如下命令：

**mkdir -p /usr/local/hive  
tar -zxvf apache-hive-3.1.0-bin.tar.gz -C /usr/local/hive**

接下来配置环境变量，执行命令：vim /etc/profile，打开profile文件，在文件末尾添加Hive配置：

export HIVE_HOME=/usr/local/hive/apache-hive-3.1.0-bin
    export PATH=$PATH:$HIVE_HOME/bin

配置好Hive环境变量后，执行命令：source /etc/profile，使环境配置生效。

# 三、配置Hive #

执行命令：cd /usr/local/hive/apache-hive-3.1.0-bin/conf，进入Hive配置目录，执行如下命令复制配置文件：

cp hive-env.sh.template hive-env.sh
    cp hive-default.xml.template hive-site.xml
    cp hive-log4j2.properties.template hive-log4j2.properties

#### 1、配置hive-env.h ####

配置java，hadoop，hive目录，如下：

export JAVA_HOME=/usr/local/java/jdk1.8.0_181
    export HADOOP_HOME=/usr/local/hadoop/hadoop-3.1.1
    export HIVE_HOME=/usr/local/hive/apache-hive-3.1.0-bin
    export HIVE_CONF_DIR=$HIVE_HOME/conf
    export HIVE_AUX_JARS_PATH=$HIVE_HOME/lib/*

#### 2、配置hive-site.xml ####

配置MySQL地址、登录名、密码、驱动等。  
下载MySQL驱动并复制到lib目录下：cp mysql-connector-java-8.0.11.jar /usr/local/hive/apache-hive-3.1.0-bin/lib。  
配置如下：

<property>
        <name>system:java.io.tmpdir</name>
        <value>/usr/local/hive/apache-hive-3.1.0-bin/tmp</value>
    </property>
    <property>
        <name>system:user.name</name>
        <value>${user.name}</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
        <description>Username to use against metastore database</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>mysql</value>
        <description>password to use against metastore database</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://192.168.0.199:3306/hive?createDatabaseIfNotExist=true&characterEncoding=UTF-8&useSSL=false</value>
        <description>
            JDBC connect string for a JDBC metastore.
            To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
            For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
        </description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.cj.jdbc.Driver</value>
        <description>Driver class name for a JDBC metastore</description>
    </property>
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
        <description>
            Enforce metastore schema version consistency.
            True: Verify that version information stored in is compatible with one from Hive jars.  Also disable automatic
            schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures
            proper metastore schema migration. (Default)
            False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
        </description>
    </property>

#### 3、配置hive-log4j2.properties ####

新建并配置日志目录：

property.hive.log.dir = /usr/local/hive/apache-hive-3.1.0-bin/logs

# 四、启动Hive #

在启动Hive前，先在MySQL下创建Hive数据库，执行如下命令：  
**schematool -dbType mysql -initSchema**  
创建好数据库后，执行命令：hive，启动并进入hive，如下图所示：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NlYWxpcg_size_16_color_FFFFFF_t_70]

# 五、验证Hive #

hive常用命令：

show databases;																	//查看数据库
    create database database_name;													//新建数据库	
    use database_name;																//使用数据库
    drop database database_name;													//删除数据库
    show tables;																	//查看数据表
    show tables in database_name;													//查看数据库下的表
    create table ht_big_data (id int, name string)  row format delimited fields terminated by '\t';		//建表
    show create table table_name;																		//查看建表语句
    load data local inpath 'bigdata.txt' overwrite into table ht_big_data;								//加载数据到表中
    load data inpath '/hdfs/hive/bigdata.txt' into table ht_big_data;									//从hdfs中追加数据到hive表
    select * from table_name;																			//查看数据

[http_hive.apache.org_downloads.html]: http://hive.apache.org/downloads.html
[CentOS 7_Apache Hadoop 3.1.1]: https://blog.csdn.net/sealir/article/details/81953727
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NlYWxpcg_size_16_color_FFFFFF_t_70]: /images/20220417/272b77948c204a3980457df108003372.png