big-data:hive 搭建及遇到的问题

刺骨的言语ヽ痛彻心扉 2022-03-17 07:20 176阅读 0赞

配置hive-site.xml(这是一个简化版的配置，配置时，只需将下面的配置项配置，其余带有$的配置项目注释掉)

其中最重要的就是元数据存储的mysql的jdbc链接参数

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
    
            <property>
                    <name>javax.jdo.option.ConnectionURL</name>
                    <value>jdbc:mysql://192.168.48.50:3306/hive</value>
            </property>
    
            <property>
                    <name>javax.jdo.option.ConnectionDriverName</name>
                    <value>com.mysql.jdbc.Driver</value>
            </property>
    
            <property>
                    <name>javax.jdo.option.ConnectionUserName</name>
                    <value>root</value>
            </property>
    
            <property>
                    <name>javax.jdo.option.ConnectionPassword</name>
                    <value>root</value>
            </property>
    
            <property>
                    <name>hive.metastore.schema.verification</name>
                    <value>false</value>
            </property>
    
            <property>
                    <name>datanucleus.readOnlyDatastore</name>
                    <value>false</value>
            </property>
    
            <property> 
                    <name>datanucleus.fixedDatastore</name>
                    <value>false</value> 
            </property>
    
            <property> 
                    <name>datanucleus.autoCreateSchema</name> 
                    <value>true</value> 
            </property>
    
            <property>
                    <name>datanucleus.autoCreateTables</name>
                    <value>true</value>
            </property>
    
           <property>
                    <name>datanucleus.autoCreateColumns</name>
                    <value>true</value>
           </property>
           <property>
                   <name>hive.server2.authentication</name>
                   <value>NONE</value>
           </property>
           
    
    
           <property>
             <name>hive.server2.authentication</name>
             <value>NONE</value>
          </property>
    </configuration>

初始化元数据

schematool -dbType mysql -initSchema

启动

hive即可

如果外界需要访问，则需要启动metastore service

nohup hive --service metastore &   
nohup hive --service hiveserver2 &

通过beeline连接metastore service

!connect jdbc:hive2://master:10000

换了一台高配电脑之后，将mysql装在linux中，创建新的数据库名为hive,然后在hive中创建一张新表，报错如下

[Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections,][Execution Error_ return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException_message_For direct MetaStore DB connections]

原因：

编码不对（因为是自己手动建hive表,泪奔）

解决办法：

ALTER DATABASE hive CHARACTER SET latin1;

第一次启动hive报错：

MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)  
解决办法：

将hive.metastore.uris 的value 置空

第一次创建表报错

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(  
message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=anonymous, access=WRITE, inode="/":hadoop:supergroup:drwxr-xr-x

[Execution Error_ return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException_message_For direct MetaStore DB connections]: https://www.cnblogs.com/bianqi/p/6953570.html