【Sqoop】Sqoop辅助工具——数据的导入与导出

r囧r小猫 2023-10-05 12:23 8阅读 0赞

#### Sqoop ####

*   *  概述
     *  安装
     *  基本使用
     *  sqoop-import 导入
     *  sqoop-export 导出

### 概述 ###

Apache Sqoop（TM）是一种旨在在Apache Hadoop和结构化数据存储（例如关系数据库）之间高效传输批量数据的工具。通过内嵌的MapReduce程序实现关系型数据库和HDFS、Hbase、Hive等数据的倒入导出。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODIzMTQ0OA_size_16_color_FFFFFF_t_70_pic_center]

### 安装 ###

1、访问sqoop的网址http://sqoop.apache.org/，选择相应的sqoop版本下载，本案例选择下载的是`1.4.7`，下载完相应的工具包后，解压Sqoop:

[root@CentOS ~]# tar -zxf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /usr/
    [root@CentOS ~]# cd /usr/
    [root@CentOS usr]# mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop-1.4.7
    [root@CentOS ~]# cd /usr/sqoop-1.4.7/

2、配置SQOOP\_HOME环境变量

[root@CentOS sqoop-1.4.7]# vi ~/.bashrc 
    M2_HOME=/usr/apache-maven-3.6.3
    SQOOP_HOME=/usr/sqoop-1.4.7
    HIVE_HOME=/usr/apache-hive-1.2.2-bin
    JAVA_HOME=/usr/java/latest
    HADOOP_HOME=/usr/hadoop-2.9.2/
    HBASE_HOME=/usr/hbase-1.2.4/
    ZOOKEEPER_HOME=/usr/zookeeper-3.4.6
    PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HBASE_HOME/bin:$M2_HOME/bin:$HIVE_HOME/bin:$SQOOP_HOME/bin:$ZOOKEEPER_HOME/bin
    CLASSPATH=.
    export JAVA_HOME
    export PATH
    export CLASSPATH
    export HADOOP_HOME
    export HBASE_HOME
    HBASE_CLASSPATH=$(/usr/hbase-1.2.4/bin/hbase classpath)
    HADOOP_CLASSPATH=/root/mysql-connector-java-5.1.49.jar
    export HADOOP_CLASSPATH
    export M2_HOME
    export HIVE_HOME
    export SQOOP_HOME
    export ZOOKEEPER_HOME
    [root@CentOS sqoop-1.4.7]# source ~/.bashrc

3、修改conf下的sqoop-env.sh.template配置文件

[root@CentOS7 sqoop-1.4.7]#  mv conf/sqoop-env-template.sh conf/sqoop-env.sh
    [root@CentOS7 sqoop-1.4.7]# vi conf/sqoop-env.sh

# Licensed to the Apache Software Foundation (ASF) under one or more
    # contributor license agreements.  See the NOTICE file distributed with
    # this work for additional information regarding copyright ownership.
    # The ASF licenses this file to You under the Apache License, Version 2.0
    # (the "License"); you may not use this file except in compliance with
    # the License.  You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    # included in all the hadoop scripts with source command
    # should not be executable directly
    # also should not be passed any arguments, since we need original $*

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODIzMTQ0OA_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/20201015174525864.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODIzMTQ0OA==,size_16,color_FFFFFF,t_70#pic_center