CentOS7搭建部署Ambari 2.6.2.0最新版(HDP-UTILS、HDP-GPL)大数据平台

柔情只为你懂 2022-09-11 04:11 132阅读 0赞

如上述博客有任何错误或者疑问，请加VX：1755337994，及时告知！万分感激！

> 注：本文基于root用户操作

# 一、安装环境准备 #

> 操作系统 centos7.5  
> hdc-data1：192.168.163.51  
> hdc-data2：192.168.163.52  
> hdc-data3：192.168.163.53  
> 【安装环境准备每台集群机器一样的操作，或者使用scp远程拷贝】

### 1、主机名IP映射配置 ###

FQDN：(Fully Qualified Domain Name)全限定域名：同时带有主机名和域名的名称。（通过符号“.”）  
例如：主机名是bigserver,域名是mycompany.com,那么FQDN就是bigserver.mycompany.com。

vi /etc/hosts
    #添加如下地址映射及FQDN（ambari注册时需要）
    192.168.163.51 hdc-data1 hdc-data1.hadoop
    192.168.163.52 hdc-data2 hdc-data2.hadoop
    192.168.163.53 hdc-data3 hdc-data3.hadoop

### 2、SSH免密登录配置 ###

ssh-keygen
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@hdc-data1
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@hdc-data2
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@hdc-data3

> ![397df1eed5e3e49bc72b88089d41b64b.png][]
> 
> image.png

### 3、关闭及禁止防火墙 ###

systemctl stop firewalld.service
    systemctl disable firewalld.service

### 4、关闭SELinux ###

> SELinux(Security-Enhanced Linux) 是美国国家安全局（NSA）对于强制访问控制的实现，是 Linux历史上最杰出的新安全子系统。NSA是在Linux社区的帮助下开发了一种访问控制体系，在这种访问控制体系的限制下，进程只能访问那些在他的任务中所需要文件。

vi /etc/sysconfig/selinux
    #修改以下内容
    SELINUX=disabled

> ![4761dadc319a7f05b0570c20508d5664.png][]
> 
> image.png

### 5、开启NTP服务 ###

yum install -y ntp
    systemctl enable ntpd 
    systemctl start ntpd

### 6、安装JDK ###

> 下载地址：[https://www.oracle.com/technetwork/cn/java/javase/downloads/jdk8-downloads-2133151-zhs.html][https_www.oracle.com_technetwork_cn_java_javase_downloads_jdk8-downloads-2133151-zhs.html]

通过文件传输上传到服务器，这里不可用wget直接下载。

mkdir -p /opt/java
    tar -zxvf jdk-8u181-linux-x64.tar.gz -C /opt/java/
    vi /etc/profile
    
    export JAVA_HOME=/opt/java/jdk1.8.0_181
    export JRE_HOME=$JAVA_HOME/jre
    export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
    export PATH=$PATH:$JAVA_HOME/bin

远程分发到其他服务器

scp -r /opt/java/jdk1.8.0_181/ root@hdc-data2:/opt/java/
    scp -r /opt/java/jdk1.8.0_181/ root@hdc-data3:/opt/java/
    
    scp /etc/profile root@hdc-data2:/etc/
    scp /etc/profile root@hdc-data3:/etc/
    
    source /etc/profile

# 二、Ambari安装 #

### 1、制作Ambari本地yum源 ###

选取一台机器即可，在此选择hdc-data1。

> [https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.0/bk\_ambari-installation/content/ch\_obtaining-public-repos.html][https_docs.hortonworks.com_HDPDocuments_Ambari-2.6.2.0_bk_ambari-installation_content_ch_obtaining-public-repos.html]
> 
> ![40abb651399b9de9dc751b2a9ce31a26.png][]
> 
> image.png
> 
> ![82f51b718adda1cff0d4e38a596bd985.png][]
> 
> image.png
> 
> ![930b67e45fa1017f565133adb1880e32.png][]
> 
> image.png

需要下载资源安装包如下：

wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari-2.6.2.0-centos7.tar.gz
    wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz
    wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
    wget http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/HDP-GPL-2.6.5.0-centos7-gpl.tar.gz

1.1 安装 Apache HTTP 服务器

yum install httpd -y
    #开启服务并设置开机自启动
    systemctl start httpd.service
    systemctl enable httpd.service

1.2 安装制作yum源工具

yum install yum-utils createrepo

1.3 创建http服务器

http根目录默认是即/var/www/html/

mkdir -p /var/www/html/ambari
    #cd /var/www/html/ambari
    #将下载好的安装包上传并解压
    tar xvf HDP-2.6.5.0-centos7-rpm.tar.gz -C /var/www/html/ambari
    tar xvf ambari-2.6.2.0-centos7.tar.gz -C /var/www/html/ambari
    tar xvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/ambari
    tar xvf HDP-GPL-2.6.5.0-centos7-gpl.tar.gz -C /var/www/html/ambari
    
    # 删除压缩包
    rm -rf ambari-2.6.2.0-centos7.tar.gz
    rm -rf HDP-2.6.5.0-centos7-rpm.tar.gz
    rm -rf HDP-UTILS-1.1.0.22-centos7.tar.gz
    rm -rf HDP-GPL-2.6.5.0-centos7-gpl.tar.gz

验证

> ![7cac8485876f51e7fb4a77ce2bda2fcd.png][]
> 
> image.png

1.4 配置ambari、HDP、HDP-UTILS的本地源

#yum install wget -y
    cd /etc/yum.repos.d/
    wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo
    wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo
    wget http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/hdp.gpl.repo

编辑ambari.repo，修改baseurl和gpgkey

[root@hdc-data1 yum.repos.d]# vi ambari.repo 
    
    #VERSION_NUMBER=2.6.2.0-155
    [ambari-2.6.2.0]
    name=ambari Version - ambari-2.6.2.0
    #baseurl=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0
    baseurl=http://192.168.163.51/ambari/ambari/centos7/2.6.2.0-155
    gpgcheck=1
    #gpgkey=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    gpgkey=http://192.168.163.51/ambari/ambari/centos7/2.6.2.0-155/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    enabled=1
    priority=1

编辑hdp.repo，修改baseurl和gpgkey

[root@hdc-data1 yum.repos.d]# vi hdp.repo 
    
    #VERSION_NUMBER=2.6.5.0-292
    [HDP-2.6.5.0]
    name=HDP Version - HDP-2.6.5.0
    #baseurl=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0
    baseurl=http://192.168.163.51/ambari/HDP/centos7/2.6.5.0-292
    gpgcheck=1
    #gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    gpgkey=http://192.168.163.51/ambari/HDP/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    enabled=1
    priority=1
    
    
    [HDP-UTILS-1.1.0.22]
    name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
    #baseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7
    baseurl=http://192.168.163.51/ambari/HDP-UTILS/centos7/1.1.0.22
    gpgcheck=1
    #gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    gpgkey=http://192.168.163.51/ambari/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    enabled=1
    priority=1

编辑hdp.gpl.repo，修改baseurl和gpgkey

[root@hdc-data1 yum.repos.d]# vi hdp.gpl.repo 
    
    #VERSION_NUMBER=2.6.5.0-292
    [HDP-GPL-2.6.5.0]
    name=HDP-GPL Version - HDP-GPL-2.6.5.0
    #baseurl=http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0
    baseurl=http://192.168.163.51/ambari/HDP-GPL/centos7/2.6.5.0-292
    gpgcheck=1
    #gpgkey=http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    gpgkey=http://192.168.163.51/ambari/HDP-GPL/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    enabled=1
    priority=1

分发到其他机器

scp /etc/yum.repos.d/ambari.repo root@hdc-data2:/etc/yum.repos.d/
    scp /etc/yum.repos.d/ambari.repo root@hdc-data3:/etc/yum.repos.d/
    scp /etc/yum.repos.d/hdp.repo root@hdc-data2:/etc/yum.repos.d/
    scp /etc/yum.repos.d/hdp.repo root@hdc-data3:/etc/yum.repos.d/
    scp /etc/yum.repos.d/hdp.gpl.repo root@hdc-data2:/etc/yum.repos.d/
    scp /etc/yum.repos.d/hdp.gpl.repo root@hdc-data3:/etc/yum.repos.d/

每台机器yum配置

yum clean all
    yum makecache
    yum list

### 2、安装Mysql数据库 ###

> Ambari安装会将安装等信息写入数据库，建议使用MariaDB数据库，也可以不安装而使用默认数据库PostgreSQL

2.1 安装及初始化设置

[root@hdc-data1 ~]# yum install mariadb-server
    [root@hdc-data1 ~]# systemctl start mariadb
    [root@hdc-data1 ~]# systemctl enable mariadb
    [root@hdc-data1 ~]# mysql_secure_installation
    
    #首先是设置密码，会提示先输入密码
    Enter current password for root (enter for none):<–初次运行直接回车
    #设置密码
    Set root password? [Y/n] <– 是否设置root用户密码，输入y并回车或直接回车
    New password: <– 设置root用户的密码
    Re-enter new password: <– 再输入一次你设置的密码
    #其他配置
    Remove anonymous users? [Y/n] <– 是否删除匿名用户，回车
    Disallow root login remotely? [Y/n] <–是否禁止root远程登录,回车,
    Remove test database and access to it? [Y/n] <– 是否删除test数据库，回车
    Reload privilege tables now? [Y/n] <– 是否重新加载权限表，回车

> **【可选】修改mysql端口**（生产环境安全考虑）  
> 查看端口
> 
> ![be2e0ac30b831a5db31d78e90381d5e1.png][]
> 
> image.png
> 
> ![67b4759d67d11e3719e2031dca93618c.png][]
> 
> image.png

2.2 安装完成后创建ambari数据库及用户

#进入mysql shell
    mysql -uroot -p

create database ambari character set utf8 ;  
    CREATE USER 'ambari'@'%'IDENTIFIED BY 'ambari123';
    GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
    FLUSH PRIVILEGES;

如果要安装Hive，创建Hive数据库和用户

create database hive character set utf8 ;  
    CREATE USER 'hive'@'%'IDENTIFIED BY 'hive123';
    GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
    FLUSH PRIVILEGES;

如果要安装Oozie，创建Oozie数据库和用户

create database oozie character set utf8 ;  
    CREATE USER 'oozie'@'%'IDENTIFIED BY 'oozie123';
    GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';
    FLUSH PRIVILEGES;

### 3、安装 ambari ###

yum install ambari-server

### 4、下载mysql驱动 ###

> [http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.40/][http_central.maven.org_maven2_mysql_mysql-connector-java_5.1.40]  
> 将mysql-connector-Java.jar复制到/usr/share/java目录下

mkdir /usr/share/java
    cp mysql-connector-java-5.1.40.jar /usr/share/java/mysql-connector-java.jar

将mysql-connector-java.jar复制到/var/lib/ambari-server/resources目录下

cp mysql-connector-java-5.1.40.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar

### 5、编辑/etc/ambari-server/conf/ambari.properties，添加如下内容 ###

server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar
    #【可选】修改默认8080端口
    #client.api.port=18080

### 6、ambaria初始化 ###

#设置mysql驱动
    ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar 
    ambari-server setup

完整初始化如下：

[root@hdc-data1 ~]# ambari-server setup
    Using python  /usr/bin/python
    Setup ambari-server
    Checking SELinux...
    SELinux status is 'disabled'
    Customize user account for ambari-server daemon [y/n] (n)? y
    Enter user account for ambari-server daemon (root):
    Adjusting ambari-server permissions and ownership...
    Checking firewall status...
    Checking JDK...
    [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
    [2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
    [3] Custom JDK
    ==============================================================================
    Enter choice (1): 3
    WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
    WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
    Path to JAVA_HOME: /opt/java/jdk1.8.0_181
    Validating JDK on Ambari Server...done.
    Checking GPL software agreement...
    GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
    Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y
    Completing setup...
    Configuring database...
    Enter advanced database configuration [y/n] (n)? y
    Configuring database...
    ==============================================================================
    Choose one of the following options:
    [1] - PostgreSQL (Embedded)
    [2] - Oracle
    [3] - MySQL / MariaDB
    [4] - PostgreSQL
    [5] - Microsoft SQL Server (Tech Preview)
    [6] - SQL Anywhere
    [7] - BDB
    ==============================================================================
    Enter choice (1): 3
    Hostname (localhost): 
    Port (3306): 
    Database name (ambari): 
    Username (ambari): 
    Enter Database Password (bigdata): 
    Re-enter password: 
    Configuring ambari database...
    Configuring remote database connection properties...
    WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
    Proceed with configuring remote database connection properties [y/n] (y)? y
    Extracting system views...
    .....ambari-admin-2.6.2.0.155.jar
    ......
    Adjusting ambari-server permissions and ownership...
    Ambari Server 'setup' completed successfully.

> 说明：  
> 1：提示是否自定义设置。输入：y 或 按回车继续  
> 2：ambari-server 账号。使用root（推荐）用户直接回车 若使用ambari用户输入：ambari  
> 3：设置JDK,输入数字3，选客户端安装jdk，如果服务器可以访问外网，则可以选1，自动下载jdk1.8，默认下载的安装目录是/usr/java/default  
> 4：如果上面选择3自定义JDK,则需要设置JAVA\_HOME  
> 5：输入：y 或 按回车继续  
> 6：输入“y“，进行数据库的高级配置  
> 7：输入“3“，选择mysql/mariadb数据库  
> 8：设置数据库的配置参数：host,port,database,username,password。根据实际情况输入，如果和括号内相同，则可以直接回车。若端口改6033则输入port:6033。  
> 9：输入：y 或 按回车继续  
> （若设置更改，可以再次执行`ambari-server setup`进行设置）

### 7、将Ambari数据库脚本导入到数据库 ###

#用Ambari用户（上面设置的用户）登录mysql
    mysql -u ambari -p
    use ambari;
    source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

> ![b578e6f30908b38415304d330639cfcc.png][]
> 
> image.png

### 8、启动Ambari ###

ambari-server start

> ![668a1ea2622c207ccfae90437eceac7a.png][]
> 
> image.png
> 
>   
> 浏览器访问： [http://hdc-data1:8080/][http_hdc-data1_8080] 默认登录用户：admin，密码：admin

### 9、WEBUI配置 ###

> ![9327d14203d6eb11571628b1d160b6d5.png][]
> 
> image.png
> 
> 创建集群名称
> 
> ![7b2076a89fc6dcf58cd4a3fc5109a466.png][]
> 
> image.png
> 
> 选择HDP版本和选择本地仓库
> 
> ![88c39db8d18461271482516837b4dc8a.png][]
> 
> image.png
> 
>   
> 配置HDP的Repository
> 
> ![4dc5c4150642911a2898ebe30b87a7ac.png][]
> 
> image.png
> 
>   
> 输入集群节点host(FQDN)和Ambari节点SSH的私钥
> 
> ![b2c2774ddf02ddf9ef703e57829411dd.png][]
> 
> image.png
> 
> 等待ambari-agents注册
> 
> ![813e5680f9a65fffc7db8b66f179f3c7.png][]
> 
> image.png
> 
>   
> 若报错
> 
> ![e7488407d8c74ac7fe70a1745437da3d.png][]
> 
> image.png
> 
> ![6ccdfe3ced0cb451a6c580138a50e2ba.png][]
> 
> image.png

【遇到问题 Confirm Hosts】  
注册ambari-agents时failed  
NetUtil.py:96 - EOF occurred in violation of protocol (\_ssl.c:579)  
SSLError: Failed to connect. Please check openssl library versions.  
解决：如下第三个方案，/etc/ambari-agent/conf/ambari-agent.ini文件只有执行该步骤进行注册才生成。修改后点击“Retry Failed”按钮，等待注册成功进入下一步。

# 1、yum upgrade openssl 已是最新本版，未解决
    # 2、vi /etc/python/cert-verification.cfg 修改 verify=disable，未解决
    [https]
    #verify=platform_default
    verify=disable
    # 3、最后解决方案 在ambari-agent的配置文件/etc/ambari-agent/conf/ambari-agent.ini
     在 ［security］标签下面增加一项
    [security]
    force_https_protocol=PROTOCOL_TLSv1_2

> ![2824fe2af652e056f383d919a3173e41.png][]
> 
> image.png

> 选择安装组件，（最小化安装HDFS+YARN+MAPREDUCE2+Ambari Metrics+SmartSense+ZooKeeper+Hbase）不必要的组件可以暂时不安装节省安装时间，后续可以再添加安装组件
> 
> ![e445cf7a52cd1b30af628959c4f97020.png][]
> 
> image.png
> 
> ![ff05d37d39da7cbd619e2b20d953833f.png][]
> 
> image.png
> 
> 实际部署时需按规划设计进行分配（注意：Hbase HA 的话在Hbase Master 点击后面的小绿色+号，添加standby master）：
> 
> ![9f1baa3d0e0a5f7bb03788e9bf03d500.png][]
> 
> image.png
> 
>   
> 工作节点角色分配(生产环境全勾了)
> 
> ![ee286c13710eaf8d46aff2589e5d0a12.png][]
> 
> image.png
> 
>   
> 组件配置信息修改  
> 通常数据目录修改到/data/目录下，有红色信息提示的组件表示要输入相应的账号密码。以下示例，可根据自己的实际情况配置。
> 
> HDFS
> 
> ![ec96e36b1da2618f0d2edd9cd0ad1d92.png][]
> 
> image.png
> 
> ![888a316dddf0182347583a845b8c1a69.png][]
> 
> image.png
> 
>   
> HIVE,若数据库端口修改按实际填写
> 
> ![c6f931d3842ed23229afe190d445ee8b.png][]
> 
> image.png
> 
>   
> Oozie
> 
> ![704beb8794523908e79c2c2005d34f52.png][]
> 
> image.png
> 
>   
> Zookeeper
> 
> ![173ba904280b14a6f0964144b01a19e2.png][]
> 
> image.png
> 
> ![f6536b31e886a0b6693ce9bfd2263c8b.png][]
> 
> image.png
> 
> ![665b1983d95b8f1778c483cd3f943112.png][]
> 
> image.png
> 
> ![ecd5c020e23639b65bcc8082f8dcf7ef.png][]
> 
> image.png

暂时修改了以下属性，具体配置信息可根据实际情况后续相应修改

<table> 
 <thead> 
  <tr> 
   <th>端口属性</th> 
   <th>修改值</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>dfs.namenode.http-address</td> 
   <td>server1.hadoop:5070 （def:50070）</td> 
  </tr> 
  <tr> 
   <td>yarn.resourcemanager.webapp.address</td> 
   <td>server2.hadoop:8888 （def:8088）</td> 
  </tr> 
  <tr> 
   <td>yarn.resourcemanager.webapp.https.address</td> 
   <td>server2.hadoop:8890(def:8090)</td> 
  </tr> 
  <tr> 
   <td>mapreduce.jobhistory.webapp.address</td> 
   <td>server2.hadoop:18888 (def:19888)</td> 
  </tr> 
 </tbody> 
</table>

<table> 
 <thead> 
  <tr> 
   <th>属性</th> 
   <th>修改值</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td><strong>HDFS</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>NameNode</td> 
   <td>/data/hadoop/hdfs/namenode</td> 
  </tr> 
  <tr> 
   <td>DataNode</td> 
   <td>/data/hadoop/hdfs/data</td> 
  </tr> 
  <tr> 
   <td>SecondaryNameNode Checkpoint directories</td> 
   <td>/data/hadoop/hdfs/namesecondary</td> 
  </tr> 
  <tr> 
   <td>Hadoop PID Dir Prefix</td> 
   <td>/data/var/run/hadoop</td> 
  </tr> 
  <tr> 
   <td>Hadoop Log Dir Prefix</td> 
   <td>/data/var/log/hadoop</td> 
  </tr> 
  <tr> 
   <td>dfs.journalnode.edits.dir</td> 
   <td>/data/hadoop/hdfs/journalnode</td> 
  </tr> 
  <tr> 
   <td><strong>Yarn</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>yarn.nodemanager.local-dirs</td> 
   <td>/data/hadoop/yarn/local</td> 
  </tr> 
  <tr> 
   <td>yarn.nodemanager.log-dirs</td> 
   <td>/data/hadoop/yarn/log</td> 
  </tr> 
  <tr> 
   <td>yarn.timeline-service.leveldb-state-store.path</td> 
   <td>/data/hadoop/yarn/timeline</td> 
  </tr> 
  <tr> 
   <td>yarn.timeline-service.leveldb-timeline-store.path</td> 
   <td>/data/hadoop/yarn/timeline</td> 
  </tr> 
  <tr> 
   <td>YARN Log Dir Prefix</td> 
   <td>/data/var/log/hadoop-yarn</td> 
  </tr> 
  <tr> 
   <td>YARN PID Dir Prefix</td> 
   <td>/data/var/run/hadoop-yarn</td> 
  </tr> 
  <tr> 
   <td><strong>Mapreduce</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>Mapreduce Log Dir Prefix</td> 
   <td>/data/var/log/hadoop-mapreduce</td> 
  </tr> 
  <tr> 
   <td>Mapreduce PID Dir Prefix</td> 
   <td>/data/var/run/hadoop-mapreduce</td> 
  </tr> 
  <tr> 
   <td>mapreduce.jobhistory.recovery.store.leveldb.path</td> 
   <td>/data/hadoop/mapreduce/jhs</td> 
  </tr> 
  <tr> 
   <td><strong>Hive</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>Hive Log Dir</td> 
   <td>/data/var/log/hive</td> 
  </tr> 
  <tr> 
   <td>Hive PID Dir</td> 
   <td>/data/var/run/hive</td> 
  </tr> 
  <tr> 
   <td><strong>HBase</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>HBase Log Dir Prefix</td> 
   <td>/data/var/log/hbase</td> 
  </tr> 
  <tr> 
   <td>HBase PID Dir</td> 
   <td>/data/var/run/hbase</td> 
  </tr> 
  <tr> 
   <td><strong>Oozie</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>Oozie Data Dir</td> 
   <td>/data/hadoop/oozie/data</td> 
  </tr> 
  <tr> 
   <td>Oozie Log Dir</td> 
   <td>/data/var/log/oozie</td> 
  </tr> 
  <tr> 
   <td>Oozie PID Dir</td> 
   <td>/data/var/run/oozie</td> 
  </tr> 
  <tr> 
   <td><strong>zookeeper</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>ZooKeeper directory</td> 
   <td>/data/hadoop/zookeeper</td> 
  </tr> 
  <tr> 
   <td>ZooKeeper Log Dir</td> 
   <td>/data/var/log/zookeeper</td> 
  </tr> 
  <tr> 
   <td>ZooKeeper PID Dir</td> 
   <td>/data/var/run/zookeeper</td> 
  </tr> 
  <tr> 
   <td><strong>ambari-infra</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>Metrics Collector log dir</td> 
   <td>/data/var/log/ambari-metrics-collector</td> 
  </tr> 
  <tr> 
   <td>Metrics Collector pid dir</td> 
   <td>/data/var/run/ambari-metrics-collector</td> 
  </tr> 
  <tr> 
   <td>Metrics Monitor log dir</td> 
   <td>/data/var/log/ambari-metrics-monitor</td> 
  </tr> 
  <tr> 
   <td>Metrics Monitor pid dir</td> 
   <td>/data/var/run/ambari-metrics-monitor</td> 
  </tr> 
  <tr> 
   <td>Aggregator checkpoint directory</td> 
   <td>/data/var/lib/ambari-metrics-collector/checkpoint</td> 
  </tr> 
  <tr> 
   <td>Metrics Grafana data dir</td> 
   <td>/data/var/lib/ambari-metrics-grafana</td> 
  </tr> 
  <tr> 
   <td>Metrics Grafana log dir</td> 
   <td>/data/var/log/ambari-metrics-grafana</td> 
  </tr> 
  <tr> 
   <td>Metrics Grafana pid dir</td> 
   <td>/data/var/run/ambari-metrics-grafana</td> 
  </tr> 
  <tr> 
   <td>hbase_log_dir</td> 
   <td>/data/var/log/ambari-metrics-collector</td> 
  </tr> 
  <tr> 
   <td>hbase_pid_dir</td> 
   <td>/data/var/run/ambari-metrics-collector/</td> 
  </tr> 
  <tr> 
   <td>hbase.tmp.dir</td> 
   <td>/data/var/lib/ambari-metrics-collector/hbase-tmp</td> 
  </tr> 
  <tr> 
   <td></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td><strong>ambari-infra</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>Infra Solr Client log dir</td> 
   <td>/data/var/log/ambari-infra-solr-client</td> 
  </tr> 
  <tr> 
   <td>Infra Solr log dir</td> 
   <td>/data/var/log/ambari-infra-solr</td> 
  </tr> 
  <tr> 
   <td>Infra Solr pid dir</td> 
   <td>/data/var/run/ambari-infra-solr</td> 
  </tr> 
  <tr> 
   <td><strong>spark</strong></td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>livy2_log_dir</td> 
   <td>/data/var/log/livy2</td> 
  </tr> 
  <tr> 
   <td>livy2_pid_dir</td> 
   <td>/data/var/run/livy2</td> 
  </tr> 
  <tr> 
   <td>spark_log_dir</td> 
   <td>/data/var/log/spark2</td> 
  </tr> 
  <tr> 
   <td>spark_pid_dir</td> 
   <td>/data/var/run/spark2</td> 
  </tr> 
 </tbody> 
</table>

> 等待安装完成
> 
> ![c2c9008f5d63d896432e2406aae243df.png][]
> 
> image.png
> 
> 直到所有节点成功安装完成才能进行下一步操作
> 
> ![bf39836941604cdcbf45ab0e4db2177b.png][]
> 
> image.png

最终结果界面类似如下：

![2392697e0a727fe0a0789b4cf2180170.png][]

image.png

**安装完成！**

# 三、后续操作 #

操作前最好先将集群打个快照备份

### 1、HDFS启用HA(高可用) ###

参考[http://www.louisvv.com/archives/1490.html][http_www.louisvv.com_archives_1490.html]

> **step1：**关闭hbase、hive相关服务
> 
> ![dc6cf26bc64f5c8ce09b0e2f47528ca6.png][]
> 
> image.png
> 
>   
> **step2:**在HDFS选择Enable NameNode HA
> 
> ![d96942728e9a2ce3ef87b3734a9e893d.png][]
> 
> image.png
> 
>   
> **step3:**输入namenode ha serverid
> 
> ![9f126f43d29e516a3bc3b76870d30a0b.png][]
> 
> image.png
> 
>   
> **step4:**默认，进入下一步
> 
> ![ed9a0e4f7e6a5c7ec0158faac29806ac.png][]
> 
> image.png
> 
>   
> **step5:**预览，直接进入下一步
> 
> ![51cfe85ecbeae4c8a37f4d4e7ff556aa.png][]
> 
> image.png
> 
>   
> **step6:**创建checkpoint，根据提示执行下列命令
> 
> ![988a5a730ead53709dbace9dda038459.png][]
> 
> image.png
> 
>   
> 根据提示在server1执行下列两条命令
> 
> ![68d5662624b395dd04a84ca1ec96fec8.png][]
> 
> image.png
> 
>   
> 命令执行完成方可进入下一步
> 
> ![9de130eab30de61047c758235b887bac.png][]
> 
> image.png
> 
>   
> **step7:**等待配置进程完成
> 
> ![8766c90a59fde85c9a6c58c951480abb.png][]
> 
> image.png
> 
>   
> **step8:**Journalnode初始化
> 
> ![0541d26a85dcaac4ec1d7ccd5adc8e8b.png][]
> 
> image.png
> 
>   
> **需要在原NameNode节点执行下列命令**

sudo su hdfs -l -c 'hdfs namenode -initializeSharedEdits'

> ![f1754d0b7452e40f0d324a92b3139315.png][]
> 
> image.png
> 
>   
> **step9:** 启动组件，进入下一步
> 
> ![db6eefbedca3ac11ad96b50007de5c70.png][]
> 
> image.png
> 
>   
> **step10:**初始化元数据，

sudo su hdfs -l -c 'hdfs zkfc -formatZK'

> ![7754dfa727cb206bb247f16058203737.png][]
> 
> image.png
> 
>   
> **注意**在 **新添加的NameNode节点**上操作

sudo su hdfs -l -c 'hdfs namenode -bootstrapStandby'

> ![682cef2f9828ac472edf73da95f716bd.png][]
> 
> image.png
> 
> ![07012c88eb224701718459bd4f582427.png][]
> 
> image.png
> 
>   
> **step11:**等待所有服务重启，完成
> 
> ![f75c8e873cd63f0e2609a52b17bd2b74.png][]
> 
> image.png
> 
> ![fd523b59bc83b21550d2bfc138919f11.png][]
> 
> image.png

[397df1eed5e3e49bc72b88089d41b64b.png]: /images/20220828/97d4992a2085474eab3b8177083abbc8.png
[4761dadc319a7f05b0570c20508d5664.png]: /images/20220828/790ffc0606db4a63820ac207896d870d.png
[https_www.oracle.com_technetwork_cn_java_javase_downloads_jdk8-downloads-2133151-zhs.html]: https://links.jianshu.com/go?to=https%3A%2F%2Fwww.oracle.com%2Ftechnetwork%2Fcn%2Fjava%2Fjavase%2Fdownloads%2Fjdk8-downloads-2133151-zhs.html
[https_docs.hortonworks.com_HDPDocuments_Ambari-2.6.2.0_bk_ambari-installation_content_ch_obtaining-public-repos.html]: https://links.jianshu.com/go?to=https%3A%2F%2Fdocs.hortonworks.com%2FHDPDocuments%2FAmbari-2.6.2.0%2Fbk_ambari-installation%2Fcontent%2Fch_obtaining-public-repos.html
[40abb651399b9de9dc751b2a9ce31a26.png]: /images/20220828/e36d8398633040d6ac708cb858cc5391.png
[82f51b718adda1cff0d4e38a596bd985.png]: /images/20220828/a7f3ba1e4f164fd1aa8e3b54c45c392a.png
[930b67e45fa1017f565133adb1880e32.png]: /images/20220828/1906229de5f147f397573948a716ce6e.png
[7cac8485876f51e7fb4a77ce2bda2fcd.png]: /images/20220828/2a295ca076ac41788eb2ae49b9e6b973.png
[be2e0ac30b831a5db31d78e90381d5e1.png]: /images/20220828/1ec6654d79d144d7bda25f0e507b755f.png
[67b4759d67d11e3719e2031dca93618c.png]: /images/20220828/81ff822134364a0e94e5dfa11b344421.png
[http_central.maven.org_maven2_mysql_mysql-connector-java_5.1.40]: https://links.jianshu.com/go?to=http%3A%2F%2Fcentral.maven.org%2Fmaven2%2Fmysql%2Fmysql-connector-java%2F5.1.40%2F
[b578e6f30908b38415304d330639cfcc.png]: /images/20220828/28fa4a5f892f492c9726354811282eca.png
[668a1ea2622c207ccfae90437eceac7a.png]: /images/20220828/1e228c729e65455c9bff383a1aa45939.png
[http_hdc-data1_8080]: https://links.jianshu.com/go?to=http%3A%2F%2Fhdc-data1%3A8080%2F
[9327d14203d6eb11571628b1d160b6d5.png]: /images/20220828/efbfc2d3391940b2a8f601c9e05016d2.png
[7b2076a89fc6dcf58cd4a3fc5109a466.png]: /images/20220828/66f28c0b8e7e4a24aaa6a04ffb1ed6fe.png
[88c39db8d18461271482516837b4dc8a.png]: /images/20220828/c9bce69a5312445ba6f0eb1a404b3476.png
[4dc5c4150642911a2898ebe30b87a7ac.png]: /images/20220828/8fa674f709a24e8d91f76779c0430c50.png
[b2c2774ddf02ddf9ef703e57829411dd.png]: /images/20220828/693d426c90444542a174bfd66f96bb62.png
[813e5680f9a65fffc7db8b66f179f3c7.png]: /images/20220828/a824f34513e940de836ccddfe59cc078.png
[e7488407d8c74ac7fe70a1745437da3d.png]: /images/20220828/d2a446ea5a0a4cb38b2cdb178fb5e39e.png
[6ccdfe3ced0cb451a6c580138a50e2ba.png]: /images/20220828/80372b30980e41bda580dc7643c99b96.png
[2824fe2af652e056f383d919a3173e41.png]: /images/20220828/715ca72e61974ac5886ff4316af1604c.png
[e445cf7a52cd1b30af628959c4f97020.png]: /images/20220828/2a3435c8874149f5bf1ac5860b97a79d.png
[ff05d37d39da7cbd619e2b20d953833f.png]: /images/20220828/f5ba5165563d42e5a5655fb54bb12acc.png
[9f1baa3d0e0a5f7bb03788e9bf03d500.png]: /images/20220828/062efcc2db40404cbbadc905cfe97653.png
[ee286c13710eaf8d46aff2589e5d0a12.png]: /images/20220828/115d9199b61d40ada74d667224d2a3b8.png
[ec96e36b1da2618f0d2edd9cd0ad1d92.png]: /images/20220828/d4328344607b4386826806a11ae9ecca.png
[888a316dddf0182347583a845b8c1a69.png]: /images/20220828/0b149aaadeb34f19ba0efb4686067d6c.png
[c6f931d3842ed23229afe190d445ee8b.png]: /images/20220828/db3e8abc65f748a0bb2e4aed8b798865.png
[704beb8794523908e79c2c2005d34f52.png]: /images/20220828/e76429fa3820439a985816bdb6f33bdf.png
[173ba904280b14a6f0964144b01a19e2.png]: /images/20220828/3e0f471d75724c41956543364c251515.png
[f6536b31e886a0b6693ce9bfd2263c8b.png]: /images/20220828/71014abebeed4cc7983e7ab288646678.png
[665b1983d95b8f1778c483cd3f943112.png]: /images/20220828/d02cac0f911b488a92bebc61e37c5576.png
[ecd5c020e23639b65bcc8082f8dcf7ef.png]: /images/20220828/9d4859be0a40453c90e11f3178308e56.png
[c2c9008f5d63d896432e2406aae243df.png]: /images/20220828/df136b9e9845480ebeda81f7342f566f.png
[bf39836941604cdcbf45ab0e4db2177b.png]: /images/20220828/f3c577651f524319b9f30ce6075afa3e.png
[2392697e0a727fe0a0789b4cf2180170.png]: /images/20220828/b0ad2e3593f34fb2acf2f6736da0be38.png
[http_www.louisvv.com_archives_1490.html]: https://links.jianshu.com/go?to=http%3A%2F%2Fwww.louisvv.com%2Farchives%2F1490.html
[dc6cf26bc64f5c8ce09b0e2f47528ca6.png]: /images/20220828/c199138745384b11905a18ddcbcf2d7a.png
[d96942728e9a2ce3ef87b3734a9e893d.png]: /images/20220828/a140a96a40bd49d49250617133d483aa.png
[9f126f43d29e516a3bc3b76870d30a0b.png]: /images/20220828/d06c8d57023b48d09553464ae8140171.png
[ed9a0e4f7e6a5c7ec0158faac29806ac.png]: /images/20220828/25b777824e2a4b349b4163bdd04295d4.png
[51cfe85ecbeae4c8a37f4d4e7ff556aa.png]: /images/20220828/95649209e1334a049c4275cfba6e81b0.png
[988a5a730ead53709dbace9dda038459.png]: /images/20220828/b531b30e1f2742ecaf9ad4909edc131a.png
[68d5662624b395dd04a84ca1ec96fec8.png]: /images/20220828/55535915717c402698a7fc7e7e8295b1.png
[9de130eab30de61047c758235b887bac.png]: /images/20220828/ee4f6121da504d31af52bba1da42c924.png
[8766c90a59fde85c9a6c58c951480abb.png]: /images/20220828/21aa71d3e88448c395ac8f16cd910178.png
[0541d26a85dcaac4ec1d7ccd5adc8e8b.png]: /images/20220828/3151d269625c4a8e91f9293875067f9b.png
[f1754d0b7452e40f0d324a92b3139315.png]: /images/20220828/1bf5cf4794714e9eb7ff780a039bdee8.png
[db6eefbedca3ac11ad96b50007de5c70.png]: /images/20220828/08eadd3e986f427f8102c12a2c7e1d73.png
[7754dfa727cb206bb247f16058203737.png]: /images/20220828/69d5df62c6694aa6b018dd4e40482895.png
[682cef2f9828ac472edf73da95f716bd.png]: /images/20220828/17617766d52245cea23bee59c5836e52.png
[07012c88eb224701718459bd4f582427.png]: /images/20220828/2c9355e199d2489a835677e5f1deaf13.png
[f75c8e873cd63f0e2609a52b17bd2b74.png]: /images/20220828/e5bb7a018aaf4065b3cf176cb0396d1d.png
[fd523b59bc83b21550d2bfc138919f11.png]: /images/20220828/4feca67cf0124148b11bd622fc84f95c.png