CDH 6.1安装过程

心已赠人 2023-02-18 03:24 2阅读 0赞

**目录**

一、JDK

二、配置镜像文件

三、上传数据库上传 mysql-5.6.39-linux-glibc2.12-x86\_64.tar 到 /usr/local/

四、安装Mysql

五、安装所有节点：yum clean allyum makecache

五、运行

--------------------

前文：

CDH的好处太多不一一列举了。

#    一、JDK #

cd /opt/software  
rm -rf  jdk1.8.0\_181  
rm -rf /opt/software/jdk-8u181-linux-x64.tar.gz

所有节点：  
yum -y install rsync  
yum -y install rpcbind

mkdir /usr/java/

主节点：  
上传 oracle-j2sdk1.8-1.8.0+update141-1.x86\_64 到/usr/java/

cd /usr/java/  
rpm -ivh oracle-j2sdk1.8-1.8.0+update141-1.x86\_64.rpm  
mv jdk1.8.0\_141-cloudera/ jdk1.8/

scp -r /usr/java/jdk1.8 root@bt-02:/usr/java/  
scp -r /usr/java/jdk1.8 root@bt-03:/usr/java/

所有节点：

vim /etc/profile

JAVA\_HOME=/usr/java/jdk1.8  
JRE\_HOME=/usr/java/jdk1.8/jre  
CLASS\_PATH=.:$JAVA\_HOME/lib/dt.jar:$JAVA\_HOME/lib/tools.jar:$JRE\_HOME/lib  
PATH=$PATH:$JAVA\_HOME/bin:$JRE\_HOME/bin  
export JAVA\_HOME JRE\_HOME CLASS\_PATH PATH

source /etc/profile

java -version

#    二、配置镜像文件 #

主节点：  
mkdir /var/www/html/cm6.1.0  
mkdir /var/www/html/cdh6.1.0

cp /opt/CDH610/\* /var/www/html/cdh6.1.0/  
cp /opt/CM610/\* /var/www/html/cm6.1.0/

createrepo /var/www/html/cm6.1.0/  
createrepo /var/www/html/cdh6.1.0/

cd /etc/yum.repos.d/   
      
vim cm.repo  
\[cmrepo\]  
name = cm\_repo  
baseurl =http://bt-01/cm6.1.0/  
enable = true  
gpgcheck = false

vim cdh.repo  
\[cdhrepo\]  
name = cdh\_repo  
baseurl =http://bt-01/cdh6.1.0/  
enable = true  
gpgcheck = false

rsync /etc/yum.repos.d/\* root@bt-02:/etc/yum.repos.d/  
rsync /etc/yum.repos.d/\* root@bt-03:/etc/yum.repos.d/

ll /etc/yum.repos.d/

主：资源包只要放在主节点

切换镜像：  
所有节点：  
cd /etc/yum.repos.d/  
rename .repo .repo.bak \*    
mv cm.repo.bak  cm.repo  
mv cdh.repo.bak  cdh.repo

# 三、上传数据库   上传 mysql-5.6.39-linux-glibc2.12-x86\_64.tar 到 /usr/local/ #

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
        保        存  
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

# 四、安装Mysql #

所有节点：  
rpm -qa | grep -i mysql  
rpm -e --nodeps MySQL-python-1.2.5-1.el7.x86\_64  (不该执行这一步)

rpm -qa | grep -i mariadb  
rpm -e --nodeps mariadb-libs-5.5.60-1.el7\_5.x86\_64

find / -name mysql  
rm -rf /etc/selinux/targeted/active/modules/100/mysql  
rm -rf /usr/lib64/mysql

主节点：  
useradd mysql  
passwd mysql

cd /usr/local  
tar -xzvf mysql-5.6.39-linux-glibc2.12-x86\_64.tar.gz  
mv mysql-5.6.39-linux-glibc2.12-x86\_64 mysql  
chown -R mysql:mysql /usr/local/mysql

/usr/local/mysql/scripts/mysql\_install\_db --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data  
cp my-default.cnf  /etc/my.cnf   
cp mysql.server /etc/init.d/mysql

vi /etc/my.cnf  
basedir=/usr/local/mysql  
datadir=/usr/local/mysql/data

vi /etc/init.d/mysql  
basedir=/usr/local/mysql  
datadir=/usr/local/mysql/data

chkconfig --add mysql  
chkconfig mysql on

service mysql start

/usr/local/mysql/bin/mysqladmin -u root password 123456

vi /etc/profile

JAVA\_HOME=/usr/java/jdk1.8  
JRE\_HOME=/usr/java/jdk1.8/jre  
MYSQL\_HOME=/usr/local/mysql  
CLASS\_PATH=.:$JAVA\_HOME/lib/dt.jar:$JAVA\_HOME/lib/tools.jar:$JRE\_HOME/lib  
PATH=$PATH:$JAVA\_HOME/bin:$JRE\_HOME/bin:$MYSQL\_HOME/bin

export JAVA\_HOME JRE\_HOME CLASS\_PATH PATH MYSQL\_HOME

source /etc/profile

mysql -u root -p123456  
grant all privileges on \*.\* to root@"%" identified by "123456";  
flush privileges;

mkdir /usr/share/java  
把 mysql-connector-java.jar 放入/usr/share/java中

这一步权限非常重要，最后一次还要操作一次

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8\_general\_ci;  
GRANT ALL ON scm.\* TO 'scm'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON amon.\* TO 'amon'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON rman.\* TO 'rman'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON hue.\* TO 'hue'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON metastore.\* TO 'hive'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON sentry.\* TO 'sentry'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON nav.\* TO 'nav'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON navms.\* TO 'navms'@'%' IDENTIFIED BY '123456';  
GRANT ALL ON oozie.\* TO 'oozie'@'%' IDENTIFIED BY '123456';  
flush privileges;

主节点：  
上传 MySQL-shared-5.6.47-1.el7.x86\_64.rpm 到 /opt/software  
rpm -ivh MySQL-shared-5.6.47-1.el7.x86\_64.rpm

\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#\#

#    五、安装   所有节点：   yum clean all   yum makecache #

重装：(误删)  
cd /etc/yum.repos.d/  
mv CentOS-Base.repo.bak CentOS-Base.repo  
yum -y install MySQL-python  
mv CentOS-Base.repo CentOS-Base.repo.bak

注意：作为依赖被安装:  
  mariadb-libs.x86\_64 1:5.5.64-1.el7

主节点：  
yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

其他节点：  
yum install cloudera-manager-daemons cloudera-manager-agent

/opt/cloudera/cm/schema/scm\_prepare\_database.sh mysql scm scm  
cp -r /var/www/html/cdh6.1.0/\* /opt/cloudera/parcel-repo/

# 五、运行 #

主节点启动：  
systemctl start cloudera-scm-server

\#所有节点启动：  
\#systemctl start cloudera-scm-agent

netstat -ntlp |grep 7180  
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

数据库用户权限访问主机确认

进入http://bt-01:7180

bt-01,bt-02,bt-03  
http://bt-01/cdh6.1.0/  
http://bt-01/cm6.1.0/

六、关闭集群  
systemctl stop cloudera-scm-server  
systemctl stop cloudera-scm-agent

相关参考：https://blog.csdn.net/qq\_39680564/article/details/100923644\#comments

# 六、补充 #

## 6.1 界面 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjY4NzA3NA_size_16_color_FFFFFF_t_70][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjY4NzA3NA_size_16_color_FFFFFF_t_70 1][]

## 6.2 发行版本 ##

<table> 
 <tbody> 
  <tr> 
   <th style="width:152px;">组件</th> 
   <th style="width:226px;">版本</th> 
   <th style="width:296px;">发行版</th> 
   <th style="width:226px;">CDH 版本</th> 
  </tr> 
  <tr> 
   <td style="width:152px;">Supervisord</td> 
   <td style="width:226px;">3.0</td> 
   <td style="width:296px;">不可用</td> 
   <td style="width:226px;">不适用</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Cloudera Manager Agent</td> 
   <td style="width:226px;">6.1.0</td> 
   <td style="width:296px;">769885.el7</td> 
   <td style="width:226px;">不适用</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Cloudera Manager Management Daemon</td> 
   <td style="width:226px;">6.1.0</td> 
   <td style="width:296px;">769885.el7</td> 
   <td style="width:226px;">不适用</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Flume NG</td> 
   <td style="width:226px;">1.8.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Hadoop</td> 
   <td style="width:226px;">3.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">HDFS</td> 
   <td style="width:226px;">3.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">HttpFS</td> 
   <td style="width:226px;">3.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">hadoop-kms</td> 
   <td style="width:226px;">3.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">MapReduce 2</td> 
   <td style="width:226px;">3.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">YARN</td> 
   <td style="width:226px;">3.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">HBase</td> 
   <td style="width:226px;">2.1.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Lily HBase Indexer</td> 
   <td style="width:226px;">1.5+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Hive</td> 
   <td style="width:226px;">2.1.1+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">HCatalog</td> 
   <td style="width:226px;">2.1.1+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Hue</td> 
   <td style="width:226px;">3.9.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Impala</td> 
   <td style="width:226px;">3.1.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Java 8</td> 
   <td style="width:226px;">java version "1.8.0_162" Java(TM) SE Runtime Environment (build 1.8.0_162-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)</td> 
   <td style="width:296px;">不可用</td> 
   <td style="width:226px;">不适用</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Kafka</td> 
   <td style="width:226px;">2.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Kite（仅限 CDH 5 ）</td> 
   <td style="width:226px;">1.0.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">kudu</td> 
   <td style="width:226px;">1.8.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Oozie</td> 
   <td style="width:226px;">5.0.0-beta1+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Parquet</td> 
   <td style="width:226px;">1.9.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Pig</td> 
   <td style="width:226px;">0.17.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">sentry</td> 
   <td style="width:226px;">2.1.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Solr</td> 
   <td style="width:226px;">7.4.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">spark</td> 
   <td style="width:226px;">2.4.0+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">Sqoop</td> 
   <td style="width:226px;">1.4.7+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">ZooKeeper</td> 
   <td style="width:226px;">3.4.5+cdh6.1.0</td> 
   <td style="width:296px;">770702</td> 
   <td style="width:226px;">CDH 6.1.0</td> 
  </tr> 
  <tr> 
   <td style="width:152px;">&nbsp;</td> 
   <td style="width:226px;">&nbsp;</td> 
   <td style="width:296px;">&nbsp;</td> 
   <td style="width:226px;">&nbsp;</td> 
  </tr> 
 </tbody> 
</table>

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjY4NzA3NA_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/2020122611263941.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjY4NzA3NA==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjY4NzA3NA_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20201226112619157.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjY4NzA3NA==,size_16,color_FFFFFF,t_70