CDH安装：tar包+parcel, 自动化脚本

以你之姓@ 2021-11-12 00:26 573阅读 0赞

### 文章目录 ###

*  1, cm-server端
 *  2, cm-agent端
 *  3, 防火墙开放端口
 *  4, 辅助脚本
 *   *  1, 服务安装顺序
     *  2, ssh-tool.sh
     *  3, xscp.sh

安装指南: [https://docs.cloudera.com/documentation/enterprise/5-12-x/topics/installation.html][https_docs.cloudera.com_documentation_enterprise_5-12-x_topics_installation.html]

*  1, 自定义本地parcel仓库路径： 先启动cloudera-scm-server，再登录7180 web页面，点击Parcels图标–>Config --> Local Parcel Repository Path, 保存重启cloudera-scm-server

[root@cdh-node1 mysql]# /usr/local/mysql/bin/mysql -uroot -p123 -e "select * from scm.CONFIGS" |grep parcel_repo_path 
    4	NULL	parcel_repo_path	/cdh5.12-el6/cdh/	NULL	NULL	2	1	NULL	NONE	NULL

*  2, 自定义agent安装parcel的路径: 解压后agent启动文件处/etc/cloudera-scm-agent/config.ini 配置parcel\_dir=/cloudera/parcels
 *  3, 以service的方式启动server, agent

#官网给的是这样的操作：
    cp $CMF_DEFAULTS/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server
    service cloudera-scm-server start
    
    #####但是会报错，需要修改文件中的一个变量：第73行
    [root@cdh-node1 mysql]# grep CMF_DEF /etc/init.d/cdh-agent
    #CMF_DEFAULTS=$(readlink -e $(dirname ${BASH_SOURCE-$0})/../default)
    CMF_DEFAULTS=$(readlink -e /cdh5.12-el6/cdh/cm-5.12.2/etc/default)

### 文章目录 ###

*  1, cm-server端
 *  2, cm-agent端
 *  3, 防火墙开放端口
 *  4, 辅助脚本
 *   *  1, 服务安装顺序
     *  2, ssh-tool.sh
     *  3, xscp.sh

基本组织架构

<table> 
 <thead> 
  <tr> 
   <th>角色</th> 
   <th align="right">启动的服务</th> 
   <th>安装方法</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>cm-server</td> 
   <td align="right">mysql,cloudera-scm-server, cloudera-scm-agent</td> 
   <td>./server.sh</td> 
  </tr> 
  <tr> 
   <td>cm-agent_1</td> 
   <td align="right">cloudera-scm-agent</td> 
   <td>./agent.sh</td> 
  </tr> 
  <tr> 
   <td>cm-agent_2</td> 
   <td align="right">cloudera-scm-agent</td> 
   <td>./agent.sh</td> 
  </tr> 
  <tr> 
   <td>cm-agent…n</td> 
   <td align="right">cloudera-scm-agent</td> 
   <td>./agent.sh</td> 
  </tr> 
 </tbody> 
</table>

脚本安装：自定义的安装包cdh5.12-cent6.tar.gz， 解压后包含的 所用到的文件及目录如下：

├── agent.sh
    ├── server.sh
    ├── cdh
    │   ├── CDH-5.12.0-1.cdh5.12.0.p0.29-el6.parcel
    │   ├── CDH-5.12.0-1.cdh5.12.0.p0.29-el6.parcel.sha
    │   ├── cloudera-manager-el6-cm5.12.2_x86_64.tar.gz
    │   └── manifest.json
    ├── mysql-jdk
    │   ├── jdk-8u211-linux-x64.rpm
    │   ├── MySQL-5.5.62-1.el6.x86_64.rpm-bundle.tar
    │   ├── MySQL-client-5.5.62-1.el6.x86_64.rpm
    │   ├── mysql-connector-java-5.1.27.jar
    │   ├── MySQL-devel-5.5.62-1.el6.x86_64.rpm
    │   ├── MySQL-embedded-5.5.62-1.el6.x86_64.rpm
    │   ├── MySQL-server-5.5.62-1.el6.x86_64.rpm
    │   ├── MySQL-shared-5.5.62-1.el6.x86_64.rpm
    │   ├── MySQL-shared-compat-5.5.62-1.el6.x86_64.rpm
    │   └── MySQL-test-5.5.62-1.el6.x86_64.rpm
    ├── rpm-deps
    │   ├── ntp-4.2.6p5-15.el6.centos.x86_64.rpm
    │   ├── ntpdate-4.2.6p5-15.el6.centos.x86_64.rpm
    │   ├── perl-5.10.1-144.el6.x86_64.rpm
    │   ├── perl-CGI-3.51-144.el6.x86_64.rpm
    │   ├── perl-devel-5.10.1-144.el6.x86_64.rpm
    │   ├── perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64.rpm
    │   ├── perl-ExtUtils-ParseXS-2.2003.0-144.el6.x86_64.rpm
    │   ├── perl-libs-5.10.1-144.el6.x86_64.rpm
    │   ├── perl-Module-Pluggable-3.90-144.el6.x86_64.rpm
    │   ├── perl-Pod-Escapes-1.04-144.el6.x86_64.rpm
    │   ├── perl-Pod-Simple-3.13-144.el6.x86_64.rpm
    │   ├── perl-Test-Harness-3.17-144.el6.x86_64.rpm
    │   ├── perl-Test-Simple-0.92-144.el6.x86_64.rpm
    │   └── perl-version-0.77-144.el6.x86_64.rpm
    └── sh-tools
        ├── ntp-master.sh
        ├── ssh-tool.sh
        └── xscp.sh

*  mysql相关包下载： [ https://dev.mysql.com/downloads/mysql/5.5.html\#downloads][https_dev.mysql.com_downloads_mysql_5.5.html_downloads]，
 *  cdh相关包下载：[ http://archive.cloudera.com/cm5/cm/5/][http_archive.cloudera.com_cm5_cm_5]， [ http://archive.cloudera.com/cdh5/parcels][http_archive.cloudera.com_cdh5_parcels]
 *  cdh-rpms相关包：  
    cloudera对配置文件分发是要用到perl等依赖包，所以需要提前下载好以防有的环境下缺少依赖而导致安装失败----具体下载方法：  
    a, 配置/etc/yum.conf参数keepcache=1  
    b, 下载rpm版的cm安装包,[http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64][http_archive.cloudera.com_cm5_redhat_6_x86_64_cm_5.12.1_RPMS_x86_64]  
    c, 在有网的环境下执行预安装： yum -y install \*.rpm  
    d, 收集安装所需的依赖包： find /var/cache/ |grep rpm > rpm.txt

# 1, cm-server端 #

server.sh 如下：

############### 提前准备 ################
    #a,,修改主机名
    #host_name=hadoop01
    #hostname $host_name
    #sed -i s@localhost.localdomain@$host_name@g /etc/sysconfig/network
    #cat /etc/sysconfig/network
    #sleep 3
    
    #b,,配置host主机名映射/etc/hosts
    #echo '1.1.1.1 hadoop01' >>/etc/hosts
    #scp /etc/hosts hadoop01:/etc
    
    #c,,配置cm-server ssh 免密码登录所有主机
    #######################################
    set -x
    hostname_check(){ 
    	# 主机名
    	hostname=`hostname`
    
    	# 判断主机名是否合法：不能有_
    	echo $hostname |egrep  '_'
    	res=$(echo $?)
    	[ $res -eq 0 ] && echo 'hostname is invaid !!! ' && exit 1
    
    	# 查看hosts文件
    	res=`grep $hostname /etc/hosts|wc -l`
    	if [ $res -ne 0 ] ;then
    	      echo ">>>>>> hostname is ok ! "
    	else  
    	      echo ">>>>>> error, hostname is not configured !!! , please edit /etc/hosts "; exit 1
    	fi
    }
    hostname_check
    
    #删除旧的包,方便反复重装
    [ -d /opt/cloudera ] && /opt/cm-5.*/etc/init.d/cloudera-scm-agent stop && \
    /opt/cm-5.*/etc/init.d/cloudera-scm-server stop
    [ -d /opt/cm*/run/cloudera-scm-agent/process ] && fuser -mk /opt/cm*/run/cloudera-scm-agent/process
    umount /opt/cm*/run/cloudera-scm-agent/process
    rm -rf /opt/cm*
    rm -rf /opt/cloudera*
    service mysqld stop
    
    #1,关闭防火墙
    service iptables stop
    chkconfig iptables off
    setenforce 0
    sed -i s/=enforcing/=disabled/ /etc/selinux/config 
    
    #1.2安装依赖包
    rpm -Uvh rpm-deps/*.rpm --nodeps --force
    #配置ntpd
    sh-tools/ntp-master.sh
    
    #2, 解压安装包
    cd cdh
    tar -zxf cloudera-manager-*.tar.gz -C /opt/
    cp CDH* /opt/cloudera/parcel-repo/
    cp manifest.json /opt/cloudera/parcel-repo/
    
    #3, 安装jdk,mysql连接
    cd ../mysql-jdk/
    rpm -qa |grep -i openjdk |xargs -n 1 rpm -e --nodeps
    rpm -ivh jdk-8u211-linux-x64.rpm 
    echo 'export JAVA_HOME=/usr/java/default'>>/etc/profile
    
    rpm -qa |grep -i mysql |xargs -n 1 rpm -e --nodeps 
    whereis mysql |xargs -n 1  rm -rf 
    rm -rf /var/lib/mysql*
    rpm -Uvh mysql*.rpm --nodeps
    \cp my.cnf /etc/
    
    #4,创建mysql数据库，配置连接权限
    if [ ! -d /usr/share/java ]; then
        mkdir /usr/share/java -p
    fi
    \cp *.jar  /opt/cm*/share/cmf/lib/
    \cp *.jar  /usr/share/java/
    
    
    ##e, 修改root密码
    #passwd=$(cat /tmp/mysql.pwd |grep password |awk -F "root@localhost: " '{print $2}')
    #mysql --connect-expired-password -uroot -p$passwd -e " set password for root@localhost = password('123456')" 
    mysql_root_pwd=test@123
    cat  > ~/.my.cnf <<EOF [client] user=root password=$mysql_root_pwd EOF
    
    chkconfig mysql on 
    service mysqld start
    service mysqld stop
    killall mysqld
    sed  -i '/mysql/s@/bin/false@/bin/bash@' /etc/passwd
    
    cat >/tmp/mysqlini.txt <<EOF uninstall plugin validate_password; ALTER USER 'root'@'localhost' IDENTIFIED BY '$mysql_root_pwd'; EOF
    
    su mysql -c "mysqld --init-file=/tmp/mysqlini.txt &" &
    sleep 5
    #mysqladmin -u root password '123456'
    mysql -e "drop database if exists hive ; create database hive default charset utf8";
    mysql -e "drop database if exists oozie; create database oozie default charset utf8";
    mysql -e "drop database if exists hue; create database hue default charset utf8";
    
    mysql -e "grant all privileges on *.* to 'scm'@'%' identified by 'scm@123' with grant option; flush privileges;";
    mysql -e "grant all privileges on *.* to 'scm'@"`hostname`" identified by 'scm@123' with grant option; flush privileges;";
    mysql -e "grant all privileges on *.* to 'root'@'%' identified by '$mysql_root_pwd' with grant option; flush privileges;";
    mysql -e "grant all privileges on *.* to 'root'@"`hostname`" identified by '$mysql_root_pwd' with grant option; flush privileges;";
    
    mysql -e "drop database if exists scm";
    /opt/cm*/share/cmf/schema/scm_prepare_database.sh mysql -uroot -p$mysql_root_pwd scm scm scm@123
    #或者直接编辑/opt/cm*/etc/cloudera-scm-server/db.properties, 配置mysql连接的参数
    #mysql -uroot -p123456 -e "drop database if exists scm;create database scm default charset utf8";
    mysql -e "install plugin validate_password SONAME 'validate_password.so';"
    
    #5, 消除警告
     #=====临时修改swappiness
     sysctl -w vm.swappiness=1
     #====永久修改swappiness
     echo "vm.swappiness=1" >> /etc/sysctl.conf
     
     #刷新交换空间配置
     swapoff -a
     swapon -a
     
     #页面压缩
     chmod +x  /etc/rc.d/rc.local
     grep /sys/kernel/mm/transparent_hugepage /etc/rc.d/rc.local &>/dev/null
     if [ $? -ne 0 ] ;then
       echo  'echo never > /sys/kernel/mm/transparent_hugepage/defrag ' >>/etc/rc.d/rc.local
       echo  'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >>/etc/rc.d/rc.local
     fi
    
     echo never >  /sys/kernel/mm/transparent_hugepage/enabled
     echo never >  /sys/kernel/mm/transparent_hugepage/defrag
    	
    #设置开机启动
    cat >/etc/init.d/cdh <<EOF #!/bin/bash # chkconfig: 2345 90 90 # description: cdh.service .... agent=/opt/cm-5.*/etc/init.d/cloudera-scm-agent server=/opt/cm-5.*/etc/init.d/cloudera-scm-server cmd=\$1 case \$1 in start) \$server start; \$agent start ;; stop) \$agent stop; \$server stop ;; status) \$server status; \$agent status;; restart) \$server restart; \$agent restart;; * ) echo "Usage : \$server start|restart|stop|status" esac EOF
    
    chmod +x /etc/init.d/cdh
    chkconfig cdh on
    
    #6，启动server, agent
    #创建scm用户
    cd /opt/cm*
    useradd --system --home=`pwd`/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
    
    #更改/opt目录为此用户所属
    chown -R cloudera-scm:cloudera-scm /opt/{ cloudera,cm*}
    [ ! -d /var/lib/cloudera-scm-server ] && mkdir /var/lib/cloudera-scm-server
    chown -R cloudera-scm:cloudera-scm /var/lib/cloudera-scm-server
    
    #启动ntpd
    service ntpd status
    [ $? -ne 0 ] && k
    ###启动server
    /opt/cm*/etc/init.d/cloudera-scm-server start
    
    
    while :
    do
      sleep 2
      echo "==================>>>> sleep 2...........waiting for server listen on 7180<<<<<<<"
      tail /opt/cm*/log/cloudera-scm-server/cloudera-scm-server.log 
    
      res=`grep "Started SelectChannelConnector@0.0.0.0:7180" /opt/cm*/log/cloudera-scm-server/cloudera-scm-server.log |wc -l`
      [[ $res -eq 1 ]] && break
    done
    
    #内存限制
    limits(){       
    		grep hdfs /etc/security/limits.conf &>/dev/null
    		[ $? -ne 0 ] && echo "session required pam_limits.so" >> /etc/pam.d/login \
      		&& cat >>/etc/security/limits.conf <<EOF @hdfs soft nproc 65536 @hdfs hard nproc nulimited @hdfs hard memlock unlimited @hdfs hard nofile 1000000 @hdfs hard stack nulimited EOF
    }
    
    #启动agent
    start_agent(){ 
       echo "........start agent......"
       [ ! -d /opt/cloudera/parcels ]  && mkdir  /opt/cloudera/parcels -p  
       [ ! -d /opt/cloudera/csd ]   && mkdir  /opt/cloudera/csd -p  
       chown -R cloudera-scm:cloudera-scm /opt/
       ps -ef |grep supervisor |awk '{print $2}' |xargs -n 1 kill -9
       /opt/cm*/etc/init.d/cloudera-scm-agent start
       sleep 3
       tail -F /opt/cm*/log/cloudera-scm-agent/cloudera-scm-agent.log 
    }
    
    limits
    start_agent

# 2, cm-agent端 #

agent.sh 如下：

############### 提前准备 ################
    #a,,修改主机名
    #hostname $host_name
    #sed -i s@localhost.localdomain@$host_name@g /etc/sysconfig/network
    #cat /etc/sysconfig/network
    
    #bb, host主机名映射
    #cat /etc/hosts 
    #hostname
    ######################################
    
    set -x
    read -p '请输入cm-server 的ip或主机名：' master_name
    echo "cloudera manager的 server是 ==>$master_name"
    ping $master_name -c 3
    res=`echo $?`
    if [ $res -eq 0 ] ;then
       echo ">>>>>> hostname is ok ! "
    else 
       echo ">>>>>> error, hostname is not configured !!! , please edit /etc/hosts "; exit 1
    fi
    
    #1,关闭防火墙
    service iptables stop
    chkconfig iptables off
    setenforce 0
    sed -i s/=enforcing/=disabled/ /etc/selinux/config 
    
    #删除旧的包,方便反复重装
    [ -d /opt/cloudera ] && /opt/cm-5.*/etc/init.d/cloudera-scm-agent stop && \
    /opt/cm-5.*/etc/init.d/cloudera-scm-server stop
    [ -d /opt/cm*/run/cloudera-scm-agent/process ] && fuser -mk /opt/cm*/run/cloudera-scm-agent/process
    umount /opt/cm*/run/cloudera-scm-agent/process
    rm -rf /opt/cm*
    rm -rf /opt/cloudera*
    
    #2, 解压安装包
    cd cdh
    tar -zxf cloudera-manager-el6-cm5.12.0_x86_64.tar.gz -C /opt/
    cd ../mysql-jdk/
    
    #3, 安装jdk,mysql连接
    rpm -ivh jdk-8u211-linux-x64.rpm 
    echo 'export JAVA_HOME=/usr/java/default'>>/etc/profile
    
    [ ! -d /usr/share/java ] &&  mkdir /usr/share/java -p
    cp mysql-connector-*.jar /usr/share/java/mysql-connector-java.jar
    cp mysql-connector-*.jar  /opt/cm*/share/cmf/lib
    #安装oozie: 把mysql连接jar包,拷贝到/opt/cloudera/parcels/lib/oozie/lib/
    
    #4, 消除警告
    #=====临时修改swappiness
    sysctl -w vm.swappiness=1
    #====永久修改swappiness
    echo "vm.swappiness=1" >> /etc/sysctl.conf
     
    #刷新交换空间配置
    swapoff -a
    swapon -a
     
    #页面压缩
    chmod +x  /etc/rc.d/rc.local
    echo  'echo never > /sys/kernel/mm/transparent_hugepage/defrag ' >>/etc/rc.d/rc.local
    echo  'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >>/etc/rc.d/rc.local
    echo never >  /sys/kernel/mm/transparent_hugepage/enabled
    echo never >  /sys/kernel/mm/transparent_hugepage/defrag
    
     #内存限制
    grep hdfs /etc/security/limits.conf &>/dev/null
    [ $? -ne 0 ] && cat >>/etc/security/limits.conf <<EOF hdfs soft nproc 65536 hdfs hard nproc nulimited hdfs soft memlock 2048 hdfs hard memlock unlimited hdfs soft nofile 100000 hdfs hard nofile 1000000 hdfs soft stack 5024 hdfs hard stack nulimited EOF
    
    #5, 启动agent
    sed -i s@server_host=localhost@server_host=$master_name@g    /opt/cm*/etc/cloudera-scm-agent/config.ini
    head /opt/cm-*/etc/cloudera-scm-agent/config.ini 
    cd /opt/cm*
    useradd --system --home=`pwd`/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
    
    cd ~
    chown -R cloudera-scm:cloudera-scm /opt/
    ps -ef |grep supervisor |awk '{print $2}' |xargs -n 1 kill -9
    find / |grep cm_guid |xargs -n 1 rm -rf
    
    #删除hbase,hive,zookeeper,hadoop,yarn
    rm -rf /var/lib/zookeeper /var/lib/hadoop* /var/lib/hive /var/lib/hbase
    rm -rf /dfs /yarn
    
    /opt/cm*/etc/init.d/cloudera-scm-agent start
    sleep 3
    tail -F /opt/cm*/log/cloudera-scm-agent/cloudera-scm-agent.log

# 3, 防火墙开放端口 #

如果服务器需要开启防火墙，则需要开放一下端口：

#一次最多15个端口
    iptables -I INPUT -p tcp -m multiport --dports 8485,8480,8020,8022,50070,9090,9095,60000,60010,8888,11000,8983,8984,8032,8030  -j ACCEPT
    
    iptables -I INPUT -p tcp -m multiport --dports 8031,8033,8088,60020,60030,50020,50010,50075,10000,10002,10101,8091,9995,9994 -j ACCEPT 
    
    iptables -I INPUT -p tcp -m multiport --dport 9083,9092,9393,9093,24042,10020,19888,10033,8042,8041,8040,2181,3181,4181,9010 -j ACCEPT
    
    iptables -I INPUT -p tcp -m multiport --dport 7184,7185,8084,8086,9997,18088,18488,7182,9000,3306,22,8087,9999,9998,9996 -j ACCEPT

# 4, 辅助脚本 #

## 1, 服务安装顺序 ##

为了更好的利用Cloudera Manager的服务自动配置能力，可以按照以下顺序添加服务

#服务安装顺序
    1, zookeeper
    2, hdfs
    3, yarn
    4, spark
    
    5, hive
    6, hbase
    
    7, oozie
    8, hue

## 2, ssh-tool.sh ##

配置免密码登陆其他主机

#免密码登录
    ssh-keygen -t rsa -f ~/.ssh/id_rsa -P ""
    
    #同步密钥
    for((host=1;host<=4;host++))
    do
       echo "---> ssh-copy-id $host"
       ssh-copy-id $host
    done

## 3, xscp.sh ##

配置文件同步

#scp 文件到-->其他节点
    file=$1
    filename=`basename $file`
    dirname=`dirname $file`
    echo "file=> "$filename  ", dir=> "$dirname
    
    #scp 循环
    for((host=1;host<=4;host++))
    do
       echo "----------- scp $file $host:$dirname----------------"
       #scp $file $host:$dirname
    done

[https_docs.cloudera.com_documentation_enterprise_5-12-x_topics_installation.html]: https://docs.cloudera.com/documentation/enterprise/5-12-x/topics/installation.html
[https_dev.mysql.com_downloads_mysql_5.5.html_downloads]: https://dev.mysql.com/downloads/mysql/5.5.html#downloads
[http_archive.cloudera.com_cm5_cm_5]: http://archive.cloudera.com/cm5/cm/5/
[http_archive.cloudera.com_cdh5_parcels]: http://archive.cloudera.com/cdh5/parcels
[http_archive.cloudera.com_cm5_redhat_6_x86_64_cm_5.12.1_RPMS_x86_64]: http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64