Hadoop——HDFS HA部署

ゝ一世哀愁。 2022-05-26 01:12 283阅读 0赞

本节目标：

掌握HDFS HA的部署

本节参考[官方文档][Link 1]进行部署。

所需软件：JDK1.8、Hadoop2.7.6、zookeeper3.4.11

节点部署情况：

<table> 
 <tbody> 
 <tr> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 <td style="vertical-align:top;"> NN </td> 
 <td style="vertical-align:top;"> DN </td> 
 <td style="vertical-align:top;"> JN </td> 
 <td style="vertical-align:top;"> ZK </td> 
 <td style="vertical-align:top;"> ZKFC </td> 
 </tr> 
 <tr> 
 <td style="vertical-align:top;"> master </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 </tr> 
 <tr> 
 <td style="vertical-align:top;"> slave1 </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 </tr> 
 <tr> 
 <td style="vertical-align:top;"> slave2 </td> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 </tr> 
 <tr> 
 <td style="vertical-align:top;"> slave3 </td> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> l&nbsp; &nbsp; </td> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 <td style="vertical-align:top;"> &nbsp; </td> 
 </tr> 
 </tbody> 
</table>

基于[完全分布式搭建][Link 2]，完成HDFS HA部署。

1、[安装zookeeper ][zookeeper] /usr/local/src/zookeeper  
环境变量：  
vi ~/.bashrc  
export ZOOKEEPER\_HOME=/usr/local/src/zookeeper-3.4.11  
export PATH=$PATH:$ZOOKEEPER\_HOME/bin  
source ~/.bashrc  
进入安装目录下的conf目录下，

mv zoo\_sample.cfg zoo.cfg

添加一下配置信息：

dataDir=/usr/local/src/zookeeper-3.4.11/data  
server.1=192.168.188.10:2888:3888  
server.2=192.168.188.11:2888:3888  
server.3=192.168.188.12:2888:3888  
/usr/local/src/zookeeper/data下创建myid  ---》内容为每个机器对应的服务号1、2、3  
同步zookeeper到slave1,slave2，修改环境变量，修改对应的myid

2、删除各节点上的masters。masters文件指定SNN，而HDFS HA不需要SNN。

3、删除各节点上的hdfs内容保存目录

4、hdfs-site.xml修改

<property> 
  <name>dfs.nameservices</name>  --->命名服务 
  <value>HDFSHA</value> 
</property> 
<property> 
  <name>dfs.ha.namenodes.HDFSHA</name>    --->命名服务下指定两台nn 
  <value>nn1,nn2</value> 
</property> 
<property> 
  <name>dfs.namenode.rpc-address.HDFSHA.nn1</name>   --》指定rpc 
  <value>master:8020</value> 
</property> 
<property> 
  <name>dfs.namenode.rpc-address.HDFSHA.nn2</name> 
  <value>slave1:8020</value> 
</property> 
<property> 
  <name>dfs.namenode.http-address.HDFSHA.nn1</name>   --》指定http 
  <value>master:50070</value> 
</property> 
<property> 
  <name>dfs.namenode.http-address.HDFSHA.nn2</name> 
  <value>slave1:50070</value> 
</property> 
<property>    -->指定JN 
  <name>dfs.namenode.shared.edits.dir</name> 
  <value>qjournal://slave1:8485;slave2:8485;slave3:8485/HDFSHA</value> 
</property> 
<property>  --》自动故障迁移 
  <name>dfs.client.failover.proxy.provider.HDFSHA</name> 
  <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> 
</property> 
<property>    ---》指定公钥文件 
  <name>dfs.ha.fencing.methods</name> 
  <value>sshfence</value> 
</property> 
<property> 
  <name>dfs.ha.fencing.ssh.private-key-files</name> 
  <value>/root/.ssh/id\_rsa</value> 
</property> 
<property>   --》指定JN中edit存放目录 
  <name>dfs.journalnode.edits.dir</name> 
  <value>/usr/local/src/journal/data</value> 
</property> 
 <property>  -->是否启用自动故障迁移 
   <name>dfs.ha.automatic-failover.enabled</name> 
   <value>true</value>

</property>

5、core-site.sh 
<property> 
  <name>fs.defaultFS</name> 
  <value>hdfs://HDFSHA</value> 
</property> 
 <property>  --》指定zk集群 
   <name>ha.zookeeper.quorum</name> 
   <value>master:2181,slave1:2181,slave2:2181</value> 
 </property>

<property> 
        <name>hadoop.tmp.dir</name>      
        <value>/usr/localc/hadoop-2.7.6/tmp<value> 
    </property>

6、同步配置文件

scp ./\*.xml slave1:/usr/local/src/hadoop-2.7.6/etc/hadoop

scp ./\*.xml slave2:/usr/local/src/hadoop-2.7.6/etc/hadoop

scp ./\*.xml slave3:/usr/local/src/hadoop-2.7.6/etc/hadoop

7、启动JN  
slave1,slave2,slave3上执行 --> hadoop-daemon.sh start journalnode

6、在master上进行格式化，并启动NN

hadoop namenode -format

hadoop-daemon.sh start namenode  
8、在另一个NN上执行： hdfs namenode –bootstrapStandby，同步元数据信息。

9、启动zk

zkServer.sh start

检查状态：zkServer.sh status

10、在其中一个NN上格式化zk：  hdfs zkfc -formatZK

11、启动hdfs

start-dfs.sh

12、HA检查

![20180423083426312][]

![20180423083409637][]

关闭其中一个活跃的NN，hadoop-daemon.sh stop namenode；看Standby状态的NN是否自动切换为Active。

注意：先启动zookeeper，然后启动hdfs。

[Link 1]: http://hadoop.apache.org/docs/r2.7.6/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html
[Link 2]: https://blog.csdn.net/ym01213/article/details/80032432
[zookeeper]: http://zookeeper.apache.org/doc/r3.4.11/zookeeperStarted.html
[20180423083426312]: /images/20220526/b8bd5aacdfe0427bbdb76c1477c1196f.png
[20180423083409637]: /images/20220526/86e32166d7f84bf8bee3278fe2e61a2c.png