Hadoop单机安装

骑猪看日落 2023-02-22 12:22 1阅读 0赞

[参考文章链接][Link 1]  
core-site.xml：

<configuration>
    	 <property>
                    <name>fs.defaultFS</name>
                    <value>hdfs://localhost:9000</value>
            </property>
            <property>
                    <name>hadoop.tmp.dir</name>
                    <value>/home/Hadoop/data/hadoopdata</value>
            </property>
    </configuration>

hdfs-site.xml：

<configuration>
    	<property>
                    <name>dfs.namenode.name.dir</name>
                    <value>/home/Hadoop/data/hadoopdata/name</value>
                    <description>为了保证元数据的安全一般配置多个不同目录</description>
            </property>
    
            <property>
                    <name>dfs.datanode.data.dir</name>
                    <value>/home/Hadoop/data/hadoopdata/data</value>
                    <description>datanode 的数据存储目录</description>
            </property>
    
            <property>
                    <name>dfs.replication</name>
                    <value>1</value>
                    <description>HDFS 的数据块的副本存储个数, 默认是3</description>
            </property>
    
            <property>
                    <name>dfs.secondary.http.address</name>
                    <value>localhost:50090</value>
                    <description>secondarynamenode 运行节点的信息，和 namenode 不同节点</description>
            </property>
    </configuration>

mapred-site.xml：

<configuration>
    	<property>
    		<name>mapreduce.framework.name</name>
    		<value>yarn</value>
        </property>
    </configuration>

yarn-site.xml：

<configuration>
    
    
    	 <property>
                    <name>yarn.resourcemanager.hostname</name>
                    <value>localhost</value>
            </property>
            
            <property>
                    <name>yarn.nodemanager.aux-services</name>
                    <value>mapreduce_shuffle</value>
                    <description>YARN 集群为 MapReduce 程序提供的 shuffle 服务</description>
            </property>
    </configuration>

报错和解决办法：  
WARN hdfs . DFSUtil: Namenode for null remains unresolved for ID null. Check your hdfs-site.xmL file to ensure namenodes are configured properly.  
原因： 未配置SSH免密:  
解决办法指令：

ssh-keygen
    ssh-copy-id localhost

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxNTg5Mjkz_size_16_color_FFFFFF_t_70]  
故以上配置localhost。

注意Tips：用ftp连接别用记事本打开修改，否则编码为gbk，导致下图错误：  
Caused by: org . apache . xerces . impl. io. Mal formedByteSequenceException: Invalid byte 1 of l-byte UTF-8 sequence .  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxNTg5Mjkz_size_16_color_FFFFFF_t_70 1]  
启动HDFS和YARN：

start-dfs.sh
    start-yarn.sh

验证是否启动成功：

JPS

![JPS][]  
访问(我虚拟机ip为:10.9.119.128)：  
http://10.9.119.128:50070/explorer.html\#/

[Link 1]: https://www.cnblogs.com/qingyunzong/p/8496127.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxNTg5Mjkz_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200703155004747.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxNTg5Mjkz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxNTg5Mjkz_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200703153913811.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxNTg5Mjkz,size_16,color_FFFFFF,t_70
[JPS]: https://img-blog.csdnimg.cn/20200703155420333.png