阿里云ECS服务器安装单机hadoop

ゝ一世哀愁。 2022-03-30 07:29 363阅读 0赞

云服务器centos版本：CentOS 7.2

参照：[https://blog.csdn.net/r562253897/article/details/81983609][https_blog.csdn.net_r562253897_article_details_81983609]

遇到的问题：

1、执行 ./sbin/start-dfs.sh后遇到如下错误：

but there is no HDFS\_NAMENODE\_USER defined, Aborting operation

but there is no HDFS\_DATENODE\_USER defined, Aborting operation

问题原因：sh脚本中未配置用户变量

解决方式：  
sbin/[start-dfs.sh][]  、 sbin/[stop-dfs.sh][] 中文件头添加如下配置 ：  
HDFS\_DATANODE\_USER=root  
HADOOP\_SECURE\_DN\_USER=hdfs  
HDFS\_NAMENODE\_USER=root  
HDFS\_SECONDARYNAMENODE\_USER=root

sbin/[start-yarn.sh][] 、sbin/[stop-yarn.sh][] 中文件头添加如下配置：  
YARN\_RESOURCEMANAGER\_USER=root  
HADOOP\_SECURE\_DN\_USER=yarn  
YARN\_NODEMANAGER\_USER=root

2、问题1解决后继续执行sbin/start-dfs.sh后提示：

Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

继续按照文档中的配置(需要切换为hadoop用户来配置)配置完成后再次执行sbin/start-dfs.sh即可,若还有权限不足的错误参考[https://www.cnblogs.com/xubing-613/p/6844564.html][https_www.cnblogs.com_xubing-613_p_6844564.html]；

3、引用文档中配置mapred-site.xml中mapreduce.framework.name的值应为yarn，没有最后的>(一个大坑)，否则会导致在hive中执行MapReduce查询时报错，hadoop各配置文件参数配置参见：[https://www.jianshu.com/p/594aa47493fc][https_www.jianshu.com_p_594aa47493fc]

[https_blog.csdn.net_r562253897_article_details_81983609]: https://blog.csdn.net/r562253897/article/details/81983609
[start-dfs.sh]: https://link.jianshu.com?t=http%3A%2F%2Fstart-dfs.sh
[stop-dfs.sh]: https://link.jianshu.com?t=http%3A%2F%2Fstop-dfs.sh
[start-yarn.sh]: https://link.jianshu.com?t=http%3A%2F%2Fstart-yarn.sh
[stop-yarn.sh]: https://link.jianshu.com?t=http%3A%2F%2Fstop-yarn.sh
[https_www.cnblogs.com_xubing-613_p_6844564.html]: https://www.cnblogs.com/xubing-613/p/6844564.html
[https_www.jianshu.com_p_594aa47493fc]: https://www.jianshu.com/p/594aa47493fc