Hadoop伪分布式搭建步骤

太过爱你忘了你带给我的痛 2024-04-07 10:11 27阅读 0赞

**Hadoop\*\*\*\*伪分布式搭建步骤**

一、环境准备

进入超级用户：

![20565e1e372844ddaabc0d901fd26f30.png][]

二、安装ssh：

![51122bdf3f4f4779ab26b0f4547bea8d.png][]

生成秘钥文件，无密码访问：

![d16b198609924ef8becebc6aa568a925.png][]

进入无密码访问目录：/root/.ssh/

![9b900cc79f69413c8d8144936f6621e2.png][]

生成authorized\_keys秘钥文件

![7163a193dac948ebada18803c8712299.png][]

修改/etc/hosts文件内容，将IP地址与计算机名关联起来。

vi /etc/hosts

![4ccc491895e746b8baf0602da11fa16a.png][]

测试无密码登录是否成功：

![77d021b1e32b47909790284fa4b3f1b2.png][]

登录成功后，请用exit命令退出远程登录。

二、安装Java语言（jdk）

1.  在/home/czc/下创建文件夹wei。

![3215c71014084d70ae9789898504ab60.png][]

2、解压jdk软件包到wei文件夹下。

![4f1f750ee98948a2a5be32afb158b183.png][]

1.  到解压的文件夹里去查看

![a28d096a45fe41f0a12e7382f2d6a9cd.png][]

4、为了后面书写简单，改一下文件夹的名字。

![b89953f5ab1043bf91c74c4020952f24.png][]

1.  编辑Java变量环境文件：/etc/profile:

![54d8f01997054726b1a017ce5c05dcd7.png][]

最后添加两行：

![8aca3a269f144985ad7507d952fe1bbb.png][]

6、使刚刚编辑的环境生效：

![4d4bc987c4c6459da23315b4b88c9d40.png][]

7、测试是否安装成功：

![12a85cb41abb4e9cafbee31a1ee67d18.png][]

三、安装hadoop

将hadoop软件包解压到/home/czc/wei/下（注意路径）：

![6f9e6a77079146a79ab869e859b6cad1.png][]

进入hadoop目录，并查看核心文件夹：

![39bbdcf3a22f4729ad0394fa53fe4367.png][]

进入hadoop配置文件的目录下：

![c614c172fd84490797983e296aeb135a.png][]

查看主要配置文件：

![ddebeb773b5e49be8244237d77404f09.png][]

修改hadoop-env.sh文件内容：

vi hadoop-env.sh

![98a584ef6df448b2b8e12a86ebdf937f.png][]

修改core-site.xml内容：

![a4e7c473da0c4b52889c445170959cd8.png][]

修改hdfs-site.xml内容：

![ba88ec49b28c42f294b94eeac7f477df.png][]

复制成一个新文件：

![5d3f0b4d80ad4373a8eee2b1d89b47b1.png][]

修改该文件内容:

![879350cb2b004adbb039c328eb25fe08.png][]

修改yarn-site.xml文件内容：

![6331581ba28143fcac866160ec27099f.png][]

修改slaves文件内容

![c8749a6198204b3ba78c35a01c512e3e.png][]

配置环境变量，使hadoop命令正常执行，（修改环境变量，/etc/profile文件）

![60033ec1f65049af9ea5bef02fe2246c.png][]

![6954f90847f54506afef378bc560281c.png][]

用source /etc/profile 命令更新：

![be0e6e45e24e4f9c9f357427dc7f91b8.png][]

测试Hadoop安装是否成功：

![03d25e0e466c46f88ba118141308b48f.png][]

格式化namenode

![dcc7c1b9626d4915ad4b1b9b50926ba9.png][]

![240711e8f47642b9afe92d407f297967.png][]

配置全部完成，对hadoop结构进行启动，（可以分布启动，也可以一次启动）

![b5ac74e0851e4390bdf4f153b3f09c2c.png][]

查看进程（6个）

![396a67ffdbaa4e0bac0f36e26adb41d4.png][]

注意：如果没有6个进程，请先关闭所有进程（stop-all.sh），再到5个配置文件中去修改，再运行，直到出现6个进程。

![e4536c965dda4b10a402af4c01162598.png][]

打开浏览器，用web页来测试：

端口50070，8088

![70433f8ba99047d5be8925cff9c016a2.png][]

![f64f54aebcf34c33b4d8bb53f3fcd4a5.png][]

[20565e1e372844ddaabc0d901fd26f30.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/bdc601fe776a490e9368694c544546e9.png
[51122bdf3f4f4779ab26b0f4547bea8d.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/05df63a76efa41949d52809b3a75cc54.png
[d16b198609924ef8becebc6aa568a925.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/d770f3529ceb4828a5ef2acd93490b11.png
[9b900cc79f69413c8d8144936f6621e2.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/7ba471857a2f4e23834bff46350969eb.png
[7163a193dac948ebada18803c8712299.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/e482bbccff7747b9a81a9941537d8898.png
[4ccc491895e746b8baf0602da11fa16a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/8ef851ae86794f6e9d50aadcfc822610.png
[77d021b1e32b47909790284fa4b3f1b2.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/e0d60f87a15649ef9ca5362c59832d78.png
[3215c71014084d70ae9789898504ab60.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/db5ae4be981b419f9a316a6a8b976893.png
[4f1f750ee98948a2a5be32afb158b183.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/50096764afd440cebab851ae9d5b88cd.png
[a28d096a45fe41f0a12e7382f2d6a9cd.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/b55bd8ecce8e4ac2b051ef86f87a622e.png
[b89953f5ab1043bf91c74c4020952f24.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/951cef91af4e487baa40cf9c4ec544d6.png
[54d8f01997054726b1a017ce5c05dcd7.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/bd27672776f24e3e9a8311808974143a.png
[8aca3a269f144985ad7507d952fe1bbb.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/b264a349149d41dd9e640026d569b174.png
[4d4bc987c4c6459da23315b4b88c9d40.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/5a05db1c3e6f43ee8aa7e2b7b159eebc.png
[12a85cb41abb4e9cafbee31a1ee67d18.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/d11a6f4e61aa4889874b6ed29eee8ba9.png
[6f9e6a77079146a79ab869e859b6cad1.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/a2ecb614bd334516bbe6af34ec83b0eb.png
[39bbdcf3a22f4729ad0394fa53fe4367.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/2e2383c966f147f6a60f83cea3575746.png
[c614c172fd84490797983e296aeb135a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/adfd22bf86e94b138669efe89c802437.png
[ddebeb773b5e49be8244237d77404f09.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/1a04c325aa154e16b2045f815665d42e.png
[98a584ef6df448b2b8e12a86ebdf937f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/00040438884c45039d0fe2b3ca4f7845.png
[a4e7c473da0c4b52889c445170959cd8.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/06226f39349a48deb5561e3da188056b.png
[ba88ec49b28c42f294b94eeac7f477df.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/0ac7cf58c92a42f4a5c8c827a5430961.png
[5d3f0b4d80ad4373a8eee2b1d89b47b1.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/2f86ed5b3c78435e950dff2372bae8db.png
[879350cb2b004adbb039c328eb25fe08.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/978aab427a9f48f0a24818d9cb33c56b.png
[6331581ba28143fcac866160ec27099f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/2dc3c82be04c423e8d0bb93b17fd7de8.png
[c8749a6198204b3ba78c35a01c512e3e.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/fc2cae46b4484c939a4cd7aa3ce19e45.png
[60033ec1f65049af9ea5bef02fe2246c.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/0cfdffdc3fd44a999c6afa3935d56bb0.png
[6954f90847f54506afef378bc560281c.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/4b65387b03864462ab32b017e2a08979.png
[be0e6e45e24e4f9c9f357427dc7f91b8.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/200a893cb0e24acf98abdcad775b9aca.png
[03d25e0e466c46f88ba118141308b48f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/418f0805d1de4647886c22d57396fe6e.png
[dcc7c1b9626d4915ad4b1b9b50926ba9.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/eb2af031b5254096a4a31c89229e0eba.png
[240711e8f47642b9afe92d407f297967.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/16f4be394f2549dca78fb6a1889160f2.png
[b5ac74e0851e4390bdf4f153b3f09c2c.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/e575957984fe418787492a7189c402fc.png
[396a67ffdbaa4e0bac0f36e26adb41d4.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/506b6a4fbd1b4397b56ce602bd166c41.png
[e4536c965dda4b10a402af4c01162598.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/070299d125fb49209609dfedae99eb08.png
[70433f8ba99047d5be8925cff9c016a2.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/c4a92faf2bda424c8504552b4929e9a1.png
[f64f54aebcf34c33b4d8bb53f3fcd4a5.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/07/de514efa1f244f02b7041cf12f5714ce.png