Spark伪分布式安装（不依赖hadoop）

我会带着你远行 2022-08-27 13:52 7阅读 0赞

上传包：

百度分享安装包地址：http://pan.baidu.com/s/1dD4BcGT [点击打开链接][Link 1]

解压 并重命名：

![SouthEast][]

进入spark100目录：

修改配置：

Cd conf

配置单击节点，故只需要写本机的hostname就可以了。

![SouthEast 1][]

重命名spark-env.sh.template 为 spark-env.sh

增加配置项：

export SPARK_MASTER_IP=icity1
    export SPARK_MASTER_PORT=7077
    export SPARK_WORKER_CORES=1
    export SPARK_WORKER_INSTANCES=1
    export SPARK_WORKER_MEMORY=1g

如图所示：

![SouthEast 2][]

![SouthEast 3][]

启动：

出错：

![SouthEast 4][]

。bash\_profile中有配置java\_home：

![SouthEast 5][]

感觉其没有自动引用到，故

最后通过在spark\_env.sh中增加

![SouthEast 6][]

解决了错误。

重新启动：

![SouthEast 7][]

监控页面效果：

![SouthEast 8][]

进入spark-shell：

![SouthEast 9][]

![SouthEast 10][]

选用了外部文件：

![SouthEast 11][]

区别于hdfs的读取方式：

![SouthEast 12][]

![SouthEast 13][]

![SouthEast 14][]

退出spark-shell：exit

![SouthEast 15][]

[Link 1]: http://pan.baidu.com/s/1dD4BcGT
[SouthEast]: /images/20220823/340215a1afbd42578d4359d648633703.png
[SouthEast 1]: /images/20220823/cee952eef0474d2bb90cf79b0a950e53.png
[SouthEast 2]: /images/20220823/20cc4a9481a448119f5491d4e9b2633b.png
[SouthEast 3]: /images/20220823/1ebc641f015b42ee88fdfa9738ade727.png
[SouthEast 4]: /images/20220823/05288da0f02d4b1198ce5fdd7baf91c3.png
[SouthEast 5]: /images/20220823/02da71b8b4c340c58f87c48ab11d7e7b.png
[SouthEast 6]: /images/20220823/7c4aba5e20ca46638c7261c8756a8e63.png
[SouthEast 7]: /images/20220823/a13bf3b47c434011b12107c7f49fdcc1.png
[SouthEast 8]: /images/20220823/92f19077c5dc42daac491d411c470a02.png
[SouthEast 9]: /images/20220823/eda762fdd59743cab967083e3c949cd4.png
[SouthEast 10]: /images/20220823/15f78eb45f75485dbe6f7745074b6e37.png
[SouthEast 11]: /images/20220823/b0e70ffc633e475b83685b01073ac9d4.png
[SouthEast 12]: /images/20220823/fccdb1860cbd44c7b933d76ea11f0b3b.png
[SouthEast 13]: /images/20220823/9bf7e88d983b4ef788065a14993d9a8d.png
[SouthEast 14]: /images/20220823/14c44cd092d34b7fabc825393ff0ad4f.png
[SouthEast 15]: /images/20220823/9bbf5e0c0c804f3c9e9fc46bbcb2b7e0.png