Yarn命令使用及wordcount解析

素颜马尾好姑娘i 2022-05-23 00:35 287阅读 0赞

**前言：**

> 前面几篇博客主要介绍了MapReduce与Yarn的架构设计及简单工作流程，本篇文章将以wordcount程序为例，简单介绍下Yarn的使用。

##### 1.wordcount示例运行 #####

[root@hadoop000 ~]# su - hadoop
    [hadoop@hadoop000 ~]$ jps
    9201 SecondaryNameNode
    9425 ResourceManager
    13875 Jps
    9540 NodeManager
    8852 NameNode
    8973 DataNode
    # 创建wordcount目录
    [hadoop@hadoop000 ~]$ hdfs dfs -mkdir -p /wordcount/input
    [hadoop@hadoop000 ~]$ vi test.log
    jepson ruoze
    hero yimi xjp
    123
    a b a
    [hadoop@hadoop000 ~]$ hdfs dfs -put test.log /wordcount/input
    [hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/input           
    Found 1 items
    -rw-r--r--   1 hadoop supergroup         37 2018-05-29 20:38 /wordcount/input/test.log
    # 执行wordcount示例jar包
    [hadoop@hadoop000 ~]$ yarn jar \
    > /opt/software/hadoop-2.8.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar \
    > wordcount \
    > /wordcount/input \
    > /wordcount/output
    18/05/29 20:40:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    18/05/29 20:40:59 INFO input.FileInputFormat: Total input files to process : 1
    18/05/29 20:41:00 INFO mapreduce.JobSubmitter: number of splits:1
    18/05/29 20:41:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1526991305992_0001
    18/05/29 20:41:01 INFO impl.YarnClientImpl: Submitted application application_1526991305992_0001
    18/05/29 20:41:01 INFO mapreduce.Job: The url to track the job: http://hadoop000:8088/proxy/application_1526991305992_0001/
    18/05/29 20:41:01 INFO mapreduce.Job: Running job: job_1526991305992_0001
    18/05/29 20:41:14 INFO mapreduce.Job: Job job_1526991305992_0001 running in uber mode : false
    18/05/29 20:41:14 INFO mapreduce.Job:  map 0% reduce 0%
    18/05/29 20:41:23 INFO mapreduce.Job:  map 100% reduce 0%
    18/05/29 20:41:29 INFO mapreduce.Job:  map 100% reduce 100%
    18/05/29 20:41:30 INFO mapreduce.Job: Job job_1526991305992_0001 completed successfully
    18/05/29 20:41:30 INFO mapreduce.Job: Counters: 49
    # 查看结果
    [hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/output
    Found 2 items
    -rw-r--r--   1 hadoop supergroup          0 2018-05-29 20:41 /wordcount/output/_SUCCESS
    -rw-r--r--   1 hadoop supergroup         51 2018-05-29 20:41 /wordcount/output/part-r-00000
    [hadoop@hadoop000 ~]$ hdfs dfs -cat /wordcount/output/part-r-00000
    123     1
    a       2
    b       1
    hero    1
    jepson  1
    ruoze   1
    xjp     1
    yimi    1

登录网页查看相关信息：[http://192.168.6.217:8088/cluster][http_192.168.6.217_8088_cluster]  
![网页显示.png][.png]

##### 2.Yarn常用命令总结 #####

yarn jar <jar>              --run a jar file
    yarn application -list      --列出在跑的job
    yarn application -kill application_1526991305992_0001（job的id） --杀掉在跑的job

##### 3.wordcount流程详解 #####

![官方案例.jpg][.jpg]  
参考：[https://blog.csdn.net/yczws1/article/details/21794873][https_blog.csdn.net_yczws1_article_details_21794873]

[http_192.168.6.217_8088_cluster]: http://192.168.6.217:8088/cluster
[.png]: /images/20220523/acff827271cf47278739fe837bbb4c34.png
[.jpg]: /images/20220523/f8a840b42aa84cb7b33d1d97421afd03.png
[https_blog.csdn.net_yczws1_article_details_21794873]: https://blog.csdn.net/yczws1/article/details/21794873