hadoop1-入门案例Grep|wordCount

傷城~ 2022-04-10 11:13 221阅读 0赞

### download and install ###

官网下载 [https://hadoop.apache.org/releases.html][https_hadoop.apache.org_releases.html]  
版本 2.8.5,解压  
`cd /opt`  
`tar -zxvf hadoop-2.8.5.tar.gz`  
目录结构  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2NzYyNjc3_size_16_color_FFFFFF_t_70]

--------------------

## Grep案例 ##

### 2.测试 ###

创建文件夹,将配置文件复制到input

[root@localhost hadoop-2.8.5]# mkdir input
    [root@localhost hadoop-2.8.5]# cp etc/hadoop/*.xml input/

进入hadoop,启动第一次测试  
output文件不能创建,会报错

[root@localhost hadoop-2.8.5]# hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.5.jar grep input/ output 'dfs[a-z.]+'

生成output文件,里面有两个文件  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2NzYyNjc3_size_16_color_FFFFFF_t_70 1]

##### 2.1统计结果 #####

SUCCESS文件0KB  
part-r-00000文件为统计结果  
![在这里插入图片描述][20190104112839760.png]

--------------------

## WordCount案例 ##

创建文件夹及文件

[root@localhost hadoop-2.8.5]# mkdir wcinput
    [root@localhost hadoop-2.8.5]# cd wcinput/
    [root@localhost wcinput]# touch wc.input
    [root@localhost wcinput]# vim wc.input

输入以下内容

hadoop yarn
    hadoop mapreduce
    iloveyou
    iloveyou

执行hadoop程序

[root@localhost hadoop-2.8.5]# hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.5.jar wordcount wcinput/ wcouput

执行结果  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2NzYyNjc3_size_16_color_FFFFFF_t_70 2]  
查看`part-r-00000`  
![在这里插入图片描述][20190104114634334.png]

[https_hadoop.apache.org_releases.html]: https://hadoop.apache.org/releases.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2NzYyNjc3_size_16_color_FFFFFF_t_70]: /images/20220331/7a35f22ea1ad490abe70cc3af14229a4.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2NzYyNjc3_size_16_color_FFFFFF_t_70 1]: /images/20220331/d441be0c12da4b3298eb61d5cb3bb500.png
[20190104112839760.png]: /images/20220331/710194ac5e7c41adaa9b82ecd4c7605e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM2NzYyNjc3_size_16_color_FFFFFF_t_70 2]: /images/20220331/34b19468dc7f443b8f7412bce682e78b.png
[20190104114634334.png]: /images/20220331/b90d1588317c414f8de5bee35e91fbe0.png