Hadoop常用操作

ゞ浴缸里的玫瑰 2023-10-01 16:28 2阅读 0赞

## **启动，关闭** ##

D:\\hadoop-2.8.3\\hadoop-2.8.3\\sbin>start-all.cmd

D:\\hadoop-2.8.3\\hadoop-2.8.3\\sbin>stop-all.cmd

start-all.sh等价于**start-dfs.sh + start-yarn.sh**

**单进程启动。**

sbin/start-dfs.sh

\---------------

**sbin/hadoop-daemons.sh --config .. --hostname .. start namenode ...**  
 **sbin/hadoop-daemons.sh --config .. --hostname .. start datanode ...**  
 **sbin/hadoop-daemons.sh --config .. --hostname .. start sescondarynamenode ...**  
 **sbin/hadoop-daemons.sh --config .. --hostname .. start zkfc ... //**

sbin/start-yarn.sh  
\--------------  
libexec/yarn-config.sh  
sbin/yarn-daemon.sh --config $YARN\_CONF\_DIR start resourcemanager  
sbin/yarn-daemons.sh --config $YARN\_CONF\_DIR start nodemanager

### **web地址** ###

http://localhost:50070/explorer.html\#/

### **新增目录** ###

#### **创建新目录** ####

hdfs dfs -mkdir -p /cmd\_create\_dir

#### **将本地文件存储至hadoop** ####

hdfs dfs -put ./demo\_mvn.jar /test/

#### **新建空文件** ####

hdfs dfs -touchz /cmd\_create\_dir/empty.txt

#### **某个文件重命名** ####

hdfs dfs -mv /cmd\_create\_dir/empty.txt /cmd\_create\_dir/rename\_empty.txt

### **修改目录** ###

### **下载文件** ###

D:\\GITSPACE\\demo\\out\\artifacts\\demo\_mvn\_jar>hadoop dfs -get /test/test.xml C:\\Users\\chang\\Desktop\\fsdownload\\security

DEPRECATED: Use of this script to execute hdfs command is deprecated.

Instead use the hdfs command for it.

D:\\GITSPACE\\demo\\out\\artifacts\\demo\_mvn\_jar>hdfs dfs -get /test/test.xml C:\\Users\\chang\\Desktop\\fsdownload\\security

get: \`C:/Users/chang/Desktop/fsdownload/security/test.xml': File exists

D:\\GITSPACE\\demo\\out\\artifacts\\demo\_mvn\_jar>hdfs dfs -get /test/test.xml C:\\Users\\chang\\Desktop\\fsdownload\\

hdfs dfs -get /cmd\_create\_dir C:\\Users\\chang\\Desktop\\fsdownload\\

#### **将hadoop指定目录下所有内容保存为一个文件，同时down至本地      hdfs dfs –getmerge /user /home/t** ####

### **删除目录** ###

hadoop fs -rm -r -skipTrash /path\_to\_file/file\_name

D:\\GITSPACE\\demo\\out\\artifacts\\demo\_mvn\_jar>hadoop fs -rm -r -skipTrash /test/word\_output\_json

Deleted /test/word\_output\_json

hadoop fs -rm -r /test/new2

#### **将正在运行的hadoop作业kill掉       hadoop job –kill \[job-id\]** ####

### **查看目录** ###

hadoop fs -cat /test/word\_output\_json/part-r-00000

#### **查看指定目录下内容** ####

hdfs dfs -ls /

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16][]

#### **打开某个已存在文件** ####

hdfs dfs -cat /test/test.xml

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 1][]

hdfs dfs -lsr /

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 2][]

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 3][]

### **HBase常用操作** ###

D:\\hbase-2.0.2-bin\\hbase-2.0.2\\bin>start-hbase.cmd

D:\\hbase-2.0.2-bin\\hbase-2.0.2\\bin>hbase shell

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 4][]

status

create 'testtable','coln1'

list\_namespace\_tables 'default'

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 5][]

hbase(main):011:0> create\_namespace 'testnamespace'

Took 0.2850 seconds

hbase(main):012:0> create 'testnamespace:testtable', 'coln1'

Created table testnamespace:testtable

Took 0.7770 seconds

=> Hbase::Table - testnamespace:testtable

hbase(main):013:0> list

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 6][]

#### **新增数据** ####

hbase(main):023:0> put "testtable", "row1", "coln1",'zhangsan'

Took 0.1300 seconds

hbase(main):024:0> put "testtable", "row2", "coln1",'lisi'

Took 0.0020 seconds

#### **查询数据** ####

hbase(main):027:0> get "testtable", "row1"

COLUMN CELL

coln1: timestamp=1639382832908, value=lisi

1 row(s)

Took 0.0180 seconds

hbase(main):028:0> get "testtable", "coln1"

COLUMN CELL

0 row(s)

Took 0.0040 seconds

hbase(main):029:0> get "testtable", "row1", "coln1"

COLUMN CELL

coln1: timestamp=1639382832908, value=lisi

1 row(s)

Took 0.0040 seconds

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 7][]

#### **删除数据** ####

delete "testtable", "row1", "coln1"

deleteall "testtable", "row1"

hbase(main):071:0> disable "testtable"

Took 0.7690 seconds

hbase(main):072:0> drop "testtable"

Took 0.2570 seconds

hbase(main):073:0> drop "testtable1"

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 8][]

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 9][]

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 10][]

![watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 11][]

[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16]: https://img-blog.csdnimg.cn/a3227f974c374470a6edc543e2ad37a1.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 1]: https://img-blog.csdnimg.cn/158abc946b2d4ad5babe299435ff6be2.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 2]: https://img-blog.csdnimg.cn/2565da16978f43e7a74318b933df30c6.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 3]: https://img-blog.csdnimg.cn/eb6cab8316c44cb6858b847c175926f6.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 4]: https://img-blog.csdnimg.cn/ce73cce1958e478b99a43209e3ab56c2.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 5]: https://img-blog.csdnimg.cn/0870e02255fa491795b2029f970dc7c6.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 6]: https://img-blog.csdnimg.cn/9a45e87adea44604bf9fb24ad3400793.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 7]: https://img-blog.csdnimg.cn/48b13979cece4a26a1239795986394e9.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 8]: https://img-blog.csdnimg.cn/6162d8ad68f84f44a2c9918b5ac537c7.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 9]: https://img-blog.csdnimg.cn/4d10cf767c834aae87f6bf4068241fed.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 10]: https://img-blog.csdnimg.cn/35a090028ae346249eb7c22a02ba7e7f.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16
[watermark_type_d3F5LXplbmhlaQ_shadow_50_text_Q1NETiBA5bCP54Gw54GwX18_size_20_color_FFFFFF_t_70_g_se_x_16 11]: https://img-blog.csdnimg.cn/2270cdbeeb1c4823a268d73ff1320e2c.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5bCP54Gw54GwX18=,size_20,color_FFFFFF,t_70,g_se,x_16