【大数据之Flume基本安装】

Love The Way You Lie 2022-01-15 08:33 259阅读 0赞

[为什么80%的码农都做不了架构师？>>> ][80_] ![hot3.png][]

**一、下载/解压缩**

\[root@hadoop0 opt\]\# wget http://mirror.bit.edu.cn/apache/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz

\--1999-05-08 19:02:36-- http://mirror.bit.edu.cn/apache/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz

Resolving mirror.bit.edu.cn... 114.247.56.117

Connecting to mirror.bit.edu.cn|114.247.56.117|:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: 52550402 (50M) \[application/octet-stream\]

Saving to: apache-flume-1.6.0-bin.tar.gz

100%\[===========================>\] 52,550,402 621K/s in 1m 41s

1999-05-08 19:04:17 (507 KB/s) - apache-flume-1.6.0-bin.tar.gz saved \[52550402/52550402\]

\[root@hadoop0 opt\]\# tar -zxvf apache-flume-1.6.0-bin.tar.gz

\[root@hadoop0 opt\]\# cd apache-flume-1.6.0-bin

\[root@hadoop0 apache-flume-1.6.0-bin\]\# ./bin/flume-ng help

Usage: ./bin/flume-ng <command> \[options\]...

commands:

help display this help text

agent run a Flume agent

avro-client run an avro Flume client

version show Flume version info

global options:

\--conf,-c <conf> use configs in <conf> directory

\--classpath,-C <cp> append to the classpath

\--dryrun,-d do not actually start Flume, just print the command

\--plugins-path <dirs> colon-separated list of plugins.d directories. See the

plugins.d section in the user guide for more details.

Default: $FLUME\_HOME/plugins.d

\-Dproperty=value sets a Java system property value

\-Xproperty=value sets a Java -X option

agent options:

\--name,-n <name> the name of this agent (required)

\--conf-file,-f <file> specify a config file (required if -z missing)

\--zkConnString,-z <str> specify the ZooKeeper connection to use (required if -f missing)

\--zkBasePath,-p <path> specify the base path in ZooKeeper for agent configs

\--no-reload-conf do not reload config file if changed

\--help,-h display help text

avro-client options:

\--rpcProps,-P <file> RPC client properties file with server connection params

\--host,-H <host> hostname to which events will be sent

\--port,-p <port> port of the avro source

\--dirname <dir> directory to stream to avro source

\--filename,-F <file> text file to stream to avro source (default: std input)

\--headerFile,-R <file> File containing event headers as key/value pairs on each new line

\--help,-h display help text

Either --rpcProps or both --host and --port must be specified.

Note that if <conf> directory is specified, then it is always included first

in the classpath.

**二、增加配置文件**

\[root@hadoop0 apache-flume-1.6.0-bin\]\# vi conf/hw.conf

agent.sources=s1

agent.channels=c1

agent.sinks=k1

agent.sources.s1.type=netcat

agent.sources.s1.channels=c1

agent.sources.s1.bind=192.168.1.111

agent.sources.s1.port=12345

agent.channels.c1.type=memory

agent.sinks.k1.type=logger

agent.sinks.k1.channel=c1

**三、启动、验证**

\[root@hadoop0 apache-flume-1.6.0-bin\]

\# ./bin/flume-ng agent -n agent -c conf -f conf/hw.conf -Dflume.root.logger=INFO,console

\[root@hadoop0 ~\]\# netstat -tnpl

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name

tcp 0 0 0.0.0.0:9447 0.0.0.0:\* LISTEN 1349/rpc.statd

tcp 0 0 0.0.0.0:111 0.0.0.0:\* LISTEN 1304/rpcbind

tcp 0 0 0.0.0.0:22 0.0.0.0:\* LISTEN 1538/sshd

tcp 0 0 127.0.0.1:631 0.0.0.0:\* LISTEN 1382/cupsd

tcp 0 0 127.0.0.1:25 0.0.0.0:\* LISTEN 1618/master

tcp 0 0 :::111 :::\* LISTEN 1304/rpcbind

tcp 0 0 :::13683 :::\* LISTEN 1349/rpc.statd

tcp 0 0 :::22 :::\* LISTEN 1538/sshd

tcp 0 0 ::1:631 :::\* LISTEN 1382/cupsd

tcp 0 0 ::1:25 :::\* LISTEN 1618/master

\[root@hadoop0 ~\]\# nc -l 12345

nc: Address already in use

\[root@hadoop0 ~\]\# nc localhost 12345

\[root@hadoop0 ~\]\# nc 192.168.1.111 12345

sdfsdfdsf

sdfsdfsdf

gaojingsong

whomai

**四、验证结果**

**![01005642_oNKa.png][]**

**![01005642_PncN.png][]**

**![01005642_2y4L.png][]**

![01005642_rcIY.png][]

转载于:https://my.oschina.net/boltwu/blog/703834

[80_]: https://my.oschina.net/u/2663968/blog/3051541
[hot3.png]: /images/20220115/d1d2efdb4a4749be84ad424ccbce8eaf.png
[01005642_oNKa.png]: /images/20220115/a0f159703acb4d21b06849091938b280.png
[01005642_PncN.png]: /images/20220115/dfadf3d89d6e4bfeb45ae7894ecca541.png
[01005642_2y4L.png]: /images/20220115/53c103a51cbd4ed38962282f2e9865fb.png
[01005642_rcIY.png]: /images/20220115/12edd8e71b0b40f18cb91914e643a7f7.png