【Apache Flume】Apache Flume高级组件

妖狐艹你老母 2023-10-05 13:58 34阅读 0赞

#### Apache Flume ####

*   *  拦截器
     *   *  Example1
         *  Example2
     *  通道选择器
     *   *  replicating
         *  Multiplexing
     *  Sink Processors
     *   *  Load balancing Sink Processor
         *  Failover Sink Processor

### 拦截器 ###

作用于Source组件，对Source封装的Event数据进行`拦截`或者是`装饰`,Flume内建了许多拦截器：

*  [Timestamp Interceptor][]：装饰类型，负责在Event Header添加时间信息
 *  [Host Interceptor][]：装饰类型，负责在Event Header添加主机信息
 *  [Static Interceptor][]：装饰类型,负责在Event Header添加自定义key和value。
 *  [Remove Header Interceptor][]:装饰类型,负责删除Event Header中指定的 key
 *  [UUID Interceptor][]：装饰类型，负责在Event Header添加uuid的随机的唯一字符串
 *  [Search and Replace Interceptor][]：装饰类型，负责搜索EventBody的内容，并且将匹配的内容进行替换
 *  [Regex Filtering Interceptor][]：拦截类型，将满足正则表达式的内容进行过滤或者匹配。
 *  [Regex Extractor Interceptor][]：装饰类型，负责搜索EventBody的内容，并且将匹配的内容添加到Event Header里面。

#### Example1 ####

测试装饰拦截器

# 声明基本组件 Source Channel Sink  example11.properties
    a1.sources

[Timestamp Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#timestamp-interceptor
[Host Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#host-interceptor
[Static Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#static-interceptor
[Remove Header Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#remove-header-interceptor
[UUID Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#uuid-interceptor
[Search and Replace Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#search-and-replace-interceptor
[Regex Filtering Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#regex-filtering-interceptor
[Regex Extractor Interceptor]: http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#regex-extractor-interceptor