大数据实时处理框架之Flink win10快速部署

女爷i 2022-11-21 01:30 716阅读 0赞

**目录**

一、依赖环境安装

1、jdk和scala依赖下载

2、flink1.9.0下载

二、环境变量配置

1、FLINK\_HOME变量配置

2、将FLINK\_HOME\\bin变量加入path

三、启动运行flink

1、启动flink

2、访问WEB UI

3、cmd命令行执行flink demo

4、WEB页面上传jar包执行

四、监控 flink JVM使用资源

1、运行 jdk 的 jvisualvm 客户端

五、一键启动脚本

1、win10搭建kafka环境

2、win10搭建flink环境

3、一键启动脚本

--------------------

## 一、依赖环境安装 ##

### 1、jdk和scala依赖下载 ###

jdk1.8：[https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html][https_www.oracle.com_java_technologies_javase_javase-jdk8-downloads.html]

scala12.11：[https://www.scala-lang.org/download/][https_www.scala-lang.org_download]

### 2、flink1.9.0下载 ###

flink官方中文文档：[https://flink.apache.org/zh/downloads.html][https_flink.apache.org_zh_downloads.html]

flink1.9.0安装包：[https://archive.apache.org/dist/flink/flink-1.9.0/][https_archive.apache.org_dist_flink_flink-1.9.0]

## 二、环境变量配置 ##

### 1、FLINK\_HOME变量配置 ###

![20201029181948681.png][]

### 2、将FLINK\_HOME\\bin变量加入path ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70][]

## 三、启动运行flink ##

### 1、启动flink ###

打开cmd，执行

start-cluster

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 1][]

### 2、访问WEB UI ###

[http://localhost:8081][http_localhost_8081]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 2][]

### 3、cmd命令行执行flink demo ###

cmd 进入 fink 的 bin 目录

cd %FLINK_HOME%\bin && D:

执行官方 demo wordcount.jar 统计 readme.txt 文件的单词个数，将结果输出到 wordcount.txt 文件

flink run %FLINK_HOME%\examples\batch\WordCount.jar -input %FLINK_HOME%\README.txt --output %FLINK_HOME%\wordcount.txt

![20201029195838852.png][]

### 4、WEB页面上传jar包执行 ###

点击 Add New 上传本地 jar 包

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 3][]

上传之后点击 submit 提交

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 4][]

## 四、监控 flink JVM使用资源 ##

### 1、运行 jdk 的 jvisualvm 客户端 ###

进入jdk bin目录下执行 jvisualvm.exe

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 5][]

选择 org.apache.flink.runtime.taskexecutor.TaskManagerRunner，点击监视查看TaskManager进程的资源利用

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 6][]

其他运行在JVM的任务也可以在 Java VisualVM 客户端下查看

## 五、一键启动脚本 ##

### 1、win10搭建kafka环境 ###

[https://blog.csdn.net/qq262593421/article/details/109327420][https_blog.csdn.net_qq262593421_article_details_109327420]

### 2、win10搭建flink环境 ###

[https://blog.csdn.net/qq262593421/article/details/109365896][https_blog.csdn.net_qq262593421_article_details_109365896]

### 3、一键启动脚本 ###

start-all.cmd

start %KAFKA_HOME%\bin\windows\zookeeper-server-start %KAFKA_HOME%\config\zookeeper.properties
    start %KAFKA_HOME%\bin\windows\kafka-server-start %KAFKA_HOME%\config\server.properties
    start %FLINK_HOME%\bin\start-cluster

执行此脚本可以多窗口同时启动 kafka flink

Flink本地模式Linux安装：[https://ci.apache.org/projects/flink/flink-docs-release-1.13/zh//docs/try-flink/local\_installation/][https_ci.apache.org_projects_flink_flink-docs-release-1.13_zh_docs_try-flink_local_installation]

[https_www.oracle.com_java_technologies_javase_javase-jdk8-downloads.html]: https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html
[https_www.scala-lang.org_download]: https://www.scala-lang.org/download/
[https_flink.apache.org_zh_downloads.html]: https://flink.apache.org/zh/downloads.html
[https_archive.apache.org_dist_flink_flink-1.9.0]: https://archive.apache.org/dist/flink/flink-1.9.0/
[20201029181948681.png]: /images/20221120/48dd76749e6c4cba92018ca38fd66047.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70]: /images/20221120/4effad767ab049179a0f658067f1c22d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20201029182729261.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx,size_16,color_FFFFFF,t_70
[http_localhost_8081]: http://localhost:8081
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20201029182949583.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx,size_16,color_FFFFFF,t_70
[20201029195838852.png]: https://img-blog.csdnimg.cn/20201029195838852.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20201029193309143.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20201029193145309.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/2020102919135914.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20201029191224441.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxMjYyNTkzNDIx,size_16,color_FFFFFF,t_70
[https_blog.csdn.net_qq262593421_article_details_109327420]: https://blog.csdn.net/qq262593421/article/details/109327420
[https_blog.csdn.net_qq262593421_article_details_109365896]: https://blog.csdn.net/qq262593421/article/details/109365896
[https_ci.apache.org_projects_flink_flink-docs-release-1.13_zh_docs_try-flink_local_installation]: https://ci.apache.org/projects/flink/flink-docs-release-1.13/zh//docs/try-flink/local_installation/