Nifi Docker集群-两个节点 (1)

本是古典何须时尚 2022-06-08 12:12 281阅读 0赞

使用Dockerfile构建nifi基础镜像：

Dockerfile:

\# Version: 0.0.1  
FROM docker.io/centos:7  
MAINTAINER heqingyong101@163.com  
  
  
ARG JDK\_VERSION=8u66  
ARG NIFI\_VERSION=1.1.0  
ARG NIFI\_HOME=/root/data\_collect  
  
  
USER root  
  
  
RUN yum -y install net-tools  
  
  
COPY jdk-$\{JDK\_VERSION\}-linux-x64.tar.gz $\{NIFI\_HOME\}/jdk-$\{JDK\_VERSION\}-linux-x64.tar.gz  
RUN cd $\{NIFI\_HOME\} && tar -xvzf jdk-$\{JDK\_VERSION\}-linux-x64.tar.gz \\  
&& rm -rf jdk-$\{JDK\_VERSION\}-linux-x64.tar.gz  
  
  
COPY nifi-$\{NIFI\_VERSION\}-bin.tar.gz $\{NIFI\_HOME\}/nifi-$\{NIFI\_VERSION\}-bin.tar.gz  
RUN cd $\{NIFI\_HOME\} && tar -xvzf nifi-$\{NIFI\_VERSION\}-bin.tar.gz \\  
&& rm -rf nifi-$\{NIFI\_VERSION\}-bin.tar.gz  
  
  
ADD lib/ $\{NIFI\_HOME\}/nifi-$\{NIFI\_VERSION\}/lib/  
  
  
RUN cd $\{NIFI\_HOME\} && mkdir conf  
ADD conf/ $\{NIFI\_HOME\}/conf/  
ADD init.sh $\{NIFI\_HOME\}/  
  
  
  
RUN chmod u+x $\{NIFI\_HOME\}/init.sh  
RUN yum -y install openssh-server openssh-clients  
RUN ssh-keygen -A  
COPY id\_rsa.pub /root/.ssh/authorized\_keys  
  
  
EXPOSE 8080  
WORKDIR /root/data\_collect  
ENTRYPOINT \["/root/data\_collect/init.sh"\]  
  
  
初始化脚本init.sh内容如下：

\#!/bin/bash  
\#\#\# ENV \#\#\#  
\#\# USE\_EMMBEDED\_ZK, should set to true for master node, others should set to false.  
\#\# HOSTNAME, hostname of every node.  
\#\# ZK\_CONNECTION\_STR, zookeeper connection string ,such as (master.younge.club:2181,slave.younge.club:2181).  
\#\# S2S\_PORT, port of site to site protocol, default 11443.  
\#\# WEB\_PORT, port of web UI, default 8080.  
\#\# IS\_CLUSTER, cluster mode if true, or single node mode, default false.  
  
  
\#\# MAX\_MEM, max memory set for nifi, default 512.  
\#\# NIFI\_VERSION, version of nifi.  
  
  
\#\# ZK\_CLI\_PORT, port zookeeper client connect to, default 2181.  
\#\# ZK\_HOSTNAME, zookeeper's hostname, single node.  
\#\# ZK\_VOTE\_PORT, zookeeper's vote port, default 2888.  
\#\# ZK\_LEADER\_PORT, zookeeper's leader port, default 3888.  
  
  
\#NIFI\_VERSION=1.1.0  
\#MAX\_MEM=1024  
  
  
cd /root/data\_collect/nifi-$\{NIFI\_VERSION\}  
  
  
\#Nifi env  
sed -i "/^\#export JAVA\_HOME=/s:.\*:export JAVA\_HOME=/root/data\_collect/jdk1.8.0\_66/:" bin/nifi-env.sh  
  
  
\#Jdk mem  
sed -i "/^java.arg.2=-Xmx512m/s/.\*/java.arg.2=-Xmx$\{MAX\_MEM:-512\}m/" conf/bootstrap.conf  
  
  
\#Nifi conf  
sed -i "/^nifi.state.management.embedded.zookeeper.start=/s/.\*/nifi.state.management.embedded.zookeeper.start=$\{USE\_EMMBEDED\_ZK:-false\}/" conf/nifi.properties  
sed -i "/^nifi.remote.input.host=/s/.\*/nifi.remote.input.host=$\{HOSTNAME\}/" conf/nifi.properties  
sed -i "/^nifi.remote.input.socket.port=/s/.\*/nifi.remote.input.socket.port=$\{S2S\_PORT:-11443\}/" conf/nifi.properties  
sed -i "/^nifi.web.http.host=/s/.\*/nifi.web.http.host=$\{HOSTNAME\}/" conf/nifi.properties  
sed -i "/^nifi.web.http.port=/s/.\*/nifi.web.http.port=$\{WEB\_PORT:-8080\}/" conf/nifi.properties  
  
  
sed -i "/^nifi.cluster.is.node/s/.\*/nifi.cluster.is.node=$\{IS\_CLUSTER:-false\}/" conf/nifi.properties  
sed -i "/^nifi.cluster.node.address=/s/.\*/nifi.cluster.node.address=$\{HOSTNAME\}/" conf/nifi.properties  
sed -i "/^nifi.zookeeper.connect.string=/s/.\*/nifi.zookeeper.connect.string=$\{ZK\_CONNECTION\_STR\}/" conf/nifi.properties  
sed -i "/^nifi.cluster.node.protocol.port=/s/.\*/nifi.cluster.node.protocol.port=$\{NODE\_PORT:-10443\}/" conf/nifi.properties  
  
  
if \[ $\{USE\_EMMBEDED\_ZK:-false\} == true \]  
then  
\#Zookeeper conf  
sed -i "/^clientPort=/s/.\*/clientPort=$\{ZK\_CLI\_PORT:-2181\}/" conf/zookeeper.properties  
sed -i "/^server.1=/s/.\*/server.1=$\{ZK\_HOSTNAME\}:$\{ZK\_VOTE\_PORT:-2888\}:$\{ZK\_LEADER\_PORT:-3888\}/" conf/zookeeper.properties  
mkdir -p state/zookeeper  
fi  
  
\#镜像后台运行  
/usr/sbin/sshd -D  
\#/root/data\_collect/nifi-$\{NIFI\_VERSION\}/bin/nifi.sh start