[kubernetes]-k8s1.18运行logstash7.11.1

左手的ㄟ右手 2022-11-13 04:19 91阅读 0赞

导语：想把logstash放到k8s里 但是看到网上都是直接运行或者cm挂载一个配置 对内存的调整好像并没有，带着疑惑测试了一下。

直接docker运行 发现默认是1G内存

看很多博客好像都没有调整的，大佬们都不需要改的么。不知道怎么通过configmap挂进去，这边只能暂时去掉config/jvm.options中对内存的限制 通过限制pod的内存来限制logstash的内存。否则filebeat上传数据多了 logstash就撑不住了。

![f8a56cfccb0120d10b7605bc65c99c13.png][]

解压完logstash安装包 并自行修改相关配置

tar zxvf logstash-7.11.1-linux-x86_64.tar.gz

修改完配置之后再压成tar.gz

tar zcvf logstash-7.11.1.tar.gz  logstash-7.11.1

创建自定义logstash镜像Dockerfile如下

FROM centos:centos7.4.1708
    MAINTAINER PDABC Enterprise Container Images <jiaminxu@hz-health.cn>
    
    ADD jdk-8u161-linux-x64.tar.gz /usr/local/java/
    ADD logstash-7.11.1.tar.gz /srv/
    ADD beat_sa_output.config /srv/logstash-7.11.1/
    RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
    
    WORKDIR /srv/logstash-7.11.1
    ENV JAVA_HOME /usr/local/java/jdk1.8.0_161
    ENV JRE_HOME ${JAVA_HOME}/jre
    ENV CLASSPATH .:${JAVA_HOME}/lib:${JRE_HOME}/lib
    ENV PATH ${JAVA_HOME}/bin:$PATH
    
    RUN /srv/logstash-7.11.1/bin/logstash-plugin install logstash-output-sensors_analytics
    
    #公开端口
    EXPOSE 5044
    
    USER root
    #设置启动命令
    ENTRYPOINT ["/srv/logstash-7.11.1/bin/logstash","-f","/srv/logstash-7.11.1/beat_sa_output.config"]

docker build -t registry.cn-shanghai.aliyuncs.com/yjk-datag/sensor-logstash:v04  .

![db2a3d1d290047e71addebe46a89f594.png][]

docker push   registry.cn-shanghai.aliyuncs.com/yjk-datag/sensor-logstash:v04
    docker run --name=sensor-logstash   -it --rm  registry.cn-shanghai.aliyuncs.com/yjk-datag/sensor-logstash:v04

启动成功

![9e3787fd7fb2fe6969c7d5546508d747.png][]

![5ae9b473bb283ab64b981b01b2209fdf.png][]

k8s启动logstash测试 直接使用了hostport yaml如下

apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: sensor-logstash-dp
    spec:
      replicas: 1
      # 在定义模板的时候必须定义labels,因为Deployment.spec.selector是必须字段,而他又必须和template.labels对应
      selector:
        matchLabels:
          app: sensor-logstash
      # template里面定义的内容会应用到下面所有的副本集里面(例如depolyment下的pod),在template.spec.containers里面不能定义labels标签。可以kubectl get pods --show-labels查看
      template:
        metadata:
          labels:
            app: sensor-logstash
        spec:
    #      nodeSelector:
    #        type: testz
          containers:
          # containers名字 和svc 和ingress绑定没关系
          - name: sensor-logstash
            image: registry.cn-shanghai.aliyuncs.com/yjk-datag/sensor-logstash:v04
            resources:
              limits:
                cpu: 2000m
                # 这里内存设置不能和logstash内存一样大 否则会重启
                memory: 6Gi
              requests:
                cpu: 1000m
                memory: 2Gi
            ports:
            - name: liveness-port
              containerPort: 5044
              hostPort: 5044
    # 后续需要jacoco端口也暴露一下
    
            livenessProbe:
              tcpSocket:
                port: 5044
              initialDelaySeconds: 30
              periodSeconds: 3
    
            readinessProbe:
              tcpSocket:
                port: 5044
              initialDelaySeconds: 30
              periodSeconds: 20

使用nodeport也可以svc.yml

---
    apiVersion: v1
    kind: Service
    metadata:
      name: sensor-logstash
    spec:
      selector:
        app: sensor-logstash
      type: NodePort
      ports:
        - name: tcp
          port: 5044
          nodePort: 30344

启动成功

![93c1d8532408386d099822e7e4f56910.png][]

参考与纠正

https://blog.csdn.net/Yan\_Chou/article/details/79015171

[f8a56cfccb0120d10b7605bc65c99c13.png]: /images/20221022/b5d8955f01884b5190cbdf33d27dd1d9.png
[db2a3d1d290047e71addebe46a89f594.png]: /images/20221022/a2e5057b67184654a2d71af667b5a540.png
[9e3787fd7fb2fe6969c7d5546508d747.png]: /images/20221022/28e8f3008e4d42c188206647acba67b1.png
[5ae9b473bb283ab64b981b01b2209fdf.png]: /images/20221022/91fd75283a5443cbb9e16ef5042c32e2.png
[93c1d8532408386d099822e7e4f56910.png]: https://img-blog.csdnimg.cn/img_convert/93c1d8532408386d099822e7e4f56910.png