手把手教你写一个通用的helm chart

谁借莪１个温暖的怀抱￠ 2021-11-14 15:12 812阅读 0赞

### 文章目录 ###

*  1. 模板介绍
 *  2. 新chart制作
 *   *  2.1 目录准备
     *  2.2 修改\`Chart.yaml\`
     *  2.3 修改\`values.yaml\`
     *  2.4 修改\`README.md\`和\`templates/NOTES.txt\`
     *  2.5 \`templates\`下yaml简要说明
 *  3. 小结

# 1. 模板介绍 #

首先，放上此模板链接：

[https://github.com/ygqygq2/charts/tree/master/mod-chart][https_github.com_ygqygq2_charts_tree_master_mod-chart]

此chart可当作POD单image的通用模板，只需要使用`sed`替换下chart名，并修改下`README.md`和`NOTES.txt`就可以了。下文，我通过复制此chart成`example-chart`来作示范说明。

[root@master1 mod-chart]# tree
    .
    ├── Chart.yaml  # chart版本信息文件
    ├── README.md  # chart说明文件
    ├── templates  # kubernetes资源yaml模板
    │   ├── configmap.yaml  # configmap模板
    │   ├── deployment-statefulset.yaml  # deployment或statefulset模板
    │   ├── _helpers.tpl  # 辅助模板和 partials
    │   ├── ingress.yaml  # ingress模板
    │   ├── NOTES.txt  # 部署chart后输出的帮助文档
    │   ├── pvc.yaml  # pvc模板
    │   ├── secret.yaml  # secret模板
    │   ├── service-headless.yaml  # service headless模板
    │   └── service.yaml  # service模板
    └── values.yaml  # 默认设置
    
    1 directory, 12 files
    [root@master1 mod-chart]# helm3 lint --strict .
    1 chart(s) linted, 0 chart(s) failed

# 2. 新chart制作 #

> 注：  
> 下文中文件内容我保留，只加注释。  
> 注释中需要修改的地方 `[*]` 标记为必选，`[-]` 标识为可选。

## 2.1 目录准备 ##

将模板`mod-chart`复制成`example-chart`，并作内容替换。

rsync -avz mod-chart/ example-chart/
    cd example-chart/
    sed -i 's@mod-chart@example-chart@g' *.*
    sed -i 's@mod-chart@example-chart@g' templates/*.*

## 2.2 修改`Chart.yaml` ##

`vim Chart.yaml`

apiVersion: v1  # 当前helm api版本，不需要修改
    appVersion: 1.14.2  # 此处为你应用程序的版本号 [*]
    description: Chart for the nginx server  # 介绍此chart是干嘛的，按需求修改
    engine: gotpl  # go模板引擎，不需要修改 [-]
    name: example-chart  # 模板名，对应目录名 [*]
    version: 1.0.0  # 此chart版本号 [*]
    home: http://www.nginx.org  # 应用程序官网 [*]
    icon: https://bitnami.com/assets/stacks/nginx/img/nginx-stack-220x234.png  # 应用程序logo地址 [*]
    keywords:  # 关键字列表 [*]
    - nginx
    - http
    - web
    - www
    - reverse proxy
    maintainers:  # 维护人员列表 [*]
    - email: 29ygq@sina.com
      name: Chinge Yang
    sources:  # 应用程序来源 [-]
    - https://github.com/bitnami/bitnami-docker-nginx

## 2.3 修改`values.yaml` ##

因为`values.yaml`设置涉及到yaml格式，yaml文件格式说明可以看这篇文章：

[http://www.ruanyifeng.com/blog/2016/07/yaml.html][http_www.ruanyifeng.com_blog_2016_07_yaml.html]

这里提几个常用的地方：

1.  使用2个空格作缩进；
2.  确认数字为字符类型时，使用双引号引起来；
3.  为了迎合helm3的规范，空定义最好将相关符号补上：

string: ""
    list: []
    map: { }

没什么特殊要求，一般需要修改的地方有`image`、`service`、`healthCheck`、`persistentVolume.mountPaths`

# Default values for mod-chart.
    # This is a YAML-formatted file.
    # Declare variables to be passed into your templates.
    
    ## Global Docker image parameters
    ## Please, note that this will override the image parameters, including dependencies, configured to use the global value
    ## Current available global Docker image parameters: imageRegistry and imagePullSecrets
    ##
    global:  # 设置后覆盖后面默认的镜像仓库
      imageRegistry: ""
      imagePullSecrets: []
    # - myRegistryKeySecretName
    
    statefulset:
      enabled: false
    
    ## String to partially override fullname template (will maintain the release name)
    ##
    nameOverride: ""
    
    ## String to fully override fullname template
    ##
    fullnameOverride: ""
    
    ## By default deploymentStrategy is set to rollingUpdate with maxSurge of 25% and maxUnavailable of 25% .
    ## You can change type to `Recreate` or can uncomment `rollingUpdate` specification and adjust them to your usage.
    deploymentStrategy: { }
      # rollingUpdate:
      # maxSurge: 25%
      # maxUnavailable: 25%
      # type: RollingUpdate
    
    # 副本个数
    replicaCount: 1
    
    # 容器image及tag
    image:
      registry: docker.io
      repository: bitnami/nginx
      tag: latest
      pullPolicy: IfNotPresent  # IfNotPresent: 有则不拉（减少流量和操作步骤），Always: 不管tag总拉（适合tag不变时更新）
      pullSecrets: []
      # - private-registry-key
    
    service:
      type: ClusterIP  # 一般不用修改
      ingressPort: 8080
      ports:
        http:  # 多端口暴露时，复制一段
          port: 8080 # Service port number for client-a port.
          protocol: TCP  # Service port protocol for client-a port.
    
    ## env set
    ## ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/
    env: []
    # - name: DEMO_GREETING
    # value: "Hello from the environment"
    # - name: DEMO_FAREWELL
    # value: "Such a sweet sorrow"
    
    ## command set
    startCommand: []
    # - "java -Xdebug -Xnoagent -Djava.compiler=NONE"
    # - "-Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"
    # - "-Djava.security.egd=file:/dev/urandom"
    # - "-jar /test.jar"
    # - "-Duser.timezone=GMT+08"
    
    ## Enable configmap and add data in configmap
    config:
      enabled: false
      subPath: ""
      mountPath: /conf
      data: { }
        
    ############################# 示例 ####################################
    ## 以下示例，挂载文件至 /conf/app.conf
    # enabled: true
    # mountPath: /conf/app.conf 
    # subPath: app.conf # 使用subPath时，上面mountPath路径写文件完整绝对路径
    # data:
    # app.conf: |-
    # appname = example-chart
    
    ## 以下示例，挂载多个文件至 /conf/ 下
    # enabled: true
    # mountPath: /conf # 不使用subPath
    # data:
    # app.conf: |-
    # appname = example-chart
    # bpp.conf: |-
    # bppname
    #
    ## 挂载多个文件至多个不同路径，需要相应修改 templates/deployment-statefulset.yaml 
    ############################# 示例 ####################################
    
    ## To use an additional secret, set enable to true and add data
    ## 用法同上，不另作说明
    secret:
      enabled: false
      mountPath: /etc/secret-volume
      subPath: ""
      readOnly: true
      data: { } 
    
    ## liveness and readiness 
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
    healthCheck:
      enabled: true
      type: tcp  # http/tcp
      port: http  # 健康检查的端口名或端口
      httpPath: '/'  # http时必须设置
      livenessInitialDelaySeconds: 10 # 初始延迟秒数
      livenessPeriodSeconds: 10 # 检测周期，默认值10，最小为1
      readinessInitialDelaySeconds: 10 # 初始延迟秒数
      readinessPeriodSeconds: 10 # 检测周期，默认值10，最小为1
     
    resources: { }
      # 容器资源设置
      # We usually recommend not to specify default resources and to leave this as a conscious
      # choice for the user. This also increases chances charts run on environments with little
      # resources, such as Minikube. If you do want to specify resources, uncomment the following
      # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
      # limits:
      # cpu: 100m
      # memory: 128Mi
      # requests:
      # cpu: 100m
      # memory: 128Mi
    
    ## Node labels and tolerations for pod assignment
    ### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
    ### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
    labels: { }
    podAnnotations: { }
    nodeSelector: { }
    tolerations: []
    affinity: { }
    annotations: { }
    
    ## Enable persistence using Persistent Volume Claims
    ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
    ##
    persistentVolume:   # 是否存储持久化
      enabled: false
      ## If defined, storageClassName: <storageClass>
      ## If set to "-", storageClassName: "", which disables dynamic provisioning
      ## If undefined (the default) or set to null, no storageClassName spec is
      ## set, choosing the default provisioner. (gp2 on AWS, azure-disk on
      ## Azure, standard on GKE, AWS & OpenStack)
      ##
      storageClass: "-"
      accessMode: ReadWriteOnce
      annotations: { }
      # helm.sh/resource-policy: keep
      size: 1Gi  # 大小
      existingClaim: { }  # 使用已存在的pvc
      mountPaths: []
      # - name: data-storage
      # mountPath: /config
      # subPath: config # 多个路径使用同一个pvc使用subPath，用法同上面config中示例说明
      # - name: data-storage
      # mountPath: /data
      # subPath: data
    
    ingress:  # 是否使用nginx暴露域名或端口
      enabled: false
      annotations: { }
        # kubernetes.io/ingress.class: nginx
        # kubernetes.io/tls-acme: "true"
      path: /
      hosts:
        - chart-example.local
      tls: []
      # - secretName: chart-example-tls
      # hosts:
      # - chart-example.local
    
    ## Add init containers. e.g. to be used to give specific permissions for data
    ## Add your own init container or uncomment and modify the given example.
    initContainers: []
    
    ## Prometheus Exporter / Metrics
    ##
    metrics:
      enabled: false
      image:
        registry: docker.io
        repository: nginx/nginx-prometheus-exporter
        tag: 0.1.0
        pullPolicy: IfNotPresent
        ## Optionally specify an array of imagePullSecrets.
        ## Secrets must be manually created in the namespace.
        ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
        ##
        pullSecrets: []
        # - myRegistrKeySecretName
      ## Metrics exporter pod Annotation and Labels
      podAnnotations:
        # prometheus.io/scrape: "true"
        # prometheus.io/port: "9113"
        ## Metrics exporter resource requests and limits
        ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
        ##
      resources: { }
    
    ## Uncomment and modify this to run a command after starting the core container.
    ## ref: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/
    lifecycle: { }
      # preStop:
      # exec:
      # command: ["/bin/bash","/pre-stop.sh"]
      # postStart:
      # exec:
      # command: ["/bin/bash","/post-start.sh"]
    
    ## Deployment additional volumes.
    deployment:
      additionalVolumes: []
    
    ## init containers
    ## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
    ## Add init containers. e.g. to be used to give specific permissions for data
    ## Add your own init container or uncomment and modify the given example.
    initContainers: { }
    # - name: fmp-volume-permission
    # image: busybox
    # imagePullPolicy: IfNotPresent
    # command: ['chown','-R', '200', '/extra-data']
    # volumeMounts:
    # - name: extra-data
    # mountPath: /extra-data
    
    ## Additional containers to be added to the core pod.
    additionalContainers: { }
    # - name: my-sidecar
    # image: nginx:latest
    # - name: lemonldap-ng-controller
    # image: lemonldapng/lemonldap-ng-controller:0.2.0
    # args:
    # - /lemonldap-ng-controller
    # - --alsologtostderr
    # - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration
    # env:
    # - name: POD_NAME
    # valueFrom:
    # fieldRef:
    # fieldPath: metadata.name
    # - name: POD_NAMESPACE
    # valueFrom:
    # fieldRef:
    # fieldPath: metadata.namespace
    # volumeMounts:
    # - name: copy-portal-skins
    # mountPath: /srv/var/lib/lemonldap-ng/portal/skins

## 2.4 修改`README.md`和`templates/NOTES.txt` ##

根据 `values.yaml`中的默认设置相应修改`README.md`，内容使用markdown语法，这里不作详细说明。  
`templates/NOTES.txt`是部署chart后输出的帮助文档，里面支持go template语法。模板里已经写成了非常通用。必要情况下，适当按应用需求来修改，这样会显得部署后提示非常友好和人性化。

## 2.5 `templates`下yaml简要说明 ##

`templates`目录下为kubernetes资源yaml文件模板，以资源名命名文件名，复杂些的可以加上资源功能或者模块名等。

`templates/secret.yaml`

{ { - if .Values.secret.enabled }}  # if用法。语法代码注意形成一致缩进习惯，便于阅读
    apiVersion: v1
    kind: Secret
    metadata:
      name: { {  template "example-chart.fullname" . }}  # 使用辅助模板时，点号可用 $ 代替，特别是在range下
      labels:
        app: { {  template "example-chart.name" . }}
        chart: { {  template "example-chart.chart" . }}
        release: "{ { .Release.Name }}"
        heritage: "{ { .Release.Service }}"
        { { - if .Values.labels }}
    { {  toYaml .Values.labels | indent 4 }}  # 使用toYaml时，不缩进，indent接空格数
        { { - end }}
    data:
    { { - range $key, $value := .Values.secret.data }}  # range用法
      { {  $key }}: { {  $value | b64enc | quote }}  # secret中value注意使用base64转换
    { { - end }}
    { { - end }}

`templates/deployment-statefulset.yaml`

{ { - if .Values.statefulset.enabled }}  # 判断使用deployment和statefulset资源api类型
    apiVersion: apps/v1
    kind: StatefulSet
    { { - else }}
    apiVersion: apps/v1
    kind: Deployment
    { { - end }}
    metadata:
      name: { {  template "example-chart.fullname" . }}
      labels:
        app: { {  template "example-chart.name" . }}
        chart: { {  template "example-chart.chart" . }}
        release: { {  .Release.Name }}
        heritage: { {  .Release.Service }}
    { { - if .Values.labels }}  # 额外的标签
    { {  toYaml .Values.labels | indent 4 }}
    { { - end }}
    { { - if .Values.annotations }}  # 自定义注释
      annotations:
    { {  toYaml .Values.annotations | indent 4 }}
    { { - end }}
    spec:
      replicas: { {  .Values.replicaCount }}  # 副本数
      { { - if .Values.statefulset.enabled }}  # statefulset需要定义serviceName
      serviceName: { {  template "example-chart.fullname" . }}-headless
      { { - end }}
      { { - if .Values.deploymentStrategy }}
      strategy:
    { {  toYaml .Values.deploymentStrategy | indent 4 }}
      { { - end }}
      selector:
        matchLabels:
          app: { {  template "example-chart.name" . }}
          release: { {  .Release.Name }}
      template:
        metadata:
          annotations:
      { { - if .Values.podAnnotations }}
    { {  toYaml .Values.podAnnotations | indent 8 }}
    { { - end }}
    { { - if .Values.metrics.podAnnotations }}
    { {  toYaml .Values.metrics.podAnnotations | indent 8 }}
    { { - end }}
          labels:
            app: { {  template "example-chart.name" . }}
            release: { {  .Release.Name }}
        spec:
    { { - include "example-chart.imagePullSecrets" . | indent 6 }}
          { { - if .Values.initContainers }}
          initContainers:
    { {  toYaml .Values.initContainers | indent 8 }}
          { { - end }}
          nodeSelector:
    { {  toYaml .Values.nodeSelector | indent 8 }}
          affinity:
    { {  toYaml .Values.affinity | indent 8 }}
          tolerations:
    { {  toYaml .Values.tolerations | indent 8 }}
          containers:
    { { - if .Values.metrics.enabled }}  # metrics容器可根据需求修改
            - name: metrics
              image: { {  template "metrics.image" . }}
              imagePullPolicy: { {  .Values.metrics.image.pullPolicy | quote }}
              command: [ '/usr/bin/exporter', '-nginx.scrape-uri', 'http://127.0.0.1:8080/status']
              ports:
              - name: metrics
                containerPort: 9113
              livenessProbe:
                httpGet:
                  path: /metrics
                  port: metrics
                initialDelaySeconds: 15
                timeoutSeconds: 5
              readinessProbe:
                httpGet:
                  path: /metrics
                  port: metrics
                initialDelaySeconds: 5
                timeoutSeconds: 1
              resources:
    { {  toYaml .Values.metrics.resources | indent 12 }}
    { { - end }}
            - name: { {  .Chart.Name }}
              image: { {  template "example-chart.image" . }}
              imagePullPolicy: { {  .Values.image.pullPolicy | quote }}
              { { - if .Values.lifecycle }}
              lifecycle:
    { {  toYaml .Values.lifecycle | indent 12 }}
              { { - end }}
              { { - if .Values.startCommand }}
              command:
    { {  toYaml .Values.startCommand |indent 12 }}
              { { - end }}
              env:
    { {  toYaml .Values.env | indent 12 }}
              resources:
    { {  toYaml .Values.resources | indent 12 }}
              ports:
                { { - range $key, $value := .Values.service.ports }}
                - name: { {  $key }}
                  containerPort: { {  $value.port }}
                  protocol: { {  $value.protocol }}
                { { - end }}
              { { - if .Values.healthCheck.enabled }}
              livenessProbe:
                { { - if eq .Values.healthCheck.type "http" }}
                httpGet:
                  path: { {  .Values.healthCheck.httpPath }}
                  port: { {  .Values.healthCheck.port }}
                { { - else }}
                tcpSocket:
                  port: { {  .Values.healthCheck.port }}
                { { - end }}
                initialDelaySeconds: { {  .Values.healthCheck.livenessInitialDelaySeconds }}
                periodSeconds: { {  .Values.healthCheck.livenessPeriodSeconds }}
              readinessProbe:
                { { - if eq .Values.healthCheck.type "http" }}
                httpGet:
                  path: { {  .Values.healthCheck.httpPath }}
                  port: { {  .Values.healthCheck.port }}
                { { - else }}
                tcpSocket:
                  port: { {  .Values.healthCheck.port }}
                { { - end }}
                initialDelaySeconds: { {  .Values.healthCheck.readinessInitialDelaySeconds }}
                periodSeconds: { {  .Values.healthCheck.readinessPeriodSeconds }}
              { { - end }}
              volumeMounts:  # 容器挂载点
                { { - if .Values.config.enabled }}
                - name: { {  template "example-chart.name" . }}-conf
                  mountPath: { {  .Values.config.mountPath }}
                  subPath: { {  .Values.config.subPath }}
                { { - end }}
                { { - if .Values.secret.enabled }}
                - name: { {  template "example-chart.name" . }}-secret
                  mountPath: { {  .Values.secret.mountPath }}
                  subPath: { {  .Values.secret.subPath }}
                  readOnly: { {  .Values.secret.readOnly }}
                { { - end }}
    { { - if .Values.persistentVolume.mountPaths }}
    { {  toYaml .Values.persistentVolume.mountPaths | indent 12 }}
    { { - end }}
            { { - if .Values.additionalContainers }}
    { {  toYaml .Values.additionalContainers | indent 8 }}
            { { - end }}
          volumes:  # volume名需要和上文volumeMounts中的名字一一对应
            { { - if .Values.config.enabled }}
            - name: { {  template "example-chart.name" . }}-conf
              configMap:
                name: { {  template "example-chart.fullname" . }}
            { { - end }}
            { { - if .Values.secret.enabled }}
            - name: { {  template "example-chart.name" . }}-secret
              secret:
                secretName: { {  template "example-chart.fullname" . }}
            { { - end }}
            { { - if .Values.deployment.additionalVolumes }}
    { {  toYaml .Values.deployment.additionalVolumes | indent 8 }}
            { { - end }}
    { { - if not .Values.statefulset.enabled }}
              { { - if .Values.persistentVolume.enabled }}
            - name: data-storage
              persistentVolumeClaim:
                claimName: { {  .Values.persistentVolume.existingClaim | default (include "example-chart.fullname" .) }}
              { { - else }}
            - name: data-storage
              emptyDir: { }
              { { - end }}
    { { - else }}
      { { - if .Values.persistentVolume.enabled }}
      volumeClaimTemplates:
      - metadata:
          name: data-storage
          labels:
            app: { {  template "example-chart.name" . }}
            chart: { {  template "example-chart.chart" . }}
            release: { {  .Release.Name }}
            heritage: { {  .Release.Service }}
        spec:
          accessModes:
            - { {  .Values.persistentVolume.accessMode | quote }}
          annotations:
          { { - range $key, $value := $.Values.persistentVolume.annotations }}
            { {  $key }}: { {  $value }}
          { { - end }}
          resources:
            requests:
              storage: { {  .Values.persistentVolume.size }}
              { { - if .Values.persistentVolume.storageClass }}
                { { - if (eq "-" .Values.persistentVolume.storageClass) }}
          storageClassName: ""
                { { - else }}
          storageClassName: "{ { .Values.persistentVolume.storageClass }}"
                { { - end }}
              { { - end }}
      { { - else }}
            - name: data-storage
              emptyDir: { }
      { { - end }}
    { { - end -}}

# 3. 小结 #

以上yaml中未作详细说明，仔细看其内容都能明白大致意思。以下是对于helm chart新手的一些建议：

1.  刚接触helm chart时，多模仿[stable][]和[bianami][]中charts的写法，特别是`values.yaml`和`templates`目录中的一些设计，差不多都已经非常统一了。这样遇到自己有相似需求，可直接使用相应的功能块，写出来的chart也显得非常专业。
2.  使用chart时，最好使用`helm`命令`fetch`下来，大概读一遍其chart内容，这样看多了，自然就越来越熟悉，而且出错时，也便于自己排查问题。

参考资料：  
\[1\] [https://helm.sh/][https_helm.sh]  
\[2\] [https://whmzsu.github.io/helm-doc-zh-cn/][https_whmzsu.github.io_helm-doc-zh-cn]

[https_github.com_ygqygq2_charts_tree_master_mod-chart]: https://github.com/ygqygq2/charts/tree/master/mod-chart
[http_www.ruanyifeng.com_blog_2016_07_yaml.html]: http://www.ruanyifeng.com/blog/2016/07/yaml.html
[stable]: https://kubernetes-charts.storage.googleapis.com
[bianami]: https://charts.bitnami.com/bitnami
[https_helm.sh]: https://helm.sh/
[https_whmzsu.github.io_helm-doc-zh-cn]: https://whmzsu.github.io/helm-doc-zh-cn/