helm部署harbor

Love The Way You Lie 2022-12-12 04:35 115阅读 0赞

helm部署harbor。chart 最新版本：1.5.0

--------------------

*  namespace：

public-service-ns.yaml

apiVersion: v1
    kind: Namespace
    metadata:
      name: public-service

kubectl apply -f public-service-ns.yaml

*  nfs存储：

yum install -y nfs-utils rpcbind
    
    mkdir -p /data/harbor/{ chartmuseum,jobservice,registry,database,redis,trivy}
    
    vim /etc/exports

/data/harbor    192.168.30.0/24(rw,sync,no_root_squash)

chmod -R 777 /data/harbor
    
    exportfs -arv
    
    systemctl enable rpcbind && systemctl start rpcbind
    
    systemctl enable nfs && systemctl start nfs

nfs部署完毕。对于需要使用nfs的node节点，都要安装nfs：

yum install -y nfs-utils

harbor-pv.yaml

apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: harbor-chartmuseum
      labels:
        app: harbor
        component: chartmuseum
    spec:
      capacity:
        storage: 5Gi
      accessModes:
        - ReadWriteOnce
      persistentVolumeReclaimPolicy: Recycle
      nfs:
        server: 192.168.30.129
        path: /data/harbor/chartmuseum
      
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: harbor-jobservice
      labels:
        app: harbor
        component: jobservice
    spec:
      capacity:
        storage: 1Gi
      accessModes: 
        - ReadWriteOnce
      persistentVolumeReclaimPolicy: Recycle
      nfs:
        server: 192.168.30.129
        path: /data/harbor/jobservice
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: harbor-registry
      labels:
        app: harbor
        component: registry
    spec:
      capacity:
        storage: 10Gi
      accessModes: 
        - ReadWriteOnce
      persistentVolumeReclaimPolicy: Recycle
      nfs:
        server: 192.168.30.129
        path: /data/harbor/registry
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: harbor-database
      labels:
        app: harbor
        component: database
    spec:
      capacity:
        storage: 5Gi
      accessModes: 
        - ReadWriteOnce
      persistentVolumeReclaimPolicy: Recycle
      nfs:
        server: 192.168.30.129
        path: /data/harbor/database
        
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: harbor-redis
      labels:
        app: harbor
        component: redis
    spec:
      capacity:
        storage: 1Gi
      accessModes: 
        - ReadWriteOnce
      persistentVolumeReclaimPolicy: Recycle
      nfs:
        server: 192.168.30.129
        path: /data/harbor/redis
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: harbor-trivy
      labels:
        app: harbor
        component: trivy
    spec:
      capacity:
        storage: 5Gi
      accessModes: 
        - ReadWriteOnce
      persistentVolumeReclaimPolicy: Recycle
      nfs:
        server: 192.168.30.129
        path: /data/harbor/trivy

kubectl apply -f harbor-pv.yaml
    
    kubectl get pv
     
    NAME                 CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
    harbor-chartmuseum   5Gi        RWO            Recycle          Available                                   7s
    harbor-database      5Gi        RWO            Recycle          Available                                   7s
    harbor-jobservice    1Gi        RWO            Recycle          Available                                   7s
    harbor-redis         1Gi        RWO            Recycle          Available                                   7s
    harbor-registry      10Gi       RWO            Recycle          Available                                   7s
    harbor-trivy         5Gi        RWO            Recycle          Available                                   7s

*  配置：

helm repo add harbor https://helm.goharbor.io
    
    helm repo update
    
    helm pull harbor/harbor
    
    tar xf harbor-1.5.0.tgz
    
    vim harbor/values.yaml              #更改

expose:
      ingress:
        hosts:
          core: harbor.lzxlinux.com
          notary: notary.lzxlinux.com
          
    externalURL: https://harbor.lzxlinux.com
    
    persistence:
      persistentVolumeClaim:
        registry:
          size: 10Gi
        chartmuseum:
          size: 5Gi
        jobservice:
          size: 1Gi
        database:
          size: 5Gi
        redis:
          size: 1Gi
        trivy:
          size: 5Gi
    
    harborAdminPassword: "Harbor12345"              # harbor admin密码
    
    nginx:
      resources:
        requests:
          memory: 256Mi
          cpu: 100m
    
    portal:
      resources:
        requests:
          memory: 256Mi
          cpu: 100m
    
    core:
      resources:
        requests:
          memory: 256Mi
          cpu: 100m
          
    jobservice:
      resources:
        requests:
          memory: 256Mi
          cpu: 100m
    
    registry:
      registry:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
      controller:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
    
    chartmuseum:
      resources:
        requests:
          memory: 256Mi
          cpu: 100m
    
    clair:
      clair:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
      adapter:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
            
    trivy:
      resources:
        requests:
          cpu: 200m
          memory: 512Mi
        limits:
          cpu: 1
          memory: 1Gi
          
    notary:
      server:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
      signer:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
            
    database:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
        
    redis:
        resources:
          requests:
            memory: 256Mi
            cpu: 100m

vim harbor/templates/_helpers.tpl

{ { /* Helm required labels */}}
    { { - define "harbor.labels" -}}
    app: "{ {  template "harbor.name" . }}"
    { { - end -}}
    
    { { /* matchLabels */}}
    { { - define "harbor.matchLabels" -}}
    app: "{ {  template "harbor.name" . }}"
    { { - end -}}

vim harbor/templates/chartmuseum/chartmuseum-pvc.yaml

{ { - if .Values.chartmuseum.enabled }}
    { { - $persistence := .Values.persistence -}}
    { { - if $persistence.enabled }}
    { { - $chartmuseum := $persistence.persistentVolumeClaim.chartmuseum -}}
    { { - if and (not $chartmuseum.existingClaim) (eq $persistence.imageChartStorage.type "filesystem") }}
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: { {  template "harbor.chartmuseum" . }}
      { { - if eq $persistence.resourcePolicy "keep" }}
      annotations:
        helm.sh/resource-policy: keep
      { { - end }}
      labels:
    { {  include "harbor.labels" . | indent 4 }}
        component: chartmuseum
    spec:
      accessModes:
        - { {  $chartmuseum.accessMode }}
      resources:
        requests:
          storage: { {  $chartmuseum.size }}
      { { - if $chartmuseum.storageClass }}
        { { - if eq "-" $chartmuseum.storageClass }}
      storageClassName: ""
        { { - else }}
      storageClassName: { {  $chartmuseum.storageClass }}
        { { - end }}
      { { - end }}
      selector:
        matchLabels:
    { {  include "harbor.matchLabels" . | indent 6 }}
          component: chartmuseum
    { { - end }}
    { { - end }}
    { { - end }}

vim harbor/templates/jobservice/jobservice-pvc.yaml

{ { - $jobservice := .Values.persistence.persistentVolumeClaim.jobservice -}}
    { { - if and .Values.persistence.enabled (not $jobservice.existingClaim) }}
    { { - if eq .Values.jobservice.jobLogger "file" }}
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: { {  template "harbor.jobservice" . }}
      { { - if eq .Values.persistence.resourcePolicy "keep" }}
      annotations:
        helm.sh/resource-policy: keep
      { { - end }}
      labels:
    { {  include "harbor.labels" . | indent 4 }}
        component: jobservice
    spec:
      accessModes:
        - { {  $jobservice.accessMode }}
      resources:
        requests:
          storage: { {  $jobservice.size }}
      { { - if $jobservice.storageClass }}
        { { - if eq "-" $jobservice.storageClass }}
      storageClassName: ""
        { { - else }}
      storageClassName: { {  $jobservice.storageClass }}
        { { - end }}
      { { - end }}
      selector:
        matchLabels:
    { {  include "harbor.matchLabels" . | indent 6 }}
          component: jobservice
    { { - end }}
    { { - end }}

vim harbor/templates/registry/registry-pvc.yaml

{ { - if .Values.persistence.enabled }}
    { { - $registry := .Values.persistence.persistentVolumeClaim.registry -}}
    { { - if and (not $registry.existingClaim) (eq .Values.persistence.imageChartStorage.type "filesystem") }}
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: { {  template "harbor.registry" . }}
      { { - if eq .Values.persistence.resourcePolicy "keep" }}
      annotations:
        helm.sh/resource-policy: keep
      { { - end }}
      labels:
    { {  include "harbor.labels" . | indent 4 }}
        component: registry
    spec:
      accessModes:
        - { {  $registry.accessMode }}
      resources:
        requests:
          storage: { {  $registry.size }}
      { { - if $registry.storageClass }}
        { { - if eq "-" $registry.storageClass }}
      storageClassName: ""
        { { - else }}
      storageClassName: { {  $registry.storageClass }}
        { { - end }}
      { { - end }}
      selector:
        matchLabels:
    { {  include "harbor.matchLabels" . | indent 6 }}
          component: registry
    { { - end }}
    { { - end }}

vim harbor/templates/database/database-ss.yaml

{ { - if eq .Values.database.type "internal" -}}
    { { - $database := .Values.persistence.persistentVolumeClaim.database -}}
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: "{ {  template "harbor.database" . }}"
      labels:
    { {  include "harbor.labels" . | indent 4 }}
        component: database
    spec:
      replicas: 1
      serviceName: "{ {  template "harbor.database" . }}"
      selector:
        matchLabels:
    { {  include "harbor.matchLabels" . | indent 6 }}
          component: database
      template:
        metadata:
          labels:
    { {  include "harbor.labels" . | indent 8 }}
            component: database
          annotations:
            checksum/secret: { {  include (print $.Template.BasePath "/database/database-secret.yaml") . | sha256sum }}
    { { - if .Values.database.podAnnotations }}
    { {  toYaml .Values.database.podAnnotations | indent 8 }}
    { { - end }}
        spec:
    { { - if .Values.database.internal.serviceAccountName }}
          serviceAccountName: { {  .Values.database.internal.serviceAccountName }}
    { { - end -}}
          { { - with .Values.imagePullSecrets }}
          imagePullSecrets:
            { { - toYaml . | nindent 8 }}
          { { - end }}
          initContainers:
          - name: "change-permission-of-directory"
            securityContext:
              runAsUser: 0
            image: { {  .Values.database.internal.image.repository }}:{ {  .Values.database.internal.image.tag }}
            imagePullPolicy: { {  .Values.imagePullPolicy }}
            command: ["/bin/sh"]
            args: ["-c", "chown -R postgres:postgres /var/lib/postgresql/data"]
            volumeMounts:
            - name: database-data
              mountPath: /var/lib/postgresql/data
              subPath: { {  $database.subPath }}
          - name: "remove-lost-found"
            image: { {  .Values.database.internal.image.repository }}:{ {  .Values.database.internal.image.tag }}
            imagePullPolicy: { {  .Values.imagePullPolicy }}
            command: ["rm", "-Rf", "/var/lib/postgresql/data/lost+found"]
            volumeMounts:
            - name: database-data
              mountPath: /var/lib/postgresql/data
              subPath: { {  $database.subPath }}
          containers:
          - name: database
            image: { {  .Values.database.internal.image.repository }}:{ {  .Values.database.internal.image.tag }}
            imagePullPolicy: { {  .Values.imagePullPolicy }}
            livenessProbe:
              exec:
                command:
                - /docker-healthcheck.sh
              initialDelaySeconds: 300
              periodSeconds: 10
            readinessProbe:
              exec:
                command:
                - /docker-healthcheck.sh
              initialDelaySeconds: 1
              periodSeconds: 10
    { { - if .Values.database.internal.resources }}
            resources:
    { {  toYaml .Values.database.internal.resources | indent 10 }}
    { { - end }}
            envFrom:
              - secretRef:
                  name: "{ {  template "harbor.database" . }}"
            volumeMounts:
            - name: database-data
              mountPath: /var/lib/postgresql/data
              subPath: { {  $database.subPath }}
          { { - if not .Values.persistence.enabled }}
          volumes:
          - name: "database-data"
            emptyDir: { }
          { { - else if $database.existingClaim }}
          volumes:
          - name: "database-data"
            persistentVolumeClaim:
              claimName: { {  $database.existingClaim }}
          { { - end -}}
        { { - with .Values.database.internal.nodeSelector }}
          nodeSelector:
    { {  toYaml . | indent 8 }}
        { { - end }}
        { { - with .Values.database.internal.affinity }}
          affinity:
    { {  toYaml . | indent 8 }}
        { { - end }}
        { { - with .Values.database.internal.tolerations }}
          tolerations:
    { {  toYaml . | indent 8 }}
        { { - end }}
      { { - if and .Values.persistence.enabled (not $database.existingClaim) }}
      volumeClaimTemplates:
      - metadata:
          name: "database-data"
          labels:
    { {  include "harbor.labels" . | indent 8 }}
        spec:
          accessModes: [{ {  $database.accessMode | quote }}]
          { { - if $database.storageClass }}
          { { - if (eq "-" $database.storageClass) }}
          storageClassName: ""
          { { - else }}
          storageClassName: "{ { $database.storageClass }}"
          { { - end }}
          { { - end }}
          resources:
            requests:
              storage: { {  $database.size | quote }}
          selector:
            matchLabels:
    { {  include "harbor.matchLabels" . | indent 10 }}
              component: database
      { { - end -}}
      { { - end -}}

vim harbor/templates/redis/statefulset.yaml

{ { - if eq .Values.redis.type "internal" -}}
    { { - $redis := .Values.persistence.persistentVolumeClaim.redis -}}
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: { {  template "harbor.redis" . }}
      labels:
    { {  include "harbor.labels" . | indent 4 }}
        component: redis
    spec:
      replicas: 1
      serviceName: { {  template "harbor.redis" . }}
      selector:
        matchLabels:
    { {  include "harbor.matchLabels" . | indent 6 }}
          component: redis
      template:
        metadata:
          labels:
    { {  include "harbor.labels" . | indent 8 }}
            component: redis
    { { - if .Values.redis.podAnnotations }}
          annotations:
    { {  toYaml .Values.redis.podAnnotations | indent 8 }}
    { { - end }}
        spec:
          securityContext:
            runAsUser: 0
    { { - if .Values.redis.internal.serviceAccountName }}
          serviceAccountName: { {  .Values.redis.internal.serviceAccountName }}
    { { - end -}}
          { { - with .Values.imagePullSecrets }}
          imagePullSecrets:
            { { - toYaml . | nindent 8 }}
          { { - end }}
          containers:
          - name: redis
            image: { {  .Values.redis.internal.image.repository }}:{ {  .Values.redis.internal.image.tag }}
            imagePullPolicy: { {  .Values.imagePullPolicy }}
            livenessProbe:
              tcpSocket:
                port: 6379
              initialDelaySeconds: 300
              periodSeconds: 10
            readinessProbe:
              tcpSocket:
                port: 6379
              initialDelaySeconds: 1
              periodSeconds: 10
    { { - if .Values.redis.internal.resources }}
            resources:
    { {  toYaml .Values.redis.internal.resources | indent 10 }}
    { { - end }}
            volumeMounts:
            - name: data
              mountPath: /var/lib/redis
              subPath: { {  $redis.subPath }}
          { { - if not .Values.persistence.enabled }}
          volumes:
          - name: data
            emptyDir: { }
          { { - else if $redis.existingClaim }}
          volumes:
          - name: data
            persistentVolumeClaim:
              claimName: { {  $redis.existingClaim }}
          { { - end -}}
        { { - with .Values.redis.internal.nodeSelector }}
          nodeSelector:
    { {  toYaml . | indent 8 }}
        { { - end }}
        { { - with .Values.redis.internal.affinity }}
          affinity:
    { {  toYaml . | indent 8 }}
        { { - end }}
        { { - with .Values.redis.internal.tolerations }}
          tolerations:
    { {  toYaml . | indent 8 }}
        { { - end }}
      { { - if and .Values.persistence.enabled (not $redis.existingClaim) }}
      volumeClaimTemplates:
      - metadata:
          name: data
          labels:
    { {  include "harbor.labels" . | indent 8 }}
        spec:
          accessModes: [{ {  $redis.accessMode | quote }}]
          { { - if $redis.storageClass }}
          { { - if (eq "-" $redis.storageClass) }}
          storageClassName: ""
          { { - else }}
          storageClassName: "{ { $redis.storageClass }}"
          { { - end }}
          { { - end }}
          resources:
            requests:
              storage: { {  $redis.size | quote }}
          selector:
            matchLabels:
    { {  include "harbor.matchLabels" . | indent 10 }}
              component: redis
      { { - end -}}
      { { - end -}}

vim harbor/templates/trivy/trivy-sts.yaml

{ { - if .Values.trivy.enabled }}
    { { - $trivy := .Values.persistence.persistentVolumeClaim.trivy }}
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: { {  template "harbor.trivy" . }}
      labels:
    { {  include "harbor.labels" . | indent 4 }}
        component: trivy
    spec:
      replicas: { {  .Values.trivy.replicas }}
      serviceName: { {   template "harbor.trivy" . }}
      selector:
        matchLabels:
    { {  include "harbor.matchLabels" . | indent 6 }}
          component: trivy
      template:
        metadata:
          labels:
    { {  include "harbor.labels" . | indent 8 }}
            component: trivy
          annotations:
            checksum/secret: { {  include (print $.Template.BasePath "/trivy/trivy-secret.yaml") . | sha256sum }}
    { { - if and .Values.internalTLS.enabled (eq .Values.internalTLS.certSource "auto") }}
            checksum/tls: { {  include (print $.Template.BasePath "/internal/auto-tls.yaml") . | sha256sum }}
    { { - else if and .Values.internalTLS.enabled (eq .Values.internalTLS.certSource "manual") }}
            checksum/tls: { {  include (print $.Template.BasePath "/trivy/trivy-tls.yaml") . | sha256sum }}
    { { - end }}
    { { - if .Values.trivy.podAnnotations }}
    { {  toYaml .Values.trivy.podAnnotations | indent 8 }}
    { { - end }}
        spec:
    { { - with .Values.imagePullSecrets }}
          imagePullSecrets:
            { { - toYaml . | nindent 8 }}
    { { - end }}
    { { - if .Values.trivy.serviceAccountName }}
          serviceAccountName: { {  .Values.trivy.serviceAccountName }}
    { { - end }}
          securityContext:
            runAsUser: 0
          automountServiceAccountToken: false
          containers:
            - name: trivy
              image: { {  .Values.trivy.image.repository }}:{ {  .Values.trivy.image.tag }}
              imagePullPolicy: { {  .Values.imagePullPolicy }}
              env:
              { { - if has "trivy" .Values.proxy.components }}
                - name: HTTP_PROXY
                  value: "{ { .Values.proxy.httpProxy }}"
                - name: HTTPS_PROXY
                  value: "{ { .Values.proxy.httpsProxy }}"
                - name: NO_PROXY
                  value: "{ {  template "harbor.noProxy" . }}"
              { { - end }}
                - name: "SCANNER_LOG_LEVEL"
                  value: { {  .Values.logLevel | quote }}
                - name: "SCANNER_TRIVY_CACHE_DIR"
                  value: "/home/scanner/.cache/trivy"
                - name: "SCANNER_TRIVY_REPORTS_DIR"
                  value: "/home/scanner/.cache/reports"
                - name: "SCANNER_TRIVY_DEBUG_MODE"
                  value: { {  .Values.trivy.debugMode | quote }}
                - name: "SCANNER_TRIVY_VULN_TYPE"
                  value: { {  .Values.trivy.vulnType | quote }}
                - name: "SCANNER_TRIVY_GITHUB_TOKEN"
                  valueFrom:
                    secretKeyRef:
                      name: { {  template "harbor.trivy" . }}
                      key: gitHubToken
                - name: "SCANNER_TRIVY_SEVERITY"
                  value: { {  .Values.trivy.severity | quote }}
                - name: "SCANNER_TRIVY_IGNORE_UNFIXED"
                  value: { {  .Values.trivy.ignoreUnfixed | default false | quote }}
                - name: "SCANNER_TRIVY_SKIP_UPDATE"
                  value: { {  .Values.trivy.skipUpdate | default false | quote }}
                - name: "SCANNER_TRIVY_INSECURE"
                  value: { {  .Values.trivy.insecure | default false | quote }}
                - name: SCANNER_API_SERVER_ADDR
                  value: ":{ {  template "harbor.trivy.containerPort" . }}"
                { { - if .Values.internalTLS.enabled }}
                - name: INTERNAL_TLS_ENABLED
                  value: "true"
                - name: SCANNER_API_SERVER_TLS_KEY
                  value: /etc/harbor/ssl/trivy/tls.key
                - name: SCANNER_API_SERVER_TLS_CERTIFICATE
                  value: /etc/harbor/ssl/trivy/tls.crt
                { { - end }}
                - name: "SCANNER_REDIS_URL"
                  valueFrom:
                    secretKeyRef:
                      name: { {  template "harbor.trivy" . }}
                      key: redisURL
                - name: "SCANNER_STORE_REDIS_URL"
                  valueFrom:
                    secretKeyRef:
                      name: { {  template "harbor.trivy" . }}
                      key: redisURL
                - name: "SCANNER_JOB_QUEUE_REDIS_URL"
                  valueFrom:
                    secretKeyRef:
                      name: { {  template "harbor.trivy" . }}
                      key: redisURL
              ports:
                - name: api-server
                  containerPort: { {  template "harbor.trivy.containerPort" . }}
              volumeMounts:
              - name: data
                mountPath: /home/scanner/.cache
                subPath: { {  .Values.persistence.persistentVolumeClaim.trivy.subPath }}
                readOnly: false
              { { - if .Values.internalTLS.enabled }}
              - name: trivy-internal-certs
                mountPath: /etc/harbor/ssl/trivy
              { { - end }}
              { { - if .Values.caBundleSecretName }}
    { {  include "harbor.caBundleVolumeMount" . | indent 10 }}
              { { - end }}
              livenessProbe:
                httpGet:
                  scheme: { {  include "harbor.component.scheme" . | upper }}
                  path: /probe/healthy
                  port: api-server
                initialDelaySeconds: 5
                periodSeconds: 10
                successThreshold: 1
                failureThreshold: 10
              readinessProbe:
                httpGet:
                  scheme: { {  include "harbor.component.scheme" . | upper }}
                  path: /probe/ready
                  port: api-server
                initialDelaySeconds: 5
                periodSeconds: 10
                successThreshold: 1
                failureThreshold: 3
              resources:
    { {  toYaml .Values.trivy.resources | indent 12 }}
          { { - if or (or .Values.internalTLS.enabled .Values.caBundleSecretName) (or (not .Values.persistence.enabled) $trivy.existingClaim) }}
          volumes:
          { { - if .Values.internalTLS.enabled }}
          - name: trivy-internal-certs
            secret:
              secretName: { {  template "harbor.internalTLS.trivy.secretName" . }}
          { { - end }}
          { { - if .Values.caBundleSecretName }}
    { {  include "harbor.caBundleVolume" . | indent 6 }}
          { { - end }}
          { { - if not .Values.persistence.enabled }}
          - name: "data"
            emptyDir: { }
          { { - else if $trivy.existingClaim }}
          - name: "data"
            persistentVolumeClaim:
              claimName: { {  $trivy.existingClaim }}
          { { - end }}
          { { - end }}
        { { - with .Values.trivy.nodeSelector }}
          nodeSelector:
    { {  toYaml . | indent 8 }}
        { { - end }}
        { { - with .Values.trivy.affinity }}
          affinity:
    { {  toYaml . | indent 8 }}
        { { - end }}
        { { - with .Values.trivy.tolerations }}
          tolerations:
    { {  toYaml . | indent 8 }}
        { { - end }}
    { { - if and .Values.persistence.enabled (not $trivy.existingClaim) }}
      volumeClaimTemplates:
      - metadata:
          name: data
          labels:
    { {  include "harbor.labels" . | indent 8 }}
        spec:
          accessModes: [{ {  $trivy.accessMode | quote }}]
          { { - if $trivy.storageClass }}
          { { - if (eq "-" $trivy.storageClass) }}
          storageClassName: ""
          { { - else }}
          storageClassName: "{ { $trivy.storageClass }}"
          { { - end }}
          { { - end }}
          resources:
            requests:
              storage: { {  $trivy.size | quote }}
          selector:
            matchLabels:
    { {  include "harbor.matchLabels" . | indent 10 }}
              component: trivy
    { { - end }}
    { { - end }}

渲染模板，查看上面修改是否存在问题：

helm template --dry-run myharbor harbor/ -n public-service

*  部署：

helm install myharbor harbor/ -n public-service
    
    helm ls -n public-service
    
    NAME    	NAMESPACE     	REVISION	UPDATED                                	STATUS  	CHART       	APP VERSION
    myharbor	public-service	1       	2020-09-29 14:44:54.069561678 +0800 CST	deployed	harbor-1.5.0	2.1.0

kubectl get pvc -n public-service
    
    NAME                                       STATUS   VOLUME               CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    data-myharbor-harbor-redis-0               Bound    harbor-redis         1Gi        RWO                           30s
    data-myharbor-harbor-trivy-0               Bound    harbor-trivy         5Gi        RWO                           30s
    database-data-myharbor-harbor-database-0   Bound    harbor-database      5Gi        RWO                           30s
    myharbor-harbor-chartmuseum                Bound    harbor-chartmuseum   5Gi        RWO                           32s
    myharbor-harbor-jobservice                 Bound    harbor-jobservice    1Gi        RWO                           32s
    myharbor-harbor-registry                   Bound    harbor-registry      10Gi       RWO                           32s
    
    kubectl get pod -n public-service
     
    NAME                                             READY   STATUS    RESTARTS   AGE
    myharbor-harbor-chartmuseum-c85bd9bc9-kvppc      1/1     Running   0          3m21s
    myharbor-harbor-clair-7f5f4885d6-qk9t8           2/2     Running   3          3m21s
    myharbor-harbor-core-56769ffc8d-52n6m            1/1     Running   0          3m22s
    myharbor-harbor-database-0                       1/1     Running   0          3m22s
    myharbor-harbor-jobservice-6c9cb4c87b-8wdvf      1/1     Running   0          3m22s
    myharbor-harbor-notary-server-6f57f9d879-qd57m   1/1     Running   1          3m22s
    myharbor-harbor-notary-signer-6df44c949b-l8vth   1/1     Running   1          3m22s
    myharbor-harbor-portal-75dd5995b9-vgvzh          1/1     Running   0          3m21s
    myharbor-harbor-redis-0                          1/1     Running   0          3m22s
    myharbor-harbor-registry-5f5fd59b9b-npkbv        2/2     Running   0          3m22s
    myharbor-harbor-trivy-0                          1/1     Running   0          3m22s

*  访问：

添加hosts：`harbor.lzxlinux.com`，使用初始账号密码 `admin`/`Harbor12345` 登录即可。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70_pic_center]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70_pic_center 1]

*  docker添加harbor证书：

对于需要使用harbor仓库的节点，都要添加harbor证书。

kubectl get secrets -n public-service myharbor-harbor-ingress -o jsonpath="{.data.ca\.crt}" | base64 --decode

mkdir -p /etc/docker/certs.d/harbor.lzxlinux.com
    
    cat <<EOF > /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt
    -----BEGIN CERTIFICATE-----
    MIIC9TCCAd2gAwIBAgIRAKImYyOICM1GmaHRbS98RMgwDQYJKoZIhvcNAQELBQAw
    FDESMBAGA1UEAxMJaGFyYm9yLWNhMB4XDTIwMDkyOTA3MDA1MVoXDTIxMDkyOTA3
    MDA1MVowFDESMBAGA1UEAxMJaGFyYm9yLWNhMIIBIjANBgkqhkiG9w0BAQEFAAOC
    AQ8AMIIBCgKCAQEAwm0bFysRzMbGiAK31XWiZRhGx76qDLObGtF5xYpINfABkxC7
    VglL66OC+iga2b1+MgLesdDkhW028jsrfPAnyl+S+xEgScytY9DKcRM8Ga72WYAw
    DiKe9CdUhRtZ4o0SSCcmK38167R7R5YGpbuuoTmuBKXr5QE3FQot+yOZywpHHIBF
    jhoz8LrvN2P3LKJlKhCi7mE+WPhbUrkDFo1iuYuaxN0bd9MOZvukNK3WqJh48zE4
    33jcRLbwN79+731MeKE+hRswVEYbl2o6uqCP0cmJA6LFDyxVSCkcYyuo/ANoEw9G
    bUga7IN2zI2vI7iCAxdLrfrECW7l0DeGih+g7wIDAQABo0IwQDAOBgNVHQ8BAf8E
    BAMCAqQwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMA8GA1UdEwEB/wQF
    MAMBAf8wDQYJKoZIhvcNAQELBQADggEBAJANkTXThJebPlhxFLrG6xzN5J/Dgsiw
    06GvLLlKU7ED34RlXaZ6YneQU47Zo67OtiO5NmAfu8Ts3Ak1fcEsmCAGWomL2+Ka
    1zDilaFJA9le7L7wGHdusXt3isWpgebIC3x1lsjV5h3vIz2/9243gHR2Cic3mUnl
    hFNzOChx2GKQASy0wq1Na2/ceoIRLL4vUJmtxmeRgyR7240+Sv9kcPEa2k9/Hp4k
    IDAktVbB7J+mtiUPGQWD/53GZssDqadOlv9ShAZfU6uC9+tCSWgmrQJQqlxh2PZ2
    Z/FPgVfkSR8nCJTohheWaHVLBECBwZHiGyNqiMJnQ4aqr1jMjErgnMw=
    -----END CERTIFICATE-----
    EOF

echo '192.168.30.129 harbor.lzxlinux.com' >> /etc/hosts
    
    docker login harbor.lzxlinux.com -u admin
    
    Password:               # Harbor12345
    WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
    Configure a credential helper to remove this warning. See
    https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    
    Login Succeeded

可以看到登录成功，这种方式不需要重启docker。

*  push插件：

helm plugin install https://github.com/chartmuseum/helm-push
    
    helm plugin ls
    
    NAME	VERSION	DESCRIPTION                      
    push	0.8.1  	Push chart package to ChartMuseum

*  推送chart：

harbor新建项目`public`，

helm repo add myharbor --ca-file /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt https://harbor.lzxlinux.com/chartrepo/public --username=admin --password=Harbor12345
    
    helm repo ls
    
    NAME    	URL                                                   
    stable  	http://mirror.azure.cn/kubernetes/charts              
    aliyun  	https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
    harbor  	https://helm.goharbor.io                              
    myharbor	https://harbor.lzxlinux.com/chartrepo/public

这里的 repo 的地址是 `<Harbor URL>/chartrepo/<项目名称>`，harbor 中每个项目是分开的 repo。

helm push harbor myharbor --ca-file /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70_pic_center 2]

helm部署harbor完成。已存放至个人github：[kubernetes][]

--------------------

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70_pic_center]: /images/20221123/a9e34990155d468ca554cde34201aa99.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70_pic_center 1]: /images/20221123/2a501ef4235a4f3cbc7012821631c388.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70_pic_center 2]: /images/20221123/758728118b1a4eb7911d6e5a24d340b3.png
[kubernetes]: https://github.com/Tobewont/kubernetes