Kubernetes入门（五）

Myth丶恋晨 2022-03-18 14:18 607阅读 0赞

## 30.7 部署Harbor ##

--------------------

Harbor是由VMWare中国团队开源的容器镜像仓库。事实上，harbor是在Docker Registry上进行了相应的企业级扩展，从而获得了更加广泛的应用，这些新的企业级特性包括：管理用户界面，基于角色的访问控制，水平扩展，同步，AD/LDAP集成以及日志审计等。

github地址：[https://github.com/goharbor/harbor/releases][https_github.com_goharbor_harbor_releases]

--------------------

### 手动部署 ###

之前已经搭建了kubernetes集群，其中192.168.30.150作为master2节点，我们把它作为harbor服务器。

*  下载最新的docker-compose二进制文件：

[root@master2 ~]# vim /etc/hosts #添加一行
    54.231.48.160 github-production-release-asset-2e65be.s3.amazonaws.com
    
    [root@master2 ~]# curl -L https://github.com/docker/compose/releases/download/1.24.0-rc1/docker-compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose
    
    [root@master2 ~]# chmod +x /usr/bin/docker-compose

*  下载harbor离线安装包：

[root@master2 ~]# wget https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.0-rc2.tgz
    
    [root@master2 ~]# tar zxf harbor-offline-installer-v1.7.0-rc2.tgz

*  准备CA证书：

[root@master2 ~]# mkdir /data/ && cd /data
    
    上传证书 harbor.lzxlinux.com.key.tar.gz                #自制证书
    
    [root@master2 data]# tar zxf harbor.lzxlinux.com.key.tar.gz
    
    [root@master2 data]# ls cert/
    ca.crt  server.crt  server.key

*  修改配置文件：

[root@master2 cert]# cd harbor/
    
    [root@master2 harbor]# vim harbor.cfg #做下面修改
    hostname = harbor.lzxlinux.com
    ui_url_protocol = https

*  执行安装：

[root@master2 harbor]# sh install.sh

[root@master2 harbor]# docker ps
    CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS                   PORTS                                                              NAMES
    5cad50fe8189        goharbor/nginx-photon:v1.7.0             "nginx -g 'daemon of…"   5 minutes ago       Up 5 minutes (healthy)   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp   nginx
    7f536f3285ff        goharbor/harbor-jobservice:v1.7.0        "/harbor/start.sh"       5 minutes ago       Up 5 minutes                                                                                harbor-jobservice
    f4c2f03e9315        goharbor/harbor-portal:v1.7.0            "nginx -g 'daemon of…"   5 minutes ago       Up 5 minutes (healthy)   80/tcp                                                             harbor-portal
    45755eaf06e3        goharbor/harbor-core:v1.7.0              "/harbor/start.sh"       5 minutes ago       Up 5 minutes (healthy)                                                                      harbor-core
    d1b57df6021c        goharbor/harbor-adminserver:v1.7.0       "/harbor/start.sh"       5 minutes ago       Up 5 minutes (healthy)                                                                      harbor-adminserver
    f65f40cac8cc        goharbor/harbor-db:v1.7.0                "/entrypoint.sh post…"   5 minutes ago       Up 5 minutes (healthy)   5432/tcp                                                           harbor-db
    dd50b19871b8        goharbor/harbor-registryctl:v1.7.0       "/harbor/start.sh"       5 minutes ago       Up 5 minutes (healthy)                                                                      registryctl
    034de27d8a84        goharbor/registry-photon:v2.6.2-v1.7.0   "/entrypoint.sh /etc…"   5 minutes ago       Up 5 minutes (healthy)   5000/tcp                                                           registry
    dd0c1abf428c        goharbor/redis-photon:v1.7.0             "docker-entrypoint.s…"   5 minutes ago       Up 5 minutes             6379/tcp                                                           redis
    97530153c3b7        goharbor/harbor-log:v1.7.0               "/bin/sh -c /usr/loc…"   5 minutes ago       Up 5 minutes (healthy)   127.0.0.1:1514->10514/tcp                                          harbor-log
    0baf1dbfb8d0        ff281650a721                             "/opt/bin/flanneld -…"   2 hours ago         Up 2 hours                                                                                  k8s_kube-flannel_kube-flannel-ds-amd64-lfgv5_kube-system_f40a3fe8-34d6-11e9-83ed-000c297ff3a2_0
    96ce131643a0        mirrorgooglecontainers/pause-amd64:3.1   "/pause"                 2 hours ago         Up 2 hours                                                                                  k8s_POD_kube-flannel-ds-amd64-lfgv5_kube-system_f40a3fe8-34d6-11e9-83ed-000c297ff3a2_0

*  浏览器访问：

编辑Windows上的hosts文件，路径：`C:\Windows\System32\drivers\etc\hosts`，增加一行：192.168.30.150 [harbor.lzxlinux.com][] 。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70]

用户名：admin ，密码：Harbor12345（默认密码）

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 1]

*  新建项目：

一般项目不设为公开

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 2]

*  拉取公共镜像：

[root@master2 harbor]# docker pull busybox
    
    [root@master2 harbor]# docker pull tomcat

*  打标签：

[root@master2 harbor]# docker tag busybox harbor.lzxlinux.com/lzx/busybox
    
    [root@master2 harbor]# echo 192.168.30.150 harbor.lzxlinux.com >> /etc/hosts
    
    [root@master2 harbor]# docker login https://harbor.lzxlinux.com
    Username: admin
    Password: 
    Error response from daemon: Get https://harbor.lzxlinux.com/v2/: x509: certificate signed by unknown authority

*  解决509问题：

[root@master2 harbor]# echo -n | openssl s_client -showcerts -connect harbor.lzxlinux.com:443 2>/dev/null |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/P' >> /etc/ssl/certs/ca-bundle.trust.crt 
    
    [root@master2 harbor]# systemctl restart docker
    
    [root@master2 harbor]# docker-compose up -d # -d 后台启动

如果上面方法还是不行，可以这样操作

# mkdir -p /etc/docker/certs.d/harbor.lzxlinux.com
    
    # cp /data/cert/ca.crt /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt
    
    # chmod 400 !$
    
    # systemctl restart docker

*  再次登录：

[root@master2 harbor]# docker login https://harbor.lzxlinux.com
    Username: admin
    Password: 
    WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
    Configure a credential helper to remove this warning. See
    https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    
    Login Succeeded

*  推送busybox镜像：

[root@master2 harbor]# docker push harbor.lzxlinux.com/lzx/busybox
    The push refers to repository [harbor.lzxlinux.com/lzx/busybox]
    683f499823be: Pushed 
    latest: digest: sha256:bbb143159af9eabdf45511fd5aab4fd2475d4c0e7fd4a5e154b98e838488e510 size: 527

*  浏览器查看：

刷新一下，可以查看到新传上来的镜像。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 3]

*  推送tomcat镜像：

[root@master2 harbor]# docker tag tomcat harbor.lzxlinux.com/lzx/tomcat
    
    [root@master2 harbor]# docker push harbor.lzxlinux.com/lzx/tomcat

再次刷新查看

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 4]

--------------------

## 30.8 在Kubernetes中使用harbor ##

--------------------

废了很大劲部署harbor，那它有什么作用呢？接下来在kubernetes中使用它。

--------------------

以下操作在master节点中执行。

*  创建secret：

[root@master ~]# kubectl create secret docker-registry my-secret --docker-server=harbor.lzxlinux.com --docker-username=admin --docker-password=Harbor12345
    
    [root@master ~]# kubectl get secret
    NAME                  TYPE                                  DATA      AGE
    default-token-lfx98   kubernetes.io/service-account-token   3         1d
    my-secret             kubernetes.io/dockerconfigjson        1         38s                   # secret可创建多个，在拉取镜像时指定

*  推送httpd镜像：

[root@master ~]# docker pull httpd
    
    [root@master ~]# docker tag httpd harbor.lzxlinux.com/lzx/httpd
    
    [root@master ~]# docker login https://harbor.lzxlinux.com
    
    [root@master ~]# docker push harbor.lzxlinux.com/lzx/httpd

刷新查看

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 5]

*  定义一个pod：

[root@master ~]# vim httpd-pod.yaml

apiVersion: v1
    kind: Pod
    metadata:
      name: httpd-pod
    spec:
      containers:
      - image: harbor.lzxlinux.com/lzx/httpd:latest
        name: httpd-pod
      imagePullSecrets:
      - name: my-secret                     #引用my-secret

[root@master ~]# kubectl create -f httpd-pod.yaml
    
    [root@master ~]# kubectl get pod
    NAME                     READY     STATUS             RESTARTS   AGE
    httpd-pod                0/1       ImagePullBackOff   0          53s                #状态不对
    nginx-6f858d4d45-wwnm9   1/1       Running            0          4h
    
    [root@master ~]# kubectl describe pod httpd-pod
      Warning  Failed     25s (x2 over 53s)  kubelet, 192.168.30.130  Error: ErrImagePull
      Normal   BackOff    11s (x2 over 52s)  kubelet, 192.168.30.130  Back-off pulling image "harbor.lzxlinux.com/lzx/httpd:latest"
      Warning  Failed     11s (x2 over 52s)  kubelet, 192.168.30.130  Error: ImagePullBackOff
      Normal   Pulling    0s (x3 over 1m)    kubelet, 192.168.30.130  pulling image "harbor.lzxlinux.com/lzx/httpd:latest"

*  解决问题：

[root@master2 harbor]# scp /data/cert/ca.crt 192.168.30.129:/root/
    
    [root@master2 harbor]# scp /data/cert/ca.crt 192.168.30.130:/root/

[root@node1 ~]# mkdir -p /etc/docker/certs.d/harbor.lzxlinux.com
    
    [root@node1 ~]# cp ca.crt /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt
    
    [root@node1 ~]# chmod 400 !$
    
    [root@node1 ~]# systemctl restart docker
    
    [root@node1 ~]# vim /etc/hosts #增加一行
    192.168.30.150 harbor.lzxlinux.com

[root@node2 ~]# mkdir -p /etc/docker/certs.d/harbor.lzxlinux.com
    
    [root@node2 ~]# cp ca.crt /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt
    
    [root@node2 ~]# chmod 400 !$
    
    [root@node2 ~]# systemctl restart docker
    
    [root@node2 ~]# vim /etc/hosts #增加一行
    192.168.30.150 harbor.lzxlinux.com

*  继续创建pod：

[root@master ~]# kubectl delete pod httpd-pod 
    
    [root@master ~]# kubectl create -f httpd-pod.yaml
    
    [root@master ~]# kubectl describe pod httpd-pod
      Normal  Scheduled  <invalid>  default-scheduler        Successfully assigned default/httpd-pod to 192.168.30.130
      Normal  Pulling    <invalid>  kubelet, 192.168.30.130  pulling image "harbor.lzxlinux.com/lzx/httpd:latest"
      Normal  Pulled     <invalid>  kubelet, 192.168.30.130  Successfully pulled image "harbor.lzxlinux.com/lzx/httpd:latest"
      Normal  Created    <invalid>  kubelet, 192.168.30.130  Created container
      Normal  Started    <invalid>  kubelet, 192.168.30.130  Started container
      
    [root@master ~]# kubectl get pod
    NAME                     READY     STATUS             RESTARTS   AGE
    httpd-pod                1/1       Running            0          3m
    nginx-6f858d4d45-wwnm9   1/1       Running            0          4h

这次成功创建，正在运行中。刷新浏览器，可以看到下载次数已更新。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 6]

--------------------

免费证书申请：[https://freessl.cn/][https_freessl.cn]

harbor`443: connect: connection refused`问题解决：[https://blog.csdn.net/miss1181248983/article/details/88661825][https_blog.csdn.net_miss1181248983_article_details_88661825]

[https_github.com_goharbor_harbor_releases]: https://github.com/goharbor/harbor/releases
[harbor.lzxlinux.com]: http://harbor.lzxlinux.com
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70]: /images/20220318/85a635949eba40a1aa0a5ad98f18a065.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 1]: /images/20220318/481bbcb8544e4c22bacb97e34a6d8bcc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 2]: /images/20220318/55f644b9855c48329a5ccbf58960c180.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 3]: /images/20220318/75c6bcb24f2f426184f3c73646950867.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 4]: /images/20220318/c676a17a84b342e3b0d2a86efc6d6f4b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 5]: /images/20220318/4bdbb99d7da54640b8dd80d5609fd448.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 6]: /images/20220318/b62339825f25452facbb5fe8bfc0a0da.png
[https_freessl.cn]: https://note.youdao.com/
[https_blog.csdn.net_miss1181248983_article_details_88661825]: https://blog.csdn.net/miss1181248983/article/details/88661825