Docker + Gitlab + Gitlab CI（三）

「爱情、让人受尽委屈。」 2023-05-29 13:30 10阅读 0赞

本文通过Docker + Gitlab + Gitlab CI实现容器的自动化部署。

环境如下：

<table> 
 <thead> 
  <tr> 
   <th align="center">ip</th> 
   <th align="center">role</th> 
   <th align="center">domain</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td align="center">192.168.1.51</td> 
   <td align="center">prod manager</td> 
   <td align="center">none</td> 
  </tr> 
  <tr> 
   <td align="center">192.168.1.52</td> 
   <td align="center">prod worker1</td> 
   <td align="center">none</td> 
  </tr> 
  <tr> 
   <td align="center">192.168.1.53</td> 
   <td align="center">prod worker2</td> 
   <td align="center">none</td> 
  </tr> 
  <tr> 
   <td align="center">~</td> 
   <td align="center">~</td> 
   <td align="center">~</td> 
  </tr> 
  <tr> 
   <td align="center">192.168.1.54</td> 
   <td align="center">dev manager</td> 
   <td align="center">none</td> 
  </tr> 
  <tr> 
   <td align="center">192.168.1.55</td> 
   <td align="center">dns/dev worker1</td> 
   <td align="center">none</td> 
  </tr> 
  <tr> 
   <td align="center">192.168.1.56</td> 
   <td align="center">dev worker2</td> 
   <td align="center">none</td> 
  </tr> 
  <tr> 
   <td align="center">~</td> 
   <td align="center">~</td> 
   <td align="center">~</td> 
  </tr> 
  <tr> 
   <td align="center">192.168.1.57</td> 
   <td align="center">gitlab server</td> 
   <td align="center">gitlab.lzxlinux.cn</td> 
  </tr> 
  <tr> 
   <td align="center">192.168.1.59</td> 
   <td align="center">harbor server</td> 
   <td align="center">harbor.lzxlinux.cn</td> 
  </tr> 
 </tbody> 
</table>

> 注意：为了方便后面实验，建议自定义的域名不要与外部域名冲突，否则在连接外网情况下容易解析错误。

--------------------

### 环境准备 ###

*  全部进行准备工作：

# systemctl stop firewalld && systemctl disable firewalld
    
    # setenforce 0 && sed -i 's/=enforcing/=disabled/g' /etc/selinux/config
    
    # yum install -y epel-release vim lrzsz net-tools gcc rsync wget unzip curl git bash-completion && yum update -y && source /etc/profile

*  全部安装docker：

# curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker.repo
    
    # yum makecache fast
    
    # yum install -y docker-ce
    
    # systemctl start docker && systemctl enable docker

提高docker pull速度

# curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
    
    # systemctl restart docker

*  全部配置登录harbor：

# vim /usr/lib/systemd/system/docker.service
    ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry=harbor.lzxlinux.cn
    
    # systemctl daemon-reload && systemctl restart docker.service

*  全部配置dns：

# vim /etc/resolv.conf #增加一行，且放在公网dns前面
    
    nameserver 192.168.1.55             #该ip为dns容器所在主机ip
    nameserver 223.5.5.5

*  所有节点注册runner：

不只是manager节点需要注册runner，其余节点（包括harbor服务器）也需要注册runner，gitlab服务器不需要注册runner。

runner是shell类型，下面以192.1681.1.54注册runner为例。

# echo '54.153.54.194 packages.gitlab.com' >> /etc/hosts
    
    # curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.rpm.sh | sudo bash
    
    # yum install -y gitlab-ci-multi-runner
    
    # gitlab-ci-multi-runner status
    gitlab-runner: Service is running!
    
    # usermod -aG docker gitlab-runner
    
    # systemctl restart docker
    
    # gitlab-ci-multi-runner restart
    
    # mkdir -p /etc/gitlab/ssl
    
    # scp root@gitlab.lzxlinux.cn:/etc/gitlab/ssl/gitlab.lzxlinux.cn.crt /etc/gitlab/ssl
    
    # gitlab-ci-multi-runner register \
      --tls-ca-file=/etc/gitlab/ssl/gitlab.lzxlinux.cn.crt \
      --url "https://gitlab.lzxlinux.cn/" \
      --registration-token "4kr9ZmLMWasYxqB2tSzQ" \
      --name "dev-runner" \
      --tag-list "dev" \
      --run-untagged="false" \
      --locked="false" \
      --executor "shell"
      
    # gitlab-ci-multi-runner list
    Listing configured runners                          ConfigFile=/etc/gitlab-runner/config.toml
    dev-runner                                          Executor=shell Token=uD6SbCHWc-AKp94eprRF URL=https://gitlab.lzxlinux.cn/

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 1]

--------------------

### 搭建Swarm集群 ###

生产环境和测试环境各搭建一个swarm集群。

生产环境的swarm集群成员包括192.168.1.51、192.168.1.52、192.168.1.53，其中192.168.1.51为manager；  
测试环境的swarm集群成员包括192.168.1.54、192.168.1.55、192.168.1.56，其中192.168.1.54为manager。

[手动搭建swarm集群][swarm]

[ansible自动搭建swarm集群][ansible_swarm]

--------------------

### Flask项目部署 ###

Flask实现Ansible和Ansible-Playbook的配置+部署系统，同时带有简单的服务器管理系统和认证系统。

下面分别在两个manager节点部署这个flask项目，该项目包含两个服务：mysql和flask-ansible。

在192.168.1.54这个manager节点部署演示，192.168.1.51上部署与之一致。

*  创建新的群组及项目：

新建一个群组`Flask`，同时新建一个项目`flask-ansible`，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 2]

*  创建secret：

# echo '123456789' > mysql_password
    
    # docker secret create -l mysql_root_password password mysql_password
    
    # rm -f mysql_password

*  克隆项目：

# cd /software
    
    # mkdir flask && cd flask
    
    # git -c http.sslVerify=false clone https://gitlab.lzxlinux.cn/flask/flask-ansible.git

*  修改代码：

# vim flask-ansible/config.yml
    
    mysql:
      host: db
      port: 3306
      username: lzxlinux
      password: lzxlinuxpwd
      database: devops
    
    # vim flask-ansible/app.py #增加ip及端口
    
    if __name__ == '__main__':
        app.run(
            host='0.0.0.0',
            port=5000
        )
    
    # vim flask-ansible/Flask-Ansible.sql #修改账号及密码
    
    insert into user (username,password) values ('lzxlinux', md5('lzxlinuxpwd'));

*  构建`flask-ansible`镜像：

# vim Dockerfile
    
    FROM python:3.7
    LABEL author="LZX <admin@lzxlinux.cn>"
    
    COPY flask-ansible/ /flask-ansible
    WORKDIR /flask-ansible
    RUN pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
    EXPOSE 5000
    ENTRYPOINT ["python3", "app.py"]

# docker build -t harbor.lzxlinux.cn/flask/flask-ansible .

*  推送镜像到harbor仓库：

# docker login harbor.lzxlinux.cn
    
    # docker push harbor.lzxlinux.cn/flask/flask-ansible

集群内非manager节点登录harbor仓库，方便稍后部署项目。

# docker login harbor.lzxlinux.cn

*  部署项目：

# vim docker-compose.yml

version: "3.7"
    
    services:
      db:
        image: mysql:5.7.28
        secrets:
          - password
        environment:
          MYSQL_ROOT_PASSWORD_FILE: /run/secrets/password
          MYSQL_DATABASE: devops
        volumes:
          - db_data:/var/lib/mysql
        networks:
          - flask
        deploy:
          mode: global
          placement:
            constraints:
              - node.role == manager
    
      web:
        depends_on:
          - db
        image: harbor.lzxlinux.cn/flask/flask-ansible:latest
        ports:
          - "5000:5000"
        secrets:
          - password
        environment:
          FLASK_DB_HOST: db:3306
          FLASK_DB_PASSWORD_FILE: /run/secrets/password
        networks:
          - flask
        deploy:
          mode: replicated
          replicas: 3
          restart_policy:
            condition: on-failure
            delay: 5s
            max_attempts: 3
          update_config:
            parallelism: 3
            delay: 0s
            order: stop-first
            
    volumes:
      db_data:
    
    networks:
      flask:
        driver: overlay
    
    secrets:
      password:
        external: true

# docker stack deploy flask-ansible --compose-file docker-compose.yml

*  导入数据库：

# docker service ls
    ID                  NAME                MODE                REPLICAS            IMAGE                                           PORTS
    w6uc32889ttv        flask-ansible_db    global              1/1                 mysql:5.7.28                                    
    oemioctfvq2v        flask-ansible_web   replicated          3/3                 harbor.lzxlinux.cn/flask/flask-ansible:latest   *:5000->5000/tcp
    
    # docker cp /software/flask/flask-ansible/Flask-Ansible.sql flask-ansible_db.dtp13no8veakdiz8bs1o0xt0f.kw8sq43i2eg9qm8h0yabuw7ia:/var/lib/mysql/
    
    # docker exec -it flask-ansible_db.dtp13no8veakdiz8bs1o0xt0f.kw8sq43i2eg9qm8h0yabuw7ia bash
    
    root@11024228712d:/# mysql -uroot -p
    Enter password:
    
    mysql> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | devops             |
    | mysql              |
    | performance_schema |
    | sys                |
    +--------------------+
    5 rows in set (0.00 sec)
    
    mysql> grant all privileges on *.* to 'lzxlinux'@'%' identified by 'lzxlinuxpwd';
    Query OK, 0 rows affected, 1 warning (0.01 sec)
    
    mysql> flush privileges;
    Query OK, 0 rows affected (0.00 sec)
    
    mysql> set names utf8;
    
    mysql> source /var/lib/mysql/Flask-Ansible.sql;
    Database changed
    Query OK, 0 rows affected (0.01 sec)
    
    Query OK, 1 row affected (0.01 sec)
    
    Query OK, 0 rows affected (0.01 sec)
    
    Query OK, 1 row affected (0.00 sec)
    
    Query OK, 0 rows affected (0.00 sec)
    
    Query OK, 1 row affected (0.00 sec)
    
    Query OK, 0 rows affected (0.03 sec)
    
    Query OK, 1 row affected (0.00 sec)

*  查看部署：

打开浏览器，访问192.168.1.54这个swarm集群任一节点的5000端口，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 3]

使用账号`lzxlinux`和密码`lzxlinuxpwd`登录，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 4]

可以看到登录成功，查看首页，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 5]

项目部署成功。在192.168.1.51上重复上面步骤部署该项目。

--------------------

### Flask项目的服务更新 ###

下面实现上面部署的flask项目中服务版本的自动发布。

*  gitlab修改代码：

根据之前的配置在gitlab上修改该项目的代码，分别是三处：`config.yml`，`app.py`，`Flask-Ansible.sql`。

*  gitlab设置：

`设置` → `仓库` → `Protected Branches`，设置`No one` `Allow to push`到master分支，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 6]

*  gitlab设置变量：

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 7]

*  harbor设置权限：

harbor服务器上执行，

# visudo
    
    gitlab-runner   ALL=(root) NOPASSWD: /usr/bin/rm

*  创建新的dev分支：

克隆代码到本地主机，并修改代码。

# cd /home
    
    # git config --global user.name "admin"
    
    # git config --global user.email "admin@lzxlinux.cn"
    
    # git -c http.sslVerify=false clone https://gitlab.lzxlinux.cn/flask/flask-ansible.git
    
    # cd flask-ansible/
    
    # git checkout dev
    
    # vim Dockerfile

FROM python:3.7
    LABEL author="LZX <admin@lzxlinux.cn>"
    
    COPY . /flask-ansible
    WORKDIR /flask-ansible
    RUN pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
    EXPOSE 5000
    ENTRYPOINT ["python3", "app.py"]

# vim .gitlab-ci.yml

stages:
      - login
      - delete
      - deploy
      - release
    
    harbor-login1:
      stage: login
      script:
        - docker login harbor.lzxlinux.cn -u $HARBOR_USER -p $HARBOR_PASS
        - docker image prune -f
        - docker rm `docker ps -aq --filter status=exited` || docker ps -a
      tags:
        - dev
      only:
        - master
    
    harbor-login2:
      stage: login
      script:
        - docker login harbor.lzxlinux.cn -u $HARBOR_USER -p $HARBOR_PASS
        - docker image prune -f
        - docker rm `docker ps -aq --filter status=exited` || docker ps -a
      tags:
        - dev_worker1
      only:
        - master
    
    harbor-login3:
      stage: login
      script:
        - docker login harbor.lzxlinux.cn -u $HARBOR_USER -p $HARBOR_PASS
        - docker image prune -f
        - docker rm `docker ps -aq --filter status=exited` || docker ps -a
      tags:
        - demo
      only:
        - master
    
    harbor-login4:
      stage: login
      script:
        - docker login harbor.lzxlinux.cn -u $HARBOR_USER -p $HARBOR_PASS
        - docker image prune -f
        - docker rm `docker ps -aq --filter status=exited` || docker ps -a
      tags:
        - prod
      only:
        - master
    
    harbor-login5:
      stage: login
      script:
        - docker login harbor.lzxlinux.cn -u $HARBOR_USER -p $HARBOR_PASS
        - docker image prune -f
        - docker rm `docker ps -aq --filter status=exited` || docker ps -a
      tags:
        - prod_worker1
      only:
        - master
    
    harbor-login6:
      stage: login
      script:
        - docker login harbor.lzxlinux.cn -u $HARBOR_USER -p $HARBOR_PASS
        - docker image prune -f
        - docker rm `docker ps -aq --filter status=exited` || docker ps -a
      tags:
        - prod_worker2
      only:
        - master
    
    harbor-delete:
      stage: delete
      script:
        - sudo rm -rf /data/registry/docker/registry/v2/repositories/flask/flask-ansible/_manifests/tags/latest/
      tags:
        - harbor
      only:
        - master
    
    docker-test:
      stage: deploy
      script:
        - docker build -t harbor.lzxlinux.cn/flask/flask-ansible:latest .
        - docker push harbor.lzxlinux.cn/flask/flask-ansible:latest
        - docker service update flask-ansible_web --image harbor.lzxlinux.cn/flask/flask-ansible:latest --update-delay 0s --update-parallelism 1
      tags:
        - dev
      only:
        - master
        
    docker-release:
      stage: release
      script:
        - docker build -t harbor.lzxlinux.cn/flask/flask-ansible:$CI_COMMIT_TAG .
        - docker push harbor.lzxlinux.cn/flask/flask-ansible:$CI_COMMIT_TAG
        - docker service update flask-ansible_web --image harbor.lzxlinux.cn/flask/flask-ansible:$CI_COMMIT_TAG --update-delay 0s --update-parallelism 1
        - docker image prune -f
        - docker rm `docker ps -aq --filter status=exited` || docker ps -a
        - docker images -aq --filter reference=harbor.lzxlinux.cn/flask/flask-ansible |grep -v `docker images -aq --filter reference=harbor.lzxlinux.cn/flask/flask-ansible:$CI_COMMIT_TAG` | xargs docker rmi -f
      tags:
        - prod
      only:
        - tags

# vim templates/index.html

{% extends "base.html" %}
    {% block head %}
        <title>实践哥运维开发</title>
    {% endblock %}
    
    {% block body %}
        <div class="container">
            <div class="row">
                <div class="col"></div>
                <div class="col">
                    <div class="jumbotron">
                        <p>DOCKER GITLAB CI</p>
                    </div>
                </div>
                <div class="col"></div>
            </div>
        </div>
    {% endblock %}

修改完代码之后，应该先在本地进行代码检查，同时本地运行flask项目，看是否存在问题，这样可以在merge到master分支前避免低级错误。

*  提交代码：

在本地检查没问题后，提交到gitlab。因为没有代码风格和单元测试检查，所以直接发起merge请求，

# git add .
    
    # git commit -m 'Update .gitlab-ci.yml'
    
    # git -c http.sslVerify=false push origin dev

`合并请求` → `创建合并请求`，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 8]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 9]

登录harbor仓库报错：

Error response from daemon: Get http://harbor.lzxlinux.cn/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

解决：

harbor服务器上执行

# systemctl restart docker.service
    
    # cd /software/harbor
    
    # docker-compose down -v
    
    # docker-compose up -d

重试任务，等待全部完成，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 10]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 11]

# docker service ls
    ID                  NAME                MODE                REPLICAS            IMAGE                                           PORTS
    w6uc32889ttv        flask-ansible_db    global              1/1                 mysql:5.7.28                                    
    oemioctfvq2v        flask-ansible_web   replicated          3/3                 harbor.lzxlinux.cn/flask/flask-ansible:latest   *:5000->5000/tcp
    
    # docker ps -a
    CONTAINER ID        IMAGE                                           COMMAND                  CREATED             STATUS                        PORTS                 NAMES
    fd71f2dd02d9        harbor.lzxlinux.cn/flask/flask-ansible:latest   "python3 app.py"         11 minutes ago      Up 10 minutes                 80/tcp                flask-ansible_web.2.p2ajoi6hv40elvapgldr3d19l
    3c3350118615        harbor.lzxlinux.cn/flask/flask-ansible:latest   "python3 app.py"         About an hour ago   Exited (137) 10 minutes ago                         flask-ansible_web.2.f5fc7enksy4n3nv5oz54pwkaj
    852e01c736e8        mysql:5.7.28                                    "docker-entrypoint.s…"   About an hour ago   Up About an hour              3306/tcp, 33060/tcp   flask-ansible_db.dtp13no8veakdiz8bs1o0xt0f.c7xvgj95n1fp6m74t5h89znn9

查看harbor界面，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 12]

查看flask首页，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 13]

可以看到，首页显示的是更改过的内容，项目在测试环境的服务更新没有问题。

--------------------

### Flask项目的服务版本自动发布 ###

接下来在生产环境进行服务版本的自动更新。

*  创建新的tag`v1.0.0`：

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 14]

`CI/CD` → `流水线`，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 15]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 16]

# docker service ls
    ID                  NAME                MODE                REPLICAS            IMAGE                                           PORTS
    tbexs27u8e0o        flask-ansible_db    global              1/1                 mysql:5.7.28                                    
    r659vsorxwlu        flask-ansible_web   replicated          3/3                 harbor.lzxlinux.cn/flask/flask-ansible:v1.0.0   *:5000->5000/tcp
    
    # docker ps -a
    CONTAINER ID        IMAGE                                           COMMAND                  CREATED             STATUS              PORTS                 NAMES
    93b736bdb243        harbor.lzxlinux.cn/flask/flask-ansible:v1.0.0   "python3 app.py"         2 minutes ago       Up 2 minutes        5000/tcp              flask-ansible_web.1.kw7kdpudwwrienqlhjy3ctrqu
    8faca076407f        mysql:5.7.28                                    "docker-entrypoint.s…"   13 minutes ago      Up 13 minutes       3306/tcp, 33060/tcp   flask-ansible_db.pk7kxaba6deljmow1gsjlr3tn.l2zcgh85kmlkj98067md8p34h
    
    # docker images |grep flask-ansible
    harbor.lzxlinux.cn/flask/flask-ansible   v1.0.0              534acf106bb9        4 minutes ago       1.06GB

查看harbor界面，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 17]

查看flask首页，

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 18]

可以看到，flask项目在生产环境下的服务版本的自动发布没有问题，实现了多容器的服务更新。

--------------------

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20191106175250647.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20191106175301930.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[swarm]: https://blog.csdn.net/miss1181248983/article/details/102747741
[ansible_swarm]: https://blog.csdn.net/miss1181248983/article/details/102798071
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20191106175421617.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20191106175544913.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20191106175554701.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20191106175609667.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20191106175647175.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20191106175702697.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20191106175744405.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 9]: https://img-blog.csdnimg.cn/2019110617575983.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 10]: https://img-blog.csdnimg.cn/20191106175813753.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 11]: https://img-blog.csdnimg.cn/20191106175824944.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 12]: https://img-blog.csdnimg.cn/20191106175841716.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 13]: https://img-blog.csdnimg.cn/201911061758576.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 14]: https://img-blog.csdnimg.cn/20191106175913218.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 15]: https://img-blog.csdnimg.cn/20191106175923665.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 16]: https://img-blog.csdnimg.cn/20191106175939690.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 17]: https://img-blog.csdnimg.cn/20191106175953182.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 18]: https://img-blog.csdnimg.cn/20191106180003204.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz,size_16,color_FFFFFF,t_70