airflow部署

小灰灰 2022-09-01 09:09 137阅读 0赞

## 本地部署 ##

`创建目录并添加环境变量`

mkdir /opt/airflow
    echo "export AIRFLOW_HOME=/opt/airflow" >> /etc/profile

`安装`

# pip安装报错，改为pip3
    pip3 install apache-airflow

[报错参考pip安装失败解决方案][pip]

`初始化db`

airflow db init

`创建用户`

airflow users create \
        --username admin \
        --firstname Peter \
        --lastname Parker \
        --role Admin \
        --email spiderman@superhero.org

`启动web服务器`

airflow webserver --port 8080

`启动调度器`

airflow scheduler

`查看web页面`  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI4Njg2OTEx_size_16_color_FFFFFF_t_70]

## docker启动airflow ##

`查看当前docker-compose版本（需要>v1.27.0）`

[root@yx03 airflow-docker]# docker-compose -v
    docker-compose version 1.24.1, build 4667896b

`下载最新版本docker-compose`

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

`创建docker映射目录，下载docker-compose.yaml`

mkdir /opt/airflow-docker
    cd /opt/airflow-docker
    mkdir ./dags ./logs ./plugins
    echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env
    curl -LfO 'https://airflow.apache.org/docs/apache-airflow/2.1.2/docker-compose.yaml'

`初始化`

docker-compose up airflow-init

![在这里插入图片描述][faa28a6a25bf401e98ad46b0d9b524c4.png]  
`启动`

docker-compose up

`默认登陆账号密码`

The default account has the login `airflow` and the password `airflow`

[pip]: https://blog.csdn.net/qq_28686911/article/details/113243777
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI4Njg2OTEx_size_16_color_FFFFFF_t_70]: /images/20220829/50d41b7f981342ac8e208bd05761b085.png
[faa28a6a25bf401e98ad46b0d9b524c4.png]: /images/20220829/799f9d206d1a4c36877717667913bbfb.png