【多目标跟踪】FairMOT复现 (CVPR2020)

超、凢脫俗 2023-02-18 08:20 429阅读 0赞

代码：[https://github.com/ifzhang/FairMOT][https_github.com_ifzhang_FairMOT]

论文：[https://arxiv.org/pdf/2004.01888.pdf][https_arxiv.org_pdf_2004.01888.pdf]

我使用版本的代码：[https://download.csdn.net/download/qq\_35975447/13089984][https_download.csdn.net_download_qq_35975447_13089984]

**目录**

0.环境

1.下载

2.demo.py

3.训练单个数据集

4.在mot16上测试各种指标

参考

--------------------

## 0.环境 ##

ubuntu16.04
    cuda9.0
    python3.6 #(python3.5 is imcompatible)
    torch==1.1.0
    torchvision==0.3.0
    Cython #(install first)
    yacs
    opencv-python
    cython-bbox
    scipy
    numba
    progress
    motmetrics
    numba
    matplotlib
    lap
    openpyxl
    Pillow==6.2.2
    tensorboardX
    tensorboard==1.14

[https://download.pytorch.org/whl/cu90/torch\_stable.html][https_download.pytorch.org_whl_cu90_torch_stable.html]

torch与torchvision在上面的链接中下载。

apt-get install -y libbz2-dev
    apt update && apt install -y libsm6 libxext6
    apt-get install -y libxrender1

pip --default-time=500 install -i https://pypi.tuna.tsinghua.edu.cn/simple torch-1.1.0-cp36-cp36m-linux_x86_64.whl 
    pip --default-time=500 install -i https://pypi.tuna.tsinghua.edu.cn/simple torchvision-0.3.0-cp36-cp36m-manylinux1_x86_64.whl
    pip --default-time=500 install -i https://pypi.tuna.tsinghua.edu.cn/simple Cython
    pip --default-time=500 install -i https://pypi.tuna.tsinghua.edu.cn/simple llvmlite==0.31.0
    pip --default-time=500 install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt

cd src/lib/models/networks/DCNv2_new 
    sh make.sh

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70][]

安装ffmpeg：[https://blog.csdn.net/lwgkzl/article/details/77836207][https_blog.csdn.net_lwgkzl_article_details_77836207]

## 1.下载 ##

下载[FairMOT(ifzhang)][FairMOT_ifzhang]，与模型。（此处官方使用模型已经更新，建议下载最新版本与最新代码，再参考博客）

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 1][]

将下载的模型复制到指定的目录下。

[https://download.csdn.net/download/qq\_35975447/12529701][https_download.csdn.net_download_qq_35975447_12529701]

[https://download.csdn.net/download/qq\_35975447/12529718][https_download.csdn.net_download_qq_35975447_12529718]

## 2.demo.py ##

运行demo.py GPU>=1159MiB ，就够了。TITAN xp上测试1500帧大概花费时间221s，所以平均一帧花费147ms。

cd src
    python demo.py mot --load_model ../models/all_dla34.pth --conf_thres 0.4
    
    #指定GPU
    CUDA_VISIBLE_DEVICES=1 python demo.py mot --load_model ../models/all_dla34.pth --conf_thres 0.4

这个过程中会下载一个模型([http://dl.yf.io/dla/models/imagenet/dla34-ba72cf86.pth][http_dl.yf.io_dla_models_imagenet_dla34-ba72cf86.pth])，可能会下载超时。

自己下载的话，需要复制到对应的路径下。

cp dla34-ba72cf86.pth /root/.cache/torch/checkpoints/

运行成功后，终端显示：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 2][]

结果都保存在result文件夹下。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 3][]

需要安装ffmpeg才会生成result.mp4，Ubuntu安装ffmpeg，请参考[https://blog.csdn.net/lwgkzl/article/details/77836207][https_blog.csdn.net_lwgkzl_article_details_77836207]。安装后，再次运行终端会输出显示：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 4][]

好吧，我发现这个方法，在行人之间有大面积重合时，ID也会切换即跟丢了（难点啊难点）。

![20200618215617114.gif][]

注：如果想要测试自己的数据的话，通过修改/src/lib/opts.py中的下面路径（此处添加于2021/02/19）：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 5][]

## 3.训练单个数据集 ##

参考[https://blog.csdn.net/sinat\_33486980/article/details/105611295][https_blog.csdn.net_sinat_33486980_article_details_105611295]，发现其数据格式是该代码数据格式，直接下载[CUHK-SYSU][]，将名称改为.\\FairMOT-master\\src\\data\\cuhksysu.train中的名称一致。在FairMOT-master根目录下创建\\data\\MOT\\JDE，然后将下载的CUHKSYSU整个文件夹复制到\\data\\MOT\\JDE目录下，并修改\\FairMOT-master\\src\\lib\\cfg\\data.json，由于json语法无注释，所以直接删掉除CUHKSYSU以外不用的数据集，并修改“root”绝对路径。如下：

data.json
    
    {
        "root":"/home/FairMOT-master/data/MOT/JDE",
        "train":
        {
            "cuhksysu":"./data/cuhksysu.train"
        },
        "test_emb":
        {
            "mot15":"./data/mot15.val"
        },
        "test":
        {
            "mot15":"./data/mot15.val"
        }
    }

下载预训练模型与baseline模型：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 6][]

修改\\FairMOT-master\\experiments\\all\_dla34.sh如下：

cd src
    python train.py mot --exp_id all_dla34 --gpus 0,1 --batch_size 12 --load_model '../models/ctdet_coco_dla_2x.pth' --num_workers 0
    cd ..

然后在FairMOT-master根目录下运行：

sh experiments/all_dla34.sh

训练开始：

![20200623104015950.png][]

![20200623104059994.png][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 7][]

查看终端输出发现，结果输出在以下目录中：

![20200623104917340.png][]

在对应目录下运行就可以可视化损失函数的情况了：

tensorboard --logdir ./ --port=ip端口

![20200623105850953.png][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 8][]

## 4.在mot16上测试各种指标 ##

参考[https://github.com/Zhongdao/Towards-Realtime-MOT/blob/master/DATASET\_ZOO.md][https_github.com_Zhongdao_Towards-Realtime-MOT_blob_master_DATASET_ZOO.md]，下载mot16数据。

然后将MOT16数据集文件夹放到FairMOT-master/src/data/MOT16/目录下。运行下面命令：

CUDA_VISIBLE_DEVICES=1 python track.py mot --val_mot16 True --load_model ../models/all_dla34.pth --conf_thres 0.6 --save_all

![20200628153654453.png][]

如果想要保存video，需要修改track.py文件：

line 235:        
     save_videos=True)

如果修改以上，ffmpeg保存mp4出现问题的话，修改track.py line 132:

cmd_str = 'ffmpeg -f image2 -i {}/%05d.jpg -b 5000k -c:v mpeg4 {}'.format(output_dir, output_video_path)

保存视频时会有动态显示，最后保存好后，显示如下，如果不是这样建议自行找原因：

![20200707111732817.png][]

测试至少需要1.8-1.9G显存大小。

## 参考 ##

[1.FairMOT（ifzhang）][https_github.com_ifzhang_FairMOT]

[2.Ubuntu16.04下安装FFmpeg（超简单版）][https_blog.csdn.net_lwgkzl_article_details_77836207]

[3.CVPR 2020 多目标跟踪算法 FairMOT代码demo运行及训练][https_blog.csdn.net_sinat_33486980_article_details_105611295]

[4.Towards-Realtime-MOT（数据下载地址）][https_github.com_Zhongdao_Towards-Realtime-MOT_blob_master_DATASET_ZOO.md]

[https_github.com_ifzhang_FairMOT]: https://github.com/ifzhang/FairMOT
[https_arxiv.org_pdf_2004.01888.pdf]: https://arxiv.org/pdf/2004.01888.pdf
[https_download.csdn.net_download_qq_35975447_13089984]: https://download.csdn.net/download/qq_35975447/13089984
[https_download.pytorch.org_whl_cu90_torch_stable.html]: https://download.pytorch.org/whl/cu90/torch_stable.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200616181747445.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[https_blog.csdn.net_lwgkzl_article_details_77836207]: https://blog.csdn.net/lwgkzl/article/details/77836207
[FairMOT_ifzhang]: https://github.com/ifzhang/FairMOT.git
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200616182014495.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[https_download.csdn.net_download_qq_35975447_12529701]: https://download.csdn.net/download/qq_35975447/12529701
[https_download.csdn.net_download_qq_35975447_12529718]: https://download.csdn.net/download/qq_35975447/12529718
[http_dl.yf.io_dla_models_imagenet_dla34-ba72cf86.pth]: http://dl.yf.io/dla/models/imagenet/dla34-ba72cf86.pth
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20200617142412241.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20200617142800818.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20200618144151540.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[20200618215617114.gif]: https://img-blog.csdnimg.cn/20200618215617114.gif
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20210219115525952.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[https_blog.csdn.net_sinat_33486980_article_details_105611295]: https://blog.csdn.net/sinat_33486980/article/details/105611295
[CUHK-SYSU]: https://drive.google.com/file/d/1D7VL43kIV9uJrdSCYl53j89RE2K-IoQA/view?usp=sharing
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20200623102908316.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[20200623104015950.png]: https://img-blog.csdnimg.cn/20200623104015950.png
[20200623104059994.png]: https://img-blog.csdnimg.cn/20200623104059994.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20200623104343228.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[20200623104917340.png]: https://img-blog.csdnimg.cn/20200623104917340.png
[20200623105850953.png]: https://img-blog.csdnimg.cn/20200623105850953.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20200623105911887.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[https_github.com_Zhongdao_Towards-Realtime-MOT_blob_master_DATASET_ZOO.md]: https://github.com/Zhongdao/Towards-Realtime-MOT/blob/master/DATASET_ZOO.md
[20200628153654453.png]: https://img-blog.csdnimg.cn/20200628153654453.png
[20200707111732817.png]: https://img-blog.csdnimg.cn/20200707111732817.png