darknet-训练自己的yolov3模型

川长思鸟来 2022-02-19 07:59 316阅读 0赞

*  [Yolo v3的使用方法][Yolo v3]
    
     *  [安装darknet][darknet]
     *  [训练Pascal VOC格式的数据][Pascal VOC]
        
         *  [修改cfg文件中的voc.data][cfg_voc.data]
         *  [修改VOC.names][VOC.names]
         *  [下载预训练卷积层权重][Link 1]
         *  [修改cfg/yolov3-voc.cfg][cfg_yolov3-voc.cfg]
     *  [训练自己的模型][Link 2]
     *  [测试Yolo模型][Yolo]
        
         *  [测试单张图片：][Link 3]
     *  [生成预测结果][Link 4]
     *  [采用第三方compute\_mAP][compute_mAP]
     *  [Reference][]

# Yolo v3的使用方法 #

参考自@[zhaonan][]

## 安装darknet ##

*  下载库文件

git clone https://github.com/pjreddie/darknet
    cd darknet

*  修改Makefile

GPU=1  #0或1
    CUDNN=1  #0或1
    OPENCV=0  #0或1
    OPENMP=0
    DEBUG=0

*  编译

make

*  下载预训练模型

wget https://pjreddie.com/media/files/yolov3.weights

*  用预训练模型进行简单的测试

./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg

## 训练Pascal VOC格式的数据 ##

*  生成Labels，因为darknet不需要xml文件，需要.txt文件（格式： ）

用voc\_label.py（位于./scripts）`cat voc_label.py` 共修改四处

import xml.etree.ElementTree as ET
    import pickle
    import os
    from os import listdir, getcwd
    from os.path import join
    sets=[('2007', 'train'), ('2007', 'val'), ('2007', 'test')]  #替换为自己的数据集
    classes = ["head", "eye", "nose"]     #修改为自己的类别
    
    def convert(size, box):
        dw = 1./(size[0])
        dh = 1./(size[1])
        x = (box[0] + box[1])/2.0 - 1
        y = (box[2] + box[3])/2.0 - 1
        w = box[1] - box[0]
        h = box[3] - box[2]
        x = x*dw
        w = w*dw
        y = y*dh
        h = h*dh
        return (x,y,w,h)
    def convert_annotation(year, image_id):
        in_file = open('VOCdevkit/VOC%s/Annotations/%s.xml'%(year, image_id))  #将数据集放于当前目录下
        out_file = open('VOCdevkit/VOC%s/labels/%s.txt'%(year, image_id), 'w')
        tree=ET.parse(in_file)
        root = tree.getroot()
        size = root.find('size')
        w = int(size.find('width').text)
        h = int(size.find('height').text)
        for obj in root.iter('object'):
            difficult = obj.find('difficult').text
            cls = obj.find('name').text
            if cls not in classes or int(difficult)==1:
                continue
            cls_id = classes.index(cls)
            xmlbox = obj.find('bndbox')
            b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text), float(xmlbox.find('ymax').text))
            bb = convert((w,h), b)
            out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')
    wd = getcwd()
    for year, image_set in sets:
        if not os.path.exists('VOCdevkit/VOC%s/labels/'%(year)):
            os.makedirs('VOCdevkit/VOC%s/labels/'%(year))
        image_ids = open('VOCdevkit/VOC%s/ImageSets/Main/%s.txt'%(year, image_set)).read().strip().split()
        list_file = open('%s_%s.txt'%(year, image_set), 'w')
        for image_id in image_ids:
            list_file.write('%s/VOCdevkit/VOC%s/JPEGImages/%s.jpg\n'%(wd, year, image_id))
            convert_annotation(year, image_id)
        list_file.close()   
    os.system("cat 2007_train.txt 2007_val.txt > train.txt")     #修改为自己的数据集用作训练

wget https://pjreddie.com/media/files/voc_label.py
    python voc_label.py

在`VOCdevkit/VOC2007/labels/`中：

>     learner@learner-pc:~/darknet/scripts$ ls
>     2007_test.txt #0   dice_label.sh        imagenet_label.sh  VOCdevkit_original
>     2007_train.txt #1  gen_tactic.sh        train.txt #3        voc_label.py
>     2007_val.txt #2 get_coco_dataset.sh  VOCdevkit

这时darknet需要一个txt文件，其中包含了所有的图片

cat 2007_train.txt 2007_val.txt 2012_*.txt > train.txt

### 修改cfg文件中的voc.data ###

classes= 3    #修改为自己的类别数
    train  = /home/learner/darknet/data/voc/train.txt   #修改为自己的路径 or /home/learner/darknet/scripts/2007_test.txt
    valid  = /home/learner/darknet/data/voc/2007_test.txt   #修改为自己的路径 or /home/learner/darknet/scripts/2007_test.txt
    names = /home/learner/darknet/data/voc.names  #修改见voc.names
    backup = /home/learner/darknet/backup   #修改为自己的路径，输出的权重信息将存储其内

### 修改VOC.names ###

head  #自己需要探测的类别，一行一个
    eye
    nose

### 下载预训练卷积层权重 ###

wget https://pjreddie.com/media/files/darknet53.conv.74

### 修改cfg/yolov3-voc.cfg ###

[net]
    # Testing
     batch=64
     subdivisions=32   #每批训练的个数=batch/subvisions，根据自己GPU显存进行修改，显存不够改大一些
    # Training
    # batch=64
    # subdivisions=16
    width=416
    height=416
    channels=3
    momentum=0.9
    decay=0.0005
    angle=0
    saturation = 1.5
    exposure = 1.5
    hue=.1
    
    learning_rate=0.001
    burn_in=1000
    max_batches = 50200  #训练步数
    policy=steps
    steps=40000,45000  #开始衰减的步数
    scales=.1,.1
    
    
    
    [convolutional]
    batch_normalize=1
    filters=32
    size=3
    stride=1
    pad=1
    activation=leaky
    
    .....
    
    [convolutional]
    size=1
    stride=1
    pad=1
    filters=24   #filters = 3 * ( classes + 5 )   here,filters=3*(3+5)
    activation=linear
    
    [yolo]
    mask = 6,7,8
    anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
    classes=3    #修改为自己的类别数
    num=9
    jitter=.3
    ignore_thresh = .5
    truth_thresh = 1
    random=1
    
    [route]
    layers = -4
    
    [convolutional]
    batch_normalize=1
    filters=256
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [upsample]
    stride=2
    
    [route]
    layers = -1, 61
    
    
    
    [convolutional]
    batch_normalize=1
    filters=256
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    size=3
    stride=1
    pad=1
    filters=512
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    filters=256
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    size=3
    stride=1
    pad=1
    filters=512
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    filters=256
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    size=3
    stride=1
    pad=1
    filters=512
    activation=leaky
    
    [convolutional]
    size=1
    stride=1
    pad=1
    filters=24    #filters = 3 * ( classes + 5 )   here,filters=3*(3+5)
    activation=linear
    
    [yolo]
    mask = 3,4,5
    anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
    classes=3  #修改为自己的类别数
    num=9
    jitter=.3
    ignore_thresh = .5
    truth_thresh = 1
    random=1
    
    [route]
    layers = -4
    
    [convolutional]
    batch_normalize=1
    filters=128
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [upsample]
    stride=2
    
    [route]
    layers = -1, 36
    
    
    
    [convolutional]
    batch_normalize=1
    filters=128
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    size=3
    stride=1
    pad=1
    filters=256
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    filters=128
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    size=3
    stride=1
    pad=1
    filters=256
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    filters=128
    size=1
    stride=1
    pad=1
    activation=leaky
    
    [convolutional]
    batch_normalize=1
    size=3
    stride=1
    pad=1
    filters=256
    activation=leaky
    
    [convolutional]
    size=1
    stride=1
    pad=1
    filters=24    #filters = 3 * ( classes + 5 )   here,filters=3*(3+5)
    activation=linear
    
    [yolo]
    mask = 0,1,2
    anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
    classes=3   #修改为自己的类别数
    num=9
    jitter=.3
    ignore_thresh = .5
    truth_thresh = 1
    random=1

## 训练自己的模型 ##

1 单GPU训练：`./darknet -i <gpu_id> detector train <data_cfg> <train_cfg> <weights>`

./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74

2 多GPU训练，格式为`0,1,2,3`：`./darknet detector train <data_cfg> <model_cfg> <weights> -gpus <gpu_list>`

./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74 -gpus 0,1,2,3

## 测试Yolo模型 ##

### 测试单张图片： ###

*  测试单张图片，需要编译时有OpenCV支持：`./darknet detector test <data_cfg> <test_cfg> <weights> <image_file> #本次测试无opencv支持`
 *  `<test_cfg>`文件中`batch`和`subdivisions`两项必须为1。
 *  测试时还可以用`-thresh`和`-hier`选项指定对应参数。
 *  `./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights Eminem.jpg`
    
    ### 批量测试图片 ###
    
     *  `yolov3-voc.cfg(cfg文件夹下)`文件中`batch`和`subdivisions`两项必须为1。
     *  在detector.c中增加头文件：
        
        >     #include <unistd.h>  /* Many POSIX functions (but not all, by a large margin) */
        >     #include <fcntl.h>   /* open(), creat() - and fcntl() */
 *  在前面添加*GetFilename(char* p)函数
    
        #include "darknet.h"
        #include <sys/stat.h>  //需增加的头文件
        #include<stdio.h>
        #include<time.h>
        #include<sys/types.h>  //需增加的头文件
        static int coco_ids[] = {1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,67,70,72,73,74,75,76,77,78,79,80,81,82,84,85,86,87,88,89,90};
        
        char *GetFilename(char *p)
        { 
            static char name[30]={""};
            char *q = strrchr(p,'/') + 1;
            strncpy(name,q,20);
            return name;
        }
 *  用下面代码替换detector.c文件（example文件夹下）的**void test\_detector**函数（注意有3处要改成自己的路径）

void test_detector(char *datacfg, char *cfgfile, char *weightfile, char *filename, float thresh, float hier_thresh, char *outfile, int fullscreen)
    {
        list *options = read_data_cfg(datacfg);
        char *name_list = option_find_str(options, "names", "data/names.list");
        char **names = get_labels(name_list);
     
        image **alphabet = load_alphabet();
        network *net = load_network(cfgfile, weightfile, 0);
        set_batch_network(net, 1);
        srand(2222222);
        double time;
        char buff[256];
        char *input = buff;
        float nms=.45;
        int i=0;
        while(1){
            if(filename){
                strncpy(input, filename, 256);
                image im = load_image_color(input,0,0);
                image sized = letterbox_image(im, net->w, net->h);
            //image sized = resize_image(im, net->w, net->h);
            //image sized2 = resize_max(im, net->w);
            //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
            //resize_network(net, sized.w, sized.h);
                layer l = net->layers[net->n-1];
     
     
                float *X = sized.data;
                time=what_time_is_it_now();
                network_predict(net, X);
                printf("%s: Predicted in %f seconds.\n", input, what_time_is_it_now()-time);
                int nboxes = 0;
                detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
                //printf("%d\n", nboxes);
                //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
                if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
                    draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
                    free_detections(dets, nboxes);
                if(outfile)
                 {
                    save_image(im, outfile);
                 }
                else{
                    save_image(im, "predictions");
    #ifdef OPENCV
                    cvNamedWindow("predictions", CV_WINDOW_NORMAL); 
                    if(fullscreen){
                    cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
                    }
                    show_image(im, "predictions",0);
                    cvWaitKey(0);
                    cvDestroyAllWindows();
    #endif
                }
                free_image(im);
                free_image(sized);
                if (filename) break;
             } 
            else {
                printf("Enter Image Path: ");
                fflush(stdout);
                input = fgets(input, 256, stdin);
                if(!input) return;
                strtok(input, "\n");
       
                list *plist = get_paths(input);
                char **paths = (char **)list_to_array(plist);
                 printf("Start Testing!\n");
                int m = plist->size;
                if(access("/home/learner/darknet/data/outv3tiny_dpj",0)==-1)//"/home/learner/darknet/data"修改成自己的路径
                {
                  if (mkdir("/home/learner/darknet/data/outv3tiny_dpj",0777))//"/home/learner/darknet/data"修改成自己的路径
                   {
                     printf("creat file bag failed!!!");
                   }
                }
                for(i = 0; i < m; ++i){
                 char *path = paths[i];
                 image im = load_image_color(path,0,0);
                 image sized = letterbox_image(im, net->w, net->h);
            //image sized = resize_image(im, net->w, net->h);
            //image sized2 = resize_max(im, net->w);
            //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
            //resize_network(net, sized.w, sized.h);
            layer l = net->layers[net->n-1];
     
     
            float *X = sized.data;
            time=what_time_is_it_now();
            network_predict(net, X);
            printf("Try Very Hard:");
            printf("%s: Predicted in %f seconds.\n", path, what_time_is_it_now()-time);
            int nboxes = 0;
            detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
            //printf("%d\n", nboxes);
            //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
            if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
            draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
            free_detections(dets, nboxes);
            if(outfile){
                save_image(im, outfile);
            }
            else{
                 
                 char b[2048];
                sprintf(b,"/home/learner/darknet/data/outv3tiny_dpj/%s",GetFilename(path));//"/home/leaner/darknet/data"修改成自己的路径
                
                save_image(im, b);
                printf("save %s successfully!\n",GetFilename(path));
    /*
    #ifdef OPENCV
                //cvNamedWindow("predictions", CV_WINDOW_NORMAL); 
                if(fullscreen){
                    cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
                }
                //show_image(im, "predictions");
                //cvWaitKey(0);
                //cvDestroyAllWindows();
    #endif*/
            }
     
            free_image(im);
            free_image(sized);
            if (filename) break;
            }
          }
        }
    }

*  重新进行编译

make clean
    make

*  开始批量测试

./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights

*  输入Image Path(所有的测试文件的路径，可以复制voc.data中valid后边的路径):

/home/learner/darknet/data/voc/2007_test.txt # 完整路径

*  结果都保存在`./data/out`文件夹下

## 生成预测结果 ##

生成预测结果

*  `./darknet detector valid <data_cfg> <test_cfg> <weights> <out_file>`
 *  yolov3-voc.cfg(cfg文件夹下)`文件中`batch`和`subdivisions两项必须为1。
 *  结果生成在`<data_cfg>`的`results`指定的目录下以`<out_file>`开头的若干文件中，若`<data_cfg>`没有指定`results`，那么默认为`<darknet_root>/results`。
 *  执行语句如下：在终端只返回用时，在./results/comp4\_det\_test\_\[类名\].txt里保存测试结果

./darknet detector valid cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights

## 采用第三方compute\_mAP ##

下载第三方库：

git clone https://github.com/LianjiLi/yolo-compute-map.git

进行如下修改:

*  修改darknet/examples/detector.c中validate\_detector()
    
        char *valid_images = option_find_str(options, "valid", "./data/2007_test.txt");//改成自己的测试文件路径
        
        if(!outfile) outfile = "comp4_det_test_";
                fps = calloc(classes, sizeof(FILE *));
                for(j = 0; j < classes; ++j){
                    snprintf(buff, 1024, "%s/%s.txt", prefix, names[j]);//删除outfile参数以及对应的%s
                    fps[j] = fopen(buff, "w");
 *  重新编译
    
        make clean
        make
 *  运行valid
    
        darknet文件夹下运行./darknet detector valid cfg/voc.data cfg/yolov3-tiny.cfg backup/yolov3-tiny_164000.weights（改为自己的模型路径）
 *  在本文件夹下运行`python compute_mAP.py`
 *  说明：compute\_mAP.py中的test.txt文件内容只有文件名字，不带绝对路径，不带后缀

## Reference ##

[YOLOv3目标检测总结][YOLOv3]

[官方网站][Link 5]

[思路整理自@zhaonan][zhaonan 1]

代码改变世界

分类: [深度学习 专栏][Link 6]

标签: [经验][Link 7], [yolov3][], [配置][Link 8], [map][], [test][]

好文要顶 关注我 收藏该文 ![icon_weibo_24.png][] ![wechat.png][]

[![20170903155022.png][]][20170903155022.png 1]

[pprp][20170903155022.png 1]  
[关注 - 69][- 69]  
[粉丝 - 21][- 21]

\+加关注

[« ][Link 9] 上一篇：[CycleGAN 配置及其实现][Link 9]  
[» ][Link 10] 下一篇：[(转载)YOLO配置文件理解][Link 10]

posted @ 2018-08-23 18:22 [pprp][] 阅读(7246) 评论(0) [编辑][Link 11] [收藏][Link 12]

[Yolo v3]: https://www.cnblogs.com/pprp/p/9525508.html#yolo-v3%E7%9A%84%E4%BD%BF%E7%94%A8%E6%96%B9%E6%B3%95
[darknet]: https://www.cnblogs.com/pprp/p/9525508.html#%E5%AE%89%E8%A3%85darknet
[Pascal VOC]: https://www.cnblogs.com/pprp/p/9525508.html#%E8%AE%AD%E7%BB%83pascal-voc%E6%A0%BC%E5%BC%8F%E7%9A%84%E6%95%B0%E6%8D%AE
[cfg_voc.data]: https://www.cnblogs.com/pprp/p/9525508.html#%E4%BF%AE%E6%94%B9cfg%E6%96%87%E4%BB%B6%E4%B8%AD%E7%9A%84voc.data
[VOC.names]: https://www.cnblogs.com/pprp/p/9525508.html#%E4%BF%AE%E6%94%B9voc.names
[Link 1]: https://www.cnblogs.com/pprp/p/9525508.html#%E4%B8%8B%E8%BD%BD%E9%A2%84%E8%AE%AD%E7%BB%83%E5%8D%B7%E7%A7%AF%E5%B1%82%E6%9D%83%E9%87%8D
[cfg_yolov3-voc.cfg]: https://www.cnblogs.com/pprp/p/9525508.html#%E4%BF%AE%E6%94%B9cfgyolov3-voc.cfg
[Link 2]: https://www.cnblogs.com/pprp/p/9525508.html#%E8%AE%AD%E7%BB%83%E8%87%AA%E5%B7%B1%E7%9A%84%E6%A8%A1%E5%9E%8B
[Yolo]: https://www.cnblogs.com/pprp/p/9525508.html#%E6%B5%8B%E8%AF%95yolo%E6%A8%A1%E5%9E%8B
[Link 3]: https://www.cnblogs.com/pprp/p/9525508.html#%E6%B5%8B%E8%AF%95%E5%8D%95%E5%BC%A0%E5%9B%BE%E7%89%87
[Link 4]: https://www.cnblogs.com/pprp/p/9525508.html#%E7%94%9F%E6%88%90%E9%A2%84%E6%B5%8B%E7%BB%93%E6%9E%9C
[compute_mAP]: https://www.cnblogs.com/pprp/p/9525508.html#%E9%87%87%E7%94%A8%E7%AC%AC%E4%B8%89%E6%96%B9compute_map
[Reference]: https://www.cnblogs.com/pprp/p/9525508.html#reference
[zhaonan]: http://www.cnblogs.com/nanzhao
[YOLOv3]: https://blog.csdn.net/uncle_ll/article/details/80830112
[Link 5]: https://pjreddie.com/darknet/install/
[zhaonan 1]: https://www.cnblogs.com/nanzhao/p/Sailon.html
[Link 6]: https://www.cnblogs.com/pprp/category/1285583.html
[Link 7]: https://www.cnblogs.com/pprp/tag/%E7%BB%8F%E9%AA%8C/
[yolov3]: https://www.cnblogs.com/pprp/tag/yolov3/
[Link 8]: https://www.cnblogs.com/pprp/tag/%E9%85%8D%E7%BD%AE/
[map]: https://www.cnblogs.com/pprp/tag/map/
[test]: https://www.cnblogs.com/pprp/tag/test/
[icon_weibo_24.png]: https://common.cnblogs.com/images/icon_weibo_24.png
[wechat.png]: /images/20220219/e5ee8b711e644ec4993491d8b6568416.png
[20170903155022.png]: /images/20220219/ff6a91ffc0894612bc1724d9161715cc.png
[20170903155022.png 1]: http://home.cnblogs.com/u/pprp/
[- 69]: http://home.cnblogs.com/u/pprp/followees
[- 21]: http://home.cnblogs.com/u/pprp/followers
[Link 9]: https://www.cnblogs.com/pprp/p/9522265.html
[Link 10]: https://www.cnblogs.com/pprp/p/9525719.html
[pprp]: https://www.cnblogs.com/pprp/
[Link 11]: https://i.cnblogs.com/EditPosts.aspx?postid=9525508
[Link 12]: https://www.cnblogs.com/pprp/p/9525508.html#