检测标注数据量

不念不忘少年蓝@ 2021-11-14 12:32 250阅读 0赞

[EfficientDet][]** phi=1系统模型在2007测试集上的结果**如果要问需要标注多少检测数据就可以了，得到的答复多半是越多越好。可是不可能无限制的标注数据，也不可能只通过数据就能达到预期效果，毕竟数据和算法才最终决定效果(算力直接决定了做事效率)。

既然没有统一答案，但是从公开的数据集中还是能获得些启发的。本文以VOC中的数据为例，探讨标注数据量的问题。首先看下各主流检测算法在该voc2007测试数据集上的效果。

![20190731202304351.png][] 图片来自这里

训练voc2007和voc2012数据，在voc2007验证集上的map如下：

<table style="width:500px;"> 
 <tbody> 
 <tr> 
 <td>vgg16</td> 
 <td>res101</td> 
 </tr> 
 <tr> 
 <td>AP for aeroplane = 0.7842 AP for bicycle = 0.7931 AP for bird = 0.7588 AP for boat = 0.6525 AP for bottle = 0.6580 AP for bus = 0.8383 AP for car = 0.8788 AP for cat = 0.8665 AP for chair = 0.5870 AP for cow = 0.8266 AP for diningtable = 0.6996 AP for dog = 0.8295 AP for horse = 0.8715 AP for motorbike = 0.7896 AP for person = 0.7919 AP for pottedplant = 0.4859 AP for sheep = 0.7571 AP for sofa = 0.7235 AP for train = 0.8082 AP for tvmonitor = 0.7744 Mean AP = 0.7588</td> 
 <td>AP for aeroplane = 0.7894 AP for bicycle = 0.8308 AP for bird = 0.8232 AP for boat = 0.7213 AP for bottle = 0.7019 AP for bus = 0.8763 AP for car = 0.8801 AP for cat = 0.8857 AP for chair = 0.6598 AP for cow = 0.8709 AP for diningtable = 0.7369 AP for dog = 0.8714 AP for horse = 0.8695 AP for motorbike = 0.8031 AP for person = 0.7969 AP for pottedplant = 0.5318 AP for sheep = 0.8176 AP for sofa = 0.7948 AP for train = 0.8286 AP for tvmonitor = 0.8073 Mean AP = 0.7949</td> 
 </tr> 
 </tbody> 
</table>

voc0712训练验证集和coco中car,bus,truck,motorbike,bicycle共4k+数据一起作为训练集，voc07test上的map如下：

<table style="width:500px;"> 
 <tbody> 
 <tr> 
 <td>res101</td> 
 </tr> 
 <tr> 
 <td> AP for aeroplane = 0.8064 AP for bicycle = 0.8564 AP for bird = 0.8047 AP for boat = 0.7157 AP for bottle = 0.7057 AP for bus = 0.8712 AP for car = 0.8802 AP for cat = 0.8810 AP for chair = 0.6625 AP for cow = 0.8583 AP for diningtable = 0.7194 AP for dog = 0.8795 AP for horse = 0.8767 AP for motorbike = 0.8379 AP for person = 0.8321 AP for pottedplant = 0.5407 AP for sheep = 0.8137 AP for sofa = 0.7871 AP for train = 0.8671 AP for tvmonitor = 0.8171 Mean AP =&nbsp;0.80067 </td> 
 </tr> 
 </tbody> 
</table>

[EfficientDet][]** phi=1系统模型在2007测试集上的结果**

285 instances of class aeroplane with average precision: 0.8487
    337 instances of class bicycle with average precision: 0.8951
    459 instances of class bird with average precision: 0.8310
    263 instances of class boat with average precision: 0.7230
    469 instances of class bottle with average precision: 0.6772
    213 instances of class bus with average precision: 0.8426
    1201 instances of class car with average precision: 0.8787
    358 instances of class cat with average precision: 0.9232
    756 instances of class chair with average precision: 0.5684
    244 instances of class cow with average precision: 0.8510
    206 instances of class diningtable with average precision: 0.6899
    489 instances of class dog with average precision: 0.9124
    348 instances of class horse with average precision: 0.9028
    325 instances of class motorbike with average precision: 0.8460
    4528 instances of class person with average precision: 0.8495
    480 instances of class pottedplant with average precision: 0.5424
    242 instances of class sheep with average precision: 0.8608
    239 instances of class sofa with average precision: 0.7362
    282 instances of class train with average precision: 0.8671
    308 instances of class tvmonitor with average precision: 0.8116
    mAP: 0.8029

**phi=2自训练模型在2007测试集上的结果**

num_fp=483690.0, num_tp=11510.0
    285 instances of class aeroplane with average precision: 0.7630
    337 instances of class bicycle with average precision: 0.8456
    459 instances of class bird with average precision: 0.8028
    263 instances of class boat with average precision: 0.6754
    469 instances of class bottle with average precision: 0.6569
    213 instances of class bus with average precision: 0.8307
    1201 instances of class car with average precision: 0.8782
    358 instances of class cat with average precision: 0.9030
    756 instances of class chair with average precision: 0.5199
    244 instances of class cow with average precision: 0.7975
    206 instances of class diningtable with average precision: 0.7031
    489 instances of class dog with average precision: 0.8738
    348 instances of class horse with average precision: 0.8775
    325 instances of class motorbike with average precision: 0.8017
    4528 instances of class person with average precision: 0.8183
    480 instances of class pottedplant with average precision: 0.5057
    242 instances of class sheep with average precision: 0.7623
    239 instances of class sofa with average precision: 0.7102
    282 instances of class train with average precision: 0.8241
    308 instances of class tvmonitor with average precision: 0.7913
    mAP: 0.7670

从数据集中查看car,bus,person,bicycle,motorbike的物体数及其图片数目。

<table style="width:644px;"> 
 <tbody> 
 <tr> 
 <td style="text-align:center;vertical-align:middle;width:277px;">roi数目</td> 
 <td style="text-align:center;vertical-align:middle;width:364px;">图片数目</td> 
 </tr> 
 <tr> 
 <td style="width:277px;">merge_bicycle 1255=418+837 merge_bus 957=272+685 merge_car 4136=1644+2492 merge_motorbike 1191=390+801 merge_person 22948=5447+17401</td> 
 <td style="width:364px;">类别 &nbsp; &nbsp;图片数=单个目标+多个目标 &nbsp;（2007+2012） bicycle &nbsp;858=639+219 &nbsp;(179+76 + 460+143) bus &nbsp; &nbsp; &nbsp;664=475+189 &nbsp;(149+48 + 326+141) car &nbsp; &nbsp; &nbsp;2045=1174+871 &nbsp;(412+349 + 762+522) motorbike 824=613+211 &nbsp; (179+70 + 434+141) person &nbsp; &nbsp;11679=7057+4622 (953+1143 + 6104+3479)</td> 
 </tr> 
 </tbody> 
</table>

针对上述5类物体训练模型的结果,voc07test中相关类别共2795张图片作为测试集的结果。

<table style="width:500px;"> 
 <tbody> 
 <tr> 
 <td> voc07+voc12 5类物体 trainval数据结果11k step，res101模型效果 </td> 
 <td> voc07+voc12 20类物体 trainval数据结果11k step，res101模型效果&nbsp; </td> 
 <td> voc07+voc12 5类物体 + coco 4k+数据训练11k step,res101模型结果 </td> 
 </tr> 
 <tr> 
 <td>AP for car = 0.7842 AP for bus = 0.7696 AP for bicycle = 0.7682 AP for motorbike = 0.7506 AP for person = 0.7836</td> 
 <td>AP for car = 0.7872 AP for bus = 0.7862 AP for bicycle = 0.7915 AP for motorbike = 0.7723 AP for person = 0.7798</td> 
 <td>AP for car = 0.7973 AP for bus = 0.7814 AP for bicycle = 0.7622 AP for motorbike = 0.7746 AP for person = 0.7838</td> 
 </tr> 
 </tbody> 
</table>

当然，VOC数据集中为自然场景下的检测，样本数据当然是越多越好。如果针对特定场景下的检测，样本数据则不需要很多，标注数据量的多少需要参考实际使用场景。

参考文献：

1.  [YOLO9000: Better, Faster, Stronger][YOLO9000_ Better_ Faster_ Stronger]
2.  [The PASCAL Visual Object Classes Homepage][The_PASCAL_Visual Object Classes Homepage]

[EfficientDet]: https://github.com/xuannianz/EfficientDet
[20190731202304351.png]: /images/20211114/cf3a82f592cd46d8a045314b781253f9.png
[YOLO9000_ Better_ Faster_ Stronger]: https://arxiv.org/abs/1612.08242
[The_PASCAL_Visual Object Classes Homepage]: http://host.robots.ox.ac.uk/pascal/VOC/