Efficient Object Localization Using Convolutional Networks 论文解读

一时失言乱红尘 2021-11-19 15:08 309阅读 0赞

**paper title**: Efficient Object Localization Using Convolutional Networks  
**paper link**: https://arxiv.org/pdf/1411.4280.pdf  
**oral or demo video**：-  
**project**: -  
**github**: -  
**conf & anthor**: CVPR15, Jonathan Tompson et al.  
**arXiv submit v1**: 2014.11 google citation:405(2019.07.12)

姿态估计系列：

*  [DeepPose - human pose estimation via deep nerual networks][]
 *  [Efficient Object Localization Using Convolutional Networks][]
 *  [Convolutional Pose Machines ][Convolutional Pose Machines]
 *  [Human Pose Estimation with Iterative Error Feedback][]
 *  [Hourglass Network - stacked hourglass network for human pose estimation ][Hourglass Network - stacked hourglass network for human pose estimation]
 *  [OpenPose - Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields][]
 *  [RMPE: Regional Multi-person Pose Estimation][RMPE_ Regional Multi-person Pose Estimation]
 *  [Pose flow: Efficient Online Pose Tracking][Pose flow_ Efficient Online Pose Tracking]
 *  [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark][CrowdPose_ Efficient Crowded Scenes Pose Estimation and A New Benchmark]

**主要内容**：  
这篇文章主要使用coarse和fine两个级联网络来回归人体关节点的heat-map,并使用shared-feature的架构联合训练这两个网络，提升模型的泛化能力。这篇文章的可以看做是文章[Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation][]的拓展，可参考。

**网络结构**：  
文章提出的网络结构是一个级联的heat-map回归网络，分为coarse和fine回归两部分，coarse部分回归一个粗略的heat-map关节点位置，然后使用coarse网络得到的结果在coarse得到的feature map上截取一块区域，然后输入fine网络得到fine回归的结果，最后将coarse和fine回归网络得到的结果整合后得到最终的结果。  
级联网络的整体架构如下：  
![overview of cascade model architecture][]

(1) coarse heat-map regression model  
coarse网络是一个多分辨率输入网络，输入三层高斯金字塔图像，经过各自的卷积操作之后得到的feature map进行concat，然后进行一个SpatialDropout的操作（即feature map层面的dropout,原因是作者发现使用常规的dropout会使训练时间变长，并仍会导致over-training），最后经过卷积操作之后输出14x32x32的heat maps。  
![coarse heat-map regression model][]

(2) fine heat-map regression model  
fine网络是一个Siamese network，单个关节的网络结构图如下，由于coarse网络的输入有多个尺度，所以在feature map截取输入到fine网络也有多个尺度，所以这里的fine网络也有多个分支，对不同的尺度的输入进行处理，最后所有的feature map变换到同一尺度（对尺寸较小的feature map进行上采样）再concat，然后再进行后续的卷积操作。  
![fine regression network for a single joint][]

**损失函数**:  
(1) coarse model  
coarse模型使用MSE作为损失函数：  
 L c o a r s e = 1 / N ∗ ∑ j = 1 N ∑ x y ∣ ∣ H j ′ ( x , y ) − H j ( x , y ) ∣ ∣ 2 L\_\{coarse\}=1/N\*\\sum\_\{j=1\}^N\\sum\_\{xy\}||H'\_j(x,y)-Hj(x,y)||^2 Lcoarse=1/N∗j=1∑Nxy∑∣∣Hj′(x,y)−Hj(x,y)∣∣2  
其中 H j ′ H'\_j Hj′和 H j H\_j Hj分别表示预测和ground truth的第j个关节的heat map。

(2) fine model  
fine模型也使用MSE作为损失函数：  
 L f i n e = 1 / N ∗ ∑ j = 1 N ∑ x y ∣ ∣ G j ′ ( x , y ) − G j ( x , y ) ∣ ∣ 2 L\_\{fine\}=1/N\*\\sum\_\{j=1\}^N\\sum\_\{xy\}||G'\_j(x,y)-G\_j(x,y)||^2 Lfine=1/N∗j=1∑Nxy∑∣∣Gj′(x,y)−Gj(x,y)∣∣2  
其中 G j ′ G'\_j Gj′和 G j G\_j Gj分别表示预测和ground truth的第j个关节的heat map。

(3)联合训练时优化:  
 L = L c o a r s e + λ ∗ L f i n e L=L\_\{coarse\}+\\lambda\*L\_\{fine\} L=Lcoarse\+λ∗Lfine

**论文结果**:  
检测的人体关节点数目：FLIC-7 upper joints, MPII-14 joints  
数据集：(1)Frames Labeled In Cinema,(2)MPII dataset  
评价指标:

*  Percentage of Correct Keypoints(PCK): 在PCK中认为一个关节点被正确检测到的标准是，预测的关节点和ground truth关节点的归一化距离小于一个设定阈值的比例。
 *  Percentage of Correct Keypoints head(PCKh)：以head segment length作为基准设定比例的PCK。  
    在MPII数据集上全身关节平均的PCKh@0.5为82.0。

[DeepPose - human pose estimation via deep nerual networks]: https://blog.csdn.net/JerryZhang__/article/details/97561493
[Efficient Object Localization Using Convolutional Networks]: https://blog.csdn.net/JerryZhang__/article/details/97561974
[Convolutional Pose Machines]: https://blog.csdn.net/JerryZhang__/article/details/97562875
[Human Pose Estimation with Iterative Error Feedback]: https://blog.csdn.net/JerryZhang__/article/details/97563255
[Hourglass Network - stacked hourglass network for human pose estimation]: https://blog.csdn.net/JerryZhang__/article/details/98308729
[OpenPose - Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields]: https://blog.csdn.net/JerryZhang__/article/details/98667549
[RMPE_ Regional Multi-person Pose Estimation]: https://blog.csdn.net/JerryZhang__/article/details/100767171
[Pose flow_ Efficient Online Pose Tracking]: https://blog.csdn.net/JerryZhang__/article/details/100835787
[CrowdPose_ Efficient Crowded Scenes Pose Estimation and A New Benchmark]: https://blog.csdn.net/JerryZhang__/article/details/100937176
[Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation]: https://arxiv.org/pdf/1406.2984.pdf
[overview of cascade model architecture]: /images/20211119/05a76b2375594716ac50e1bea2febd02.png
[coarse heat-map regression model]: /images/20211119/f23af786055c4e02bd1b3453861a6ea8.png
[fine regression network for a single joint]: /images/20211119/2c4f0ecb323d4e0693aa6d64609db0c0.png