Hourglass网络的理解和代码分析

深碍√TFBOYSˉ_ 2023-06-13 09:17 4阅读 0赞

### 文章目录 ###

*   *  1. 前言
     *  2.一个简单的hourglass网络结构
     *  3. Hourglass 网络的定义：
     *   *  3.1 方法一：
         *  3.2 方法二(简陋版)：

## 1. 前言 ##

目标检测最新的一些方法中都采用了人脸关键点检测hourglass网络作为检测的主干网络，如CornerNet-Lite系列。目标检测将anchor生成box的方式替换关键点预测，采用hourglass网络的优点在于物体特征点可能出现在网络的不同层，如果采用VGG，或者ResNet网络，最后的特征图很难包含检测物体的所有关键点。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70]

## 2.一个简单的hourglass网络结构 ##

堆叠hourglass网络的结构如下图，每一个白色的box表示一个residual模块。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70 1]堆叠hourglass 网络是个递归的结构，输入从左到中间，维度增加，特征map的大小变小，从中间到右，维度减少，特征map变大，即C1嵌套C2，C2嵌套C3，依次类推，，，C5，C6, C7是residual模块串联，总网络是4层的嵌套。

直观一点，沙漏网络：哈哈哈！！！  
![在这里插入图片描述][20191115160436197.jpg]

residual模块参照下图，这个模块的特性可以对特征图升维和降维，并且不改变特征图的size（W, H）。  
每一个白色的box大致可以理解为下面的模块。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70 2]  
如果上图还是看起来复杂，看下图：（就是个利用1\*1卷积对图像升维或者降维）  
参考：resnet网络  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70 3]

## 3. Hourglass 网络的定义： ##

### 3.1 方法一： ###

1.先定义residual模块：  
residual传入参数（输入特征图数，输出特征图数）

class residual(nn.Module):
        def __init__(self, inp_dim, out_dim, k=3, stride=1):
            super(residual, self).__init__()
            p = (k - 1) // 2
    
            self.conv1 = nn.Conv2d(inp_dim, out_dim, (k, k), padding=(p, p), stride=(stride, stride), bias=False)
            self.bn1   = nn.BatchNorm2d(out_dim)
            self.relu1 = nn.ReLU(inplace=True)
    
            self.conv2 = nn.Conv2d(out_dim, out_dim, (k, k), padding=(p, p), bias=False)
            self.bn2   = nn.BatchNorm2d(out_dim)
            
            self.skip  = nn.Sequential(
                nn.Conv2d(inp_dim, out_dim, (1, 1), stride=(stride, stride), bias=False),
                nn.BatchNorm2d(out_dim)
            ) if stride != 1 or inp_dim != out_dim else nn.Sequential()
            self.relu  = nn.ReLU(inplace=True)
    
        def forward(self, x):
            conv1 = self.conv1(x)
            bn1   = self.bn1(conv1)
            relu1 = self.relu1(bn1)
    
            conv2 = self.conv2(relu1)
            bn2   = self.bn2(conv2)
    
            skip  = self.skip(x)
            return self.relu(bn2 + skip)

1.  定义几个layer，避免重复工作：

import torch
    import torch.nn as nn
     
    class merge(nn.Module):
    	def forward(self, x, y):
    		return x + y
             
    def _make_layer(inp_dim, out_dim, modules):
        layers  = [residual(inp_dim, out_dim)]
        layers += [residual(out_dim, out_dim) for _ in range(1, modules)]
        return nn.Sequential(*layers)
    
    def _make_layer_revr(inp_dim, out_dim, modules):
        layers  = [residual(inp_dim, inp_dim) for _ in range(modules - 1)]
        layers += [residual(inp_dim, out_dim)]
        return nn.Sequential(*layers)
    
    def _make_pool_layer(dim):
        return nn.MaxPool2d(kernel_size=2, stride=2)
    
    def _make_unpool_layer(dim):
        return nn.Upsample(scale_factor=2)
    
    def _make_merge_layer(dim):
        return merge()

1.  定义hourglass网络：

class hg_module(nn.Module):
        def __init__(
            self, n, dims,     #dim=[256, 256, 384, 384, 384, 512]
            modules,      #moudle=[2, 2, 2, 2, 2, 4]
             make_up_layer=_make_layer, make_pool_layer=_make_pool_layer, 
             make_hg_layer=_make_layer, make_low_layer=_make_layer,   	
             make_hg_layer_revr=_make_layer_revr，  make_unpool_layer=_make_unpool_layer, 
             make_merge_layer=_make_merge_layer
        ):   #这块就是一些ModuleList，换了个名字，网络结构流看forward函数很清晰
            super(hg_module, self).__init__()
    
            curr_mod = modules[0]  # moudule列表的每个数字 表示MoudleList要重复用几次
            next_mod = modules[1]
    
            curr_dim = dims[0]
            next_dim = dims[1]
    
            self.n    = n
            self.up1  = make_up_layer(curr_dim, curr_dim, curr_mod)  #用1次，看第2步make_up_layer函数
            #up1 的意思是接收外层的传入的特征图，若外层是1层，内层就是第二层（这里刚开始是输入图像----->第一层），up1层做residual操作特征图尺寸没变
            self.max1 = make_pool_layer(curr_dim)  # 注意啦，这里尺寸降了一次
            self.low1 = make_hg_layer(curr_dim, next_dim, curr_mod)  # resiual 操作，特征map尺寸不变，变的是特征map的维度个数
            self.low2 = hg_module(
                n - 1, dims[1:], modules[1:],  #递归啦，这里就是由第2层往第3层走， 3-->4， 4--->5
                                               #map的维度由内层256--->256, 256---->384, 384----->384, 384------>384, 384------>512
                make_up_layer=make_up_layer,      
                make_pool_layer=make_pool_layer,
                make_hg_layer=make_hg_layer,
                make_low_layer=make_low_layer,
                make_hg_layer_revr=make_hg_layer_revr,
                make_unpool_layer=make_unpool_layer,
                make_merge_layer=make_merge_layer
            ) if n > 1 else make_low_layer(next_dim, next_dim, next_mod)  #n=1时，跳出来直接到了最外层了
            self.low3 = make_hg_layer_revr(next_dim, curr_dim, curr_mod)
            self.up2  = make_unpool_layer(curr_dim) # 最外层upsample一次，因为递归开始前，输入图像------->第一层 降维了一次，
            self.merg = make_merge_layer(curr_dim)
    
        def forward(self, x):
            up1  = self.up1(x)
            max1 = self.max1(x)
            low1 = self.low1(max1)
            low2 = self.low2(low1)
            low3 = self.low3(low2)
            up2  = self.up2(low3)
            merg = self.merg(up1, up2)
            return merg

4.调用hourglass模块总函数：

hg_mods = nn.ModuleList([
                hg_module(
                    5, [256, 256, 384, 384, 384, 512], [2, 2, 2, 2, 2, 4], 
                     #5表示stack的hourglass网络层数，5个堆叠起来，
                     #[256, 256, 384, 384, 384, 512]表示总网络前半层的维度变化，因为这个网络就是沙漏网络么，越往中间走特征图的维度越高，map的size越小
                    make_pool_layer=make_pool_layer,
                    make_hg_layer=make_hg_layer
                )])

--------------------

### 3.2 方法二(简陋版)： ###

1.  residual 模块定义

class Residual(nn.Module):
        """ 残差模块，并不改变特征图的宽高 """
        def __init__(self,ins,outs):
            super(Residual,self).__init__()
            # 卷积模块
            self.convBlock = nn.Sequential(
                nn.BatchNorm2d(ins),
                nn.ReLU(inplace=True),
                nn.Conv2d(ins,outs/2,1),
                nn.BatchNorm2d(outs/2),
                nn.ReLU(inplace=True),
                nn.Conv2d(outs/2,outs/2,3,1,1),
                nn.BatchNorm2d(outs/2),
                nn.ReLU(inplace=True),
                nn.Conv2d(outs/2,outs,1)
            )
            # 跳层
            self.skipConv = nn.Conv2d(ins,outs,1)
    
        def forward(self,x):
            residual = x
            x = self.convBlock(x)
            residual = self.skipConv(residual)
            x += nn.ReLU(residual)
            return x

1.  hourglass网络定义：

class HourGlass(nn.Module):
        """不改变特征图的高宽"""
        def __init__(self,n=4,f=128):
            """ :param n: hourglass模块的层级数目 :param f: hourglass模块中的特征图数量 :return: """
            super(HourGlass,self).__init__()
            self._n = n
            self._f = f
            self._init_layers(self._n,self._f)
    
        def _init_layers(self,n,f):
            # 上分支
            setattr(self,'res'+str(n)+'_1',Residual(f,f))
            # 下分支
            setattr(self,'pool'+str(n)+'_1',nn.MaxPool2d(2,2))
            setattr(self,'res'+str(n)+'_2',Residual(f,f))
            if n > 1:
                self._init_layers(n-1,f)
            else:
                self.res_center = Residual(f,f)
            setattr(self,'res'+str(n)+'_3',Residual(f,f))
            setattr(self,'unsample'+str(n),Upsample(scale_factor=2))
    
    
        def _forward(self,x,n,f):
            # 上分支
            up1 = x
            up1 = eval('self.res'+str(n)+'_1')(up1)
            # 下分支
            low1 = eval('self.pool'+str(n)+'_1')(x)
            low1 = eval('self.res'+str(n)+'_2')(low1)
            if n > 1:
                low2 = self._forward(low1,n-1,f)
            else:
                low2 = self.res_center(low1)
            low3 = low2
            low3 = eval('self.'+'res'+str(n)+'_3')(low3)
            up2 = eval('self.'+'unsample'+str(n)).forward(low3)
    
            return up1+up2
    
        def forward(self,x):
            return self._forward(x,self._n,self._f)

参考：https://blog.csdn.net/wangzi371312/article/details/81174452  
https://blog.csdn.net/u011304078/article/details/80683985

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20191115145117269.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20191115145053540.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng==,size_16,color_FFFFFF,t_70
[20191115160436197.jpg]: https://img-blog.csdnimg.cn/20191115160436197.jpg
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20191115112110290.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20191115112458292.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODYzMjI0Ng==,size_16,color_FFFFFF,t_70