CornerNet源码阅读笔记（一）------解码过程

朴灿烈づ我的快乐病毒、 2022-01-28 01:27 214阅读 0赞

CornerNet和loss、解码相关的函数其实在kp.py和kp\_utils.py里面

解码函数如下所示：

def _decode(
        tl_heat, br_heat, tl_tag, br_tag, tl_regr, br_regr, 
        K=100, kernel=1, ae_threshold=1, num_dets=1000
    ):
        batch, cat, height, width = tl_heat.size()
    
        tl_heat = torch.sigmoid(tl_heat)
        br_heat = torch.sigmoid(br_heat)
    
        # perform nms on heatmaps
        """
        其实就是对概率图进行maxpooling
        """
        tl_heat = _nms(tl_heat, kernel=kernel)
        br_heat = _nms(br_heat, kernel=kernel)
    
        tl_scores, tl_inds, tl_clses, tl_ys, tl_xs = _topk(tl_heat, K=K)
        br_scores, br_inds, br_clses, br_ys, br_xs = _topk(br_heat, K=K)
        """
        tl_ys,tl_xs原本的shape为[batch,K]
        """
        tl_ys = tl_ys.view(batch, K, 1).expand(batch, K, K)
        tl_xs = tl_xs.view(batch, K, 1).expand(batch, K, K)
        br_ys = br_ys.view(batch, 1, K).expand(batch, K, K)
        br_xs = br_xs.view(batch, 1, K).expand(batch, K, K)
    
        if tl_regr is not None and br_regr is not None:
            tl_regr = _tranpose_and_gather_feat(tl_regr, tl_inds)
            tl_regr = tl_regr.view(batch, K, 1, 2)
            br_regr = _tranpose_and_gather_feat(br_regr, br_inds)
            br_regr = br_regr.view(batch, 1, K, 2)
    
            tl_xs = tl_xs + tl_regr[..., 0]
            tl_ys = tl_ys + tl_regr[..., 1]
            br_xs = br_xs + br_regr[..., 0]
            br_ys = br_ys + br_regr[..., 1]
    
        # all possible boxes based on top k corners (ignoring class)
        bboxes = torch.stack((tl_xs, tl_ys, br_xs, br_ys), dim=3)
    
        tl_tag = _tranpose_and_gather_feat(tl_tag, tl_inds)
        tl_tag = tl_tag.view(batch, K, 1)
        br_tag = _tranpose_and_gather_feat(br_tag, br_inds)
        br_tag = br_tag.view(batch, 1, K)
        """
        [k,1] - [1,k]隐式的都会扩张为[K,K]再相减
        dists也为[K,K]
        """
        dists  = torch.abs(tl_tag - br_tag)
    
        tl_scores = tl_scores.view(batch, K, 1).expand(batch, K, K)
        br_scores = br_scores.view(batch, 1, K).expand(batch, K, K)
        scores    = (tl_scores + br_scores) / 2
    
        # reject boxes based on classes
        tl_clses = tl_clses.view(batch, K, 1).expand(batch, K, K)
        br_clses = br_clses.view(batch, 1, K).expand(batch, K, K)
        cls_inds = (tl_clses != br_clses)
    
        # reject boxes based on distances
        dist_inds = (dists > ae_threshold)
    
        # reject boxes based on widths and heights
        width_inds  = (br_xs < tl_xs)
        height_inds = (br_ys < tl_ys)
    
        scores[cls_inds]    = -1
        scores[dist_inds]   = -1
        scores[width_inds]  = -1
        scores[height_inds] = -1
    
        scores = scores.view(batch, -1)
        scores, inds = torch.topk(scores, num_dets)
        scores = scores.unsqueeze(2)
        """
        100*100的点最终匹配为10000个box,然后用类，距离，左上角和右下角的相对位置过滤掉n个点,过滤就体现在scores变为-1，然后传递到inds
        最终通过 bboxes = _gather_feat(bboxes, inds)实现在box上的过滤
        """
        bboxes = bboxes.view(batch, -1, 4)
        bboxes = _gather_feat(bboxes, inds)
    
        clses  = tl_clses.contiguous().view(batch, -1, 1)
        clses  = _gather_feat(clses, inds).float()
    
        tl_scores = tl_scores.contiguous().view(batch, -1, 1)
        tl_scores = _gather_feat(tl_scores, inds).float()
        br_scores = br_scores.contiguous().view(batch, -1, 1)
        br_scores = _gather_feat(br_scores, inds).float()
    
        detections = torch.cat([bboxes, scores, tl_scores, br_scores, clses], dim=2)
        return detections

一、理论回顾

CornerNet的总共上下两个分支，每个分支三种输出，分别为：各点为角点的概率，偏移，表示两个点是否为同一个物体的embedding得分

左上角分支：

heat map：tl\_heat  \[batch,C,H,W\]

offset： tl\_regr  \[batch,2,H,W\]

embedding: tl\_tag    \[batch,1,H,W\]

右下角分支：

br\_heat,  tl\_heat  \[batch,C,H,W\]

br\_regr   \[batch,2,H,W\]

br\_tag  \[batch,1,H,W\]

总计tl\_heat, br\_heat, tl\_tag, br\_tag, tl\_regr, br\_regr 六个输出。

解码流程：