deeplabv3+ decoder代码详解

短命女 2024-04-18 21:40 66阅读 0赞

**目录**

看代码model.py: refine\_by\_decoder

看代码定义与注释：

函数部分输入参数解读：

BN （batch normalization）配置：

操作

获取DCNN tensor用来细化分割结果

合并两个tensor

--------------------

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70][]

论文中的decoder特点

*  两个输入
 *  分了两次上采样，存在1/16与1/4特征图。

# 看代码model.py: refine\_by\_decoder #

## 看代码定义与注释： ##

def refine_by_decoder(features,
                          end_points,
                          crop_size=None,
                          decoder_output_stride=None,
                          decoder_use_separable_conv=False,
                          model_variant=None,
                          weight_decay=0.0001,
                          reuse=None,
                          is_training=False,
                          fine_tune_batch_norm=False,
                          use_bounded_activation=False):
      """Adds the decoder to obtain sharper segmentation results.
    
      Args:
        features: A tensor of size [batch, features_height, features_width,
          features_channels].
        end_points: A dictionary from components of the network to the corresponding
          activation.
        crop_size: A tuple [crop_height, crop_width] specifying whole patch crop
          size.
        decoder_output_stride: A list of integers specifying the output stride of
          low-level features used in the decoder module.
        decoder_use_separable_conv: Employ separable convolution for decoder or not.
        model_variant: Model variant for feature extraction.
        weight_decay: The weight decay for model variables.
        reuse: Reuse the model variables or not.
        is_training: Is training or not.
        fine_tune_batch_norm: Fine-tune the batch norm parameters or not.
        use_bounded_activation: Whether or not to use bounded activations. Bounded
          activations better lend themselves to quantized inference.
    
      Returns:
        Decoder output with size [batch, decoder_height, decoder_width,
          decoder_channels].
    
      Raises:
        ValueError: If crop_size is None.
      """

我给出输入数据的具体样子。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70 1][]

### 函数部分输入参数解读： ###

**decoder\_output\_stride** ：意味着我decode后是1/4,也就是（129,129，channels）大小的tensor

**end\_point** ：是一个字典，使用方法tensor\_features=end\_point\[feature\_name\]就可以取出这个feature

**features** ：原始图片是512大小，所以这里的feature输入是1/16上采样。

**fine\_tune\_batch\_norm** ：是否精调BN（batch normalization）参数

### BN （batch normalization）配置： ###

batch_norm_params = {
          'is_training': is_training and fine_tune_batch_norm,
          'decay': 0.9997,
          'epsilon': 1e-5,
          'scale': True,
      }

之后配置了所有的decoder的参数，这意味着，所有的decode操作都要服从下面代码：

[slim.arg\_scope][slim.arg_scope]与[variable\_scope][variable_scope]教程，都很简单很短。

with slim.arg_scope(
          [slim.conv2d, slim.separable_conv2d],
          weights_regularizer=slim.l2_regularizer(weight_decay),
          activation_fn=tf.nn.relu6 if use_bounded_activation else tf.nn.relu,
          normalizer_fn=slim.batch_norm,
          padding='SAME',
          stride=1,
          reuse=reuse):
        with slim.arg_scope([slim.batch_norm], **batch_norm_params):
          with tf.variable_scope(DECODER_SCOPE, DECODER_SCOPE, [features]):

另外插播一段传递\*\*参数的教程说明：

>>> def test_args_kwargs(arg1, arg2, arg3):
    ...     print("arg1:", arg1)
    ...     print("arg2:", arg2)
    ...     print("arg3:", arg3)
    ... 
    >>> kwargs = {"arg3": 3, "arg2": "two", "arg1": 5}
    >>> test_args_kwargs(**kwargs)
    arg1: 5
    arg2: two
    arg3: 3
    >>> kwargs = {"arg3": 3, "arg2": "two", "arg": 5}
    >>> test_args_kwargs(**kwargs)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: test_args_kwargs() got an unexpected keyword argument 'arg'
    >>>

所以其实这里传入的fine\_tune的参数就是按照关键字传入，等同于：

with slim.arg_scope([slim.batch_norm], **batch_norm_params)
    
    with slim.arg_scope([slim.batch_norm], is_training=is_training and fine_tune_batch_norm, decay=0.9997, epsilon=1e-5, scale=True)

这些参数arg\_scope有所少收多少，不加以区分，因为他根本就不读，看都不看就全都扔给下面的具体操作了。

### 操作 ###

首先根据使用的网络名字(比如xception\_65,mobilenet\_v2等）获取该网络最后一层的输出。然后与feature\_extractor得到的tensor结合。

直接获取encode的输入，不使用aspp，是下图DCNN到Decoder的输入部分。

居然不能随便在图片上画，这个编辑器不好使，我要找一个好使的。凑合看吧。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70 2][]

### 获取DCNN tensor用来细化分割结果 ###

代码

feature_list = feature_extractor.networks_to_feature_maps[
                  model_variant][
                      feature_extractor.DECODER_END_POINTS][output_stride]

上面的代码实际上就是去字典里取字符串。所以feature_list:<class 'list'>取值就是：
    
    ['entry_flow/block2/unit_1/xception_module/separable_conv2_pointwise']
    
    feature_extractor.DECODER_END_POINTS = 'decoder_end_points'

根据目前网络，xception65，得到aspp操作之前的tensor的名字：

feature_name = '{}/{}'.format(
                      feature_extractor.name_scope[model_variant], name)

feature\_name='xception\_65/entry\_flow/block2/unit\_1/xception\_module/separable\_conv2\_pointwise'

通过下面代码可以根据名字获取tensor：

decoder_features_list.append(
                    slim.conv2d(
                        end_points[feature_name],
                        48,
                        1,
                        scope='feature_projection' + str(i) + scope_suffix))

上面end\_points\[feature\_name\]返回的就是DCNN输出的tensor。

然后看下这个tensor的shape，**卷积之前**是(129,129,256).根据上面的代码**卷积之后**肯定是（129,129,48）

Tensor("xception\_65/entry\_flow/block2/unit\_1/xception\_module/separable\_conv2\_pointwise/BatchNorm/FusedBatchNorm:0", shape=(?, 129, 129, 256), dtype=float32, device=/device:GPU:0)

因为：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70 3][]

取得就是1/4的特征图来细化的。

### 合并两个tensor ###

将上述DCNN得到的特征成为**细化张量**。回顾：（129,129,256）后来被卷积了，卷积核大小是1，一共48个，所以（129,129,48）

将ASPP后得到的最终的特征称为**语义张量**。回顾：（33,33,256）后来被上采样了（129,129,256）

要将上面两个特征结合。利用的是utils.split\_separate\_conv2d.

分别卷积是一个通用的方法。目前很流行，所以，就不讲了。直接看代码。

先介绍分开卷积的函数：

def split_separable_conv2d(inputs,
                               filters,
                               kernel_size=3,
                               rate=1,
                               weight_decay=0.00004,
                               depthwise_weights_initializer_stddev=0.33,
                               pointwise_weights_initializer_stddev=0.06,
                               scope=None):
      """Splits a separable conv2d into depthwise and pointwise conv2d.
    
      This operation differs from `tf.layers.separable_conv2d` as this operation
      applies activation function between depthwise and pointwise conv2d.
    
      Args:
        inputs: Input tensor with shape [batch, height, width, channels].
        filters: Number of filters in the 1x1 pointwise convolution.
        kernel_size: A list of length 2: [kernel_height, kernel_width] of
          of the filters. Can be an int if both values are the same.
        rate: Atrous convolution rate for the depthwise convolution.
        weight_decay: The weight decay to use for regularizing the model.
        depthwise_weights_initializer_stddev: The standard deviation of the
          truncated normal weight initializer for depthwise convolution.
        pointwise_weights_initializer_stddev: The standard deviation of the
          truncated normal weight initializer for pointwise convolution.
        scope: Optional scope for the operation.
    
      Returns:
        Computed features after split separable conv2d.
      """

注意这里默认卷积核大小是3.

再看细化张量与语义张量合并卷积过程：

decoder_depth = 256
                if decoder_use_separable_conv:
                  decoder_features = split_separable_conv2d(
                      tf.concat(decoder_features_list, 3),
                      filters=decoder_depth,
                      rate=1,
                      weight_decay=weight_decay,
                      scope='decoder_conv0' + scope_suffix)
                  decoder_features = split_separable_conv2d(
                      decoder_features,
                      filters=decoder_depth,
                      rate=1,
                      weight_decay=weight_decay,
                      scope='decoder_conv1' + scope_suffix)

首先是在第三个维度上concatenate了两个张量：

Tensor("decoder/concat:0", shape=(?, 129, 129, 304), dtype=float32, device=/device:GPU:0)

经过第一次split卷积得到：

Tensor("decoder/decoder\_conv0\_pointwise/Relu:0", shape=(?, 129, 129, 256), dtype=float32, device=/device:GPU:0)

再经过一次一样的256通道，3\*3卷积核的split卷积得到：

Tensor("decoder/decoder\_conv1\_pointwise/Relu:0", shape=(?, 129, 129, 256), dtype=float32, device=/device:GPU:0)

就可以返回了，于是decode过程就结束了。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/c15ef14ddff2490cb39e19957c21ad3d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70 1]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/72a86567e5784c4895d4253093cc99b3.png
[slim.arg_scope]: https://blog.csdn.net/u013249853/article/details/100596207
[variable_scope]: https://zhuanlan.zhihu.com/p/37711713
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70 2]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/f1d6060a59524ac68178799a244feaf4.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNDk4NTM_size_16_color_FFFFFF_t_70 3]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/2b90e6ee1039470c9c3581e051eca1eb.png