憨批的语义分割1——基于Mobile模型的segnet讲解

忘是亡心i 2023-05-30 09:07 55阅读 0赞

### 憨批的语义分割1——基于Mobile模型的segnet讲解 ###

*  学习前言
 *  什么是Segnet模型
 *  segnet模型的代码实现
 *   *  1、主干模型Mobilenet。
     *  2、segnet的Decoder解码部分
 *  代码测试

# 学习前言 #

最近开始设计新的领域啦，语义分割也是图像处理一个非常重要的应用方向，我查了很多资料苦于如何入门，接下来给大家讲讲里面比较基础的segnet模型。在下一个BLOG我会跟大家讲怎么训练自己的segnet模型。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center]

# 什么是Segnet模型 #

Segnet模型是一个比较基础的语义分割模型，其结构比较简单，在说其结构之前，我们先讲一下convolutional Encoder-Decoder的结构。  
其**主要结构与自编码（Autoencoder）类似，通过编码解码复原图片上每一个点所属的类别。**  
下图主要是说明利用卷积层编码与解码的过程。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center 1]  
segnet模型与上述模型类似。  
因为基于VGG模型的语义分割模型都太大了，我的电脑支撑不住，所以我用的**主干网络都是mobile模型。**  
其主要结构如图所示：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center 2]  
由结构可以看到，**其利用Encoder中提取了多次特征的f4进行处理，利用Decoder进行多次上采样Upsampling2D。最后得到一个具有一定hw的filter数量为n\_classes的图层。**  
为什么filter要用n\_classes呢，**因为其代表的就是每个像素点所属的种类。**

这么一想其实语义分割是不是也没有那么难？

用一句话概括就是 **从主干模型中提取出卷积了多次，具有一定特征的层（典型的是hw经过了4次压缩后的层），然后利用UpSampling2D函数进行三次上采样，得到输出层（语句分割的结果）。**

# segnet模型的代码实现 #

segnet模型的代码分为两部分。

## 1、主干模型Mobilenet。 ##

该部分用于特征提取，实际上就是常规的mobilenet结构，想要了解mobilenet结构的朋友们可以看看我的另一篇博客[神经网络学习小记录23——MobileNet模型的复现详解][23_MobileNet]：

from keras.models import *
    from keras.layers import *
    import keras.backend as K
    import keras
    
    IMAGE_ORDERING = 'channels_last'
    
    def relu6(x):
    	return K.relu(x, max_value=6)
    
    def _conv_block(inputs, filters, alpha, kernel=(3, 3), strides=(1, 1)):
    
    	channel_axis = 1 if IMAGE_ORDERING == 'channels_first' else -1
    	filters = int(filters * alpha)
    	x = ZeroPadding2D(padding=(1, 1), name='conv1_pad', data_format=IMAGE_ORDERING  )(inputs)
    	x = Conv2D(filters, kernel , data_format=IMAGE_ORDERING  ,
    										padding='valid',
    										use_bias=False,
    										strides=strides,
    										name='conv1')(x)
    	x = BatchNormalization(axis=channel_axis, name='conv1_bn')(x)
    	return Activation(relu6, name='conv1_relu')(x)
    
    def _depthwise_conv_block(inputs, pointwise_conv_filters, alpha,
    													depth_multiplier=1, strides=(1, 1), block_id=1):
    
    	channel_axis = 1 if IMAGE_ORDERING == 'channels_first' else -1
    	pointwise_conv_filters = int(pointwise_conv_filters * alpha)
    
    	x = ZeroPadding2D((1, 1) , data_format=IMAGE_ORDERING , name='conv_pad_%d' % block_id)(inputs)
    	x = DepthwiseConv2D((3, 3) , data_format=IMAGE_ORDERING ,
    														 padding='valid',
    														 depth_multiplier=depth_multiplier,
    														 strides=strides,
    														 use_bias=False,
    														 name='conv_dw_%d' % block_id)(x)
    	x = BatchNormalization(
    			axis=channel_axis, name='conv_dw_%d_bn' % block_id)(x)
    	x = Activation(relu6, name='conv_dw_%d_relu' % block_id)(x)
    
    	x = Conv2D(pointwise_conv_filters, (1, 1), data_format=IMAGE_ORDERING ,
    										padding='same',
    										use_bias=False,
    										strides=(1, 1),
    										name='conv_pw_%d' % block_id)(x)
    	x = BatchNormalization(axis=channel_axis,
    																name='conv_pw_%d_bn' % block_id)(x)
    	return Activation(relu6, name='conv_pw_%d_relu' % block_id)(x)
    
    def get_mobilenet_encoder( input_height=224 ,  input_width=224 , pretrained='imagenet' ):
    
    	alpha=1.0
    	depth_multiplier=1
    	dropout=1e-3
    
    
    	img_input = Input(shape=(input_height,input_width , 3 ))
    
    
    	x = _conv_block(img_input, 32, alpha, strides=(2, 2))
    	x = _depthwise_conv_block(x, 64, alpha, depth_multiplier, block_id=1) 
    	f1 = x
    
    	x = _depthwise_conv_block(x, 128, alpha, depth_multiplier,
    														strides=(2, 2), block_id=2)  
    	x = _depthwise_conv_block(x, 128, alpha, depth_multiplier, block_id=3) 
    	f2 = x
    
    	x = _depthwise_conv_block(x, 256, alpha, depth_multiplier,
    														strides=(2, 2), block_id=4)  
    	x = _depthwise_conv_block(x, 256, alpha, depth_multiplier, block_id=5) 
    	f3 = x
    
    	x = _depthwise_conv_block(x, 512, alpha, depth_multiplier,
    														strides=(2, 2), block_id=6) 
    	x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=7) 
    	x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=8) 
    	x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=9) 
    	x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=10) 
    	x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=11) 
    	f4 = x 
    
    	x = _depthwise_conv_block(x, 1024, alpha, depth_multiplier,
    														strides=(2, 2), block_id=12)  
    	x = _depthwise_conv_block(x, 1024, alpha, depth_multiplier, block_id=13) 
    	f5 = x 
    
    	return img_input , [f1 , f2 , f3 , f4 , f5 ]

## 2、segnet的Decoder解码部分 ##

这一部分对应着上面segnet模型中的解码部分。  
其关键就是**把获得的特征重新映射到比较大的图中的每一个像素点，用于每一个像素点的分类。**

from keras.models import *
    from keras.layers import *
    from nets.mobilenet import get_mobilenet_encoder
    
    IMAGE_ORDERING = 'channels_last'
    def segnet_decoder(  f , n_classes , n_up=3 ):
    
    	assert n_up >= 2
    
    	o = f
    	o = ( ZeroPadding2D( (1,1) , data_format=IMAGE_ORDERING ))(o)
    	o = ( Conv2D(512, (3, 3), padding='valid', data_format=IMAGE_ORDERING))(o)
    	o = ( BatchNormalization())(o)
    	# 进行一次UpSampling2D，此时hw变为原来的1/8
    	# 52,52,512
    	o = ( UpSampling2D( (2,2), data_format=IMAGE_ORDERING))(o)
    	o = ( ZeroPadding2D( (1,1), data_format=IMAGE_ORDERING))(o)
    	o = ( Conv2D( 256, (3, 3), padding='valid', data_format=IMAGE_ORDERING))(o)
    	o = ( BatchNormalization())(o)
    
    	# 进行一次UpSampling2D，此时hw变为原来的1/4
    	# 104,104,256
    	for _ in range(n_up-2):
    		o = ( UpSampling2D((2,2)  , data_format=IMAGE_ORDERING ) )(o)
    		o = ( ZeroPadding2D((1,1) , data_format=IMAGE_ORDERING ))(o)
    		o = ( Conv2D( 128 , (3, 3), padding='valid' , data_format=IMAGE_ORDERING ))(o)
    		o = ( BatchNormalization())(o)
    
    	# 进行一次UpSampling2D，此时hw变为原来的1/2
    	# 208,208,128
    	o = ( UpSampling2D((2,2)  , data_format=IMAGE_ORDERING ))(o)
    	o = ( ZeroPadding2D((1,1)  , data_format=IMAGE_ORDERING ))(o)
    	o = ( Conv2D( 64 , (3, 3), padding='valid'  , data_format=IMAGE_ORDERING ))(o)
    	o = ( BatchNormalization())(o)
    
    	# 此时输出为h_input/2,w_input/2,nclasses
    	o =  Conv2D( n_classes , (3, 3) , padding='same', data_format=IMAGE_ORDERING )( o )
    	
    	return o 
    
    def _segnet( n_classes , encoder  ,  input_height=416, input_width=608 , encoder_level=3):
    	# encoder通过主干网络
    	img_input , levels = encoder( input_height=input_height ,  input_width=input_width )
    
    	# 获取hw压缩四次后的结果
    	feat = levels[encoder_level]
    
    	# 将特征传入segnet网络
    	o = segnet_decoder(feat, n_classes, n_up=3 )
    
    	# 将结果进行reshape
    	o = Reshape((int(input_height/2)*int(input_width/2), -1))(o)
    	o = Softmax()(o)
    	model = Model(img_input,o)
    
    	return model
    
    def mobilenet_segnet( n_classes ,  input_height=224, input_width=224 , encoder_level=3):
    
    	model = _segnet( n_classes , get_mobilenet_encoder ,  input_height=input_height, input_width=input_width , encoder_level=encoder_level)
    	model.model_name = "mobilenet_segnet"
    	return model

# 代码测试 #

将上面两个代码分别保存为mobilenet.py和segnet.py。按照如下方式存储：  
![在这里插入图片描述][20191108202725368.png_pic_center]  
(忽略其中的predict.py，那是下一个blog讲怎么训练的时候用的。)  
此时我们运行test.py的代码：

from nets.segnet import mobilenet_segnet
    
    model = mobilenet_segnet(2,input_height=416,input_width=416)
    model.summary()

如果没有出错的话就会得到如下的结果：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center 3]  
到这里就完成了基于Mobile模型的segnet的搭建。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/20190723144346254.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA==,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center 1]: https://img-blog.csdnimg.cn/20191108195650187.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA==,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center 2]: https://img-blog.csdnimg.cn/20191108201032126.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA==,size_16,color_FFFFFF,t_70#pic_center
[23_MobileNet]: https://blog.csdn.net/weixin_44791964/article/details/102819915
[20191108202725368.png_pic_center]: https://img-blog.csdnimg.cn/20191108202725368.png#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA_size_16_color_FFFFFF_t_70_pic_center 3]: https://img-blog.csdnimg.cn/20191108202829954.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDc5MTk2NA==,size_16,color_FFFFFF,t_70#pic_center