深度学习 | 训练网络trick——mixup

布满荆棘的人生 2022-03-15 10:52 618阅读 0赞

#### 1.mixup原理介绍 ####

[mixup 论文地址][mixup]  
mixup是一种非常规的数据增强方法，一个和数据无关的简单数据增强原则，其以线性插值的方式来构建新的训练样本和标签。最终对标签的处理如下公式所示，这很简单但对于增强策略来说又很不一般。

![aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NTI5OTk3LTA0MDUzOGM0YTNmMTE0OTMucG5n][]

( x i , y i ) \\left ( x\_\{i\},y\_\{i\} \\right ) (xi,yi)， ( x j , y j ) \\left ( x\_\{j\},y\_\{j\} \\right ) (xj,yj)两个数据对是原始数据集中的训练样本对（训练样本和其对应的标签）。其中 λ \\lambda λ是一个服从B分布的参数, λ ∼ B e t a ( α , α ) \\lambda\\sim Beta\\left( \\alpha ,\\alpha \\right ) λ∼Beta(α,α) 。Beta分布的概率密度函数如下图所示，其中 α ∈ \[ 0 , + ∞ \] \\alpha \\in \\left \[ 0,+\\infty \\right \] α∈\[0,\+∞\]

![aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NTI5OTk3LTdiZmU4NWYyZWQzNzY0YzIucG5n][]  
因此 α \\alpha α是一个超参数，随着 α \\alpha α的增大，网络的训练误差就会增加，而其泛化能力会随之增强。而当 α → ∞ \\alpha \\rightarrow \\infty α→∞时，模型就会退化成最原始的训练策略。

#### 2.mixup的代码实现 ####

如下代码所示，实现mixup数据增强很简单，其实我个人认为这就是一种抑制过拟合的策略，增加了一些扰动，从而提升了模型的泛化能力。

def get_batch(x, y, step, batch_size, alpha=0.2):
        """
        get batch data
        :param x: training data
        :param y: one-hot label
        :param step: step
        :param batch_size: batch size
        :param alpha: hyper-parameter α, default as 0.2
        :return:
        """
        candidates_data, candidates_label = x, y
        offset = (step * batch_size) % (candidates_data.shape[0] - batch_size)
    
        # get batch data
        train_features_batch = candidates_data[offset:(offset + batch_size)]
        train_labels_batch = candidates_label[offset:(offset + batch_size)]
    
        # 最原始的训练方式
        if alpha == 0:
            return train_features_batch, train_labels_batch
        # mixup增强后的训练方式
        if alpha > 0:
            weight = np.random.beta(alpha, alpha, batch_size)
            x_weight = weight.reshape(batch_size, 1, 1, 1)
            y_weight = weight.reshape(batch_size, 1)
            index = np.random.permutation(batch_size)
            x1, x2 = train_features_batch, train_features_batch[index]
            x = x1 * x_weight + x2 * (1 - x_weight)
            y1, y2 = train_labels_batch, train_labels_batch[index]
            y = y1 * y_weight + y2 * (1 - y_weight)
            return x, y

#### 3.mixup增强效果展示 ####

import matplotlib.pyplot as plt
    import matplotlib.image as Image
    import numpy as np
    
    im1 = Image.imread(r"C:\Users\Daisy\Desktop\1\xyjy.png")
    im2 = Image.imread(r"C:\Users\Daisy\Desktop\1\xyjy2.png")
    for i in range(1,10):
        lam= i*0.1
        im_mixup = (im1*lam+im2*(1-lam))
        plt.subplot(3,3,i)
        plt.imshow(im_mixup)
    plt.show()

![aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NTI5OTk3LWY5NjM1ODI2NWRkZjA1YWUucG5n][]

[mixup]: https://arxiv.org/abs/1710.09412
[aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NTI5OTk3LTA0MDUzOGM0YTNmMTE0OTMucG5n]: /images/20220315/062c4c940af346209c29e59fc203c624.png
[aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NTI5OTk3LTdiZmU4NWYyZWQzNzY0YzIucG5n]: /images/20220315/ac9263437e804a62ba573af69d109f94.png
[aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NTI5OTk3LWY5NjM1ODI2NWRkZjA1YWUucG5n]: /images/20220315/77624a3b16e746d98582b91c7b06f2a9.png