单向神经网络python代码及算法分析

约定不等于承诺〃 2023-06-02 09:13 62阅读 0赞

我写这篇文章主要目的是记录，所以过程可能比较简略。

本质上学习logistic 回归的系数。

# 前向传播： #

![20191004112136945.png][]

输入样本集X，X.shape=（样本数，特征数），W.shape=（特征数，输出数）,b形状和w一样，不过利用python广播的性质，b可以是一位数组，\[输出数\]

z=x\*w+b

激活函数σ：

y=a=σ(z)

损失函数

![20191004112209778.png][]

# 反向传播： #

反向传播过程计算梯度

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMDc2Nzk3_size_16_color_FFFFFF_t_70][]

w=w-学习率\*Dw

b=b-学习率\*Db

Da=dL/da=(a-y)/(a\*(1-a))

da/dz=a(1-a)

Dz=dL/dz=(dL/da)\*(da/dz)=a-y

Dw=(dL/dz)\*(dz/dw)=Dz\*x

Db=Dz

# 代码 #

import numpy as np
    import matplotlib.pyplot as plt
    
    # define the input data
    X = np.array([[3, 3],
                  [5, 4],
                  [1, 1],
                  [2, 1]])
    # define the label
    T = np.array([[1],
                  [1],
                  [-1],
                  [-1]])
    
    # define and initialize the weight 0-1
    W = np.random.random([2, 1])
    
    #
    b=np.random.random([1])
    
    # define the learning rate
    lr = 0.1
    
    # define the neural network
    Y = 0
    
    cost=0
    # update the weight function
    def train():
        global X, Y, W, lr, T,b,cost
        # calculate 4 predicted value meanwhile ,Y(4,1)
        Y = np.sign(np.dot(X, W)+b)#[样本数，输出数]
        # T-Y get the label and predicted error E(4,1)
        E = Y - T#[样本数，输出数]=[4,1]
        # calculate the weight change
        delta_W = lr * (X.T.dot(E)) / X.shape[0]#X.T.dot(E):[2,4]*[4,1]=[2,1]
        delta_b = np.sum(E)/X.shape[0]
        # update weight value
        W = W - delta_W
        b=b-delta_b
        #cost = -1 /X.shape[0] * np.sum(Y * np.log(Y) + (1 - T) * np.log(1 - Y))
    
    
    for i in range(100):
        # update weight value
        train()
        # print the traning count
        print("training count:", i + 1)
        # print the weights
        print("weights:", W)
        print("b:",b)
        # calculate the current output
        Y = np.sign(np.dot(X, W)+b)
        # all() All the Y value is equal All the T value
        if (Y == T).all():
            print("Finish!")
            break;
    
    # draw
    # example x,y
    x1 = [3, 3]
    y1 = [4, 3]
    x2 = [1, 1]
    y2 = [2, 1]
    
    
    x=np.linspace(0,5,100)
    y=(-b-x*W[0])/W[1]
    # draw a calssify line
    plt.plot(x,y, 'r')
    # draw the example point
    plt.scatter(x1, y1, c='b')
    plt.scatter(x2, y2, c='y')
    plt.show()

[20191004112136945.png]: /images/20230601/c595b20ddfd74987bf1d4621bb75e024.png
[20191004112209778.png]: /images/20230601/3c9cd83df9664ccebb3817c1379c9759.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMDc2Nzk3_size_16_color_FFFFFF_t_70]: /images/20230601/cbfb21be427f4926b52976b3dad246d6.png