机器学习_感知机

本是古典何须时尚 2023-03-06 05:24 18阅读 0赞

感知机是二分类的线性分类模型，输入为特征向量， 输出为类别(1,-1)。 感知机目的是求能够将输入的线性数据进行划分的超平面。对误分类样本求损失函数，使其最小从而得到超平面。

感知机分为原始形式以及对偶形式。模型:![f(x) = sign(w\*x+b)][f_x_ _ sign_w_x_b]

sign（）为符号函数，小于零输出-1，大于等于0输出1

原始形式:

(1) 选取初始w，b

(2) 在训练集中选取数据(x，y)

(3)如果y(w\*x +b)<=0，更新权重（分类正确的时候不更新）

更新方法：

![w+\\eta \*y\*x->w][w_eta _y_x-_w]

![b+\\eta \*y->b][b_eta _y-_b]

(4) 转（2）直到没有误分类点，求出来的w，,b带入模型

--------------------

对偶形式:

(1)初始*alpha=\[0,0....\] n个,n=样本个数,b=0学习率![\\eta][eta]*

*    (2)计算gram矩阵*[https://zhuanlan.zhihu.com/p/105470826][https_zhuanlan.zhihu.com_p_105470826]

(3)如果![y\_\{i\}( \\sum\_\{j\}^\{N\} \\alpha \_\{j\}y\_\{i\}x\_\{j\}x\_\{i\}+b)\\leq 0][y_i_ _sum_j_N_ _alpha _j_y_i_x_j_x_i_b_leq 0] 更新权重

更新方法:

![\\alpha \_\{i\}+\\eta ->\\alpha \_\{i\}][alpha _i_eta -_alpha _i]

![b+\\eta y\_\{i\}->b][b_eta y_i_-_b]

(4)计算得到![\\alpha][alpha], b

带入到![\\alpha][alpha]x+b=0求出w

代码如下，如有错误请告知谢谢

import numpy as np
    
    
    class percePtron:
        def __init__(self, data):
            self.weight = np.array([0.0, 0.0])
            self.bias = 0.0
            self.data = data
    
        def train(self, learning_rate = 1.):
            num_change = 0
            for epoch in range(10):
                print('训练第{0}次'.format(epoch+1))
                for input in data:
                    pre = input[2]*(input[0]*self.weight[0] + input[1]*self.weight[1] + self.bias)
                    if pre <= 0:
                        num_change += 1
                        print('误差为:', pre)
    
                        self.weight[0] = self.weight[0] + input[2] *input[0] *  learning_rate
                        self.weight[1] = self.weight[1] + input[2] *input[1] * learning_rate
                        self.bias = self.bias + input[2] * learning_rate
                        print('-------第{0}次更新权重'.format(num_change))
                        print("weight : ", self.weight)
                        print("bias : ", self.bias)
    
    
        def predict(self, input):
            out =input[0]*self.weight[0] + input[1]*self.weight[1] + self.bias
            if out >=0 :
                return 1
            else:
                return -1
    
    
    class percePtron_Dual:
        def __init__(self, data):
            self.weight = np.array([0.0, 0.0])
            self.bias = 0.0
            self.data = data
    
        def train(self, learning_rate = 1.):
            train_data = self.data[:,:-1]
            train_label = self.data[:,-1:]
            m = train_data.shape[0]
            alpha = np.zeros(m)
            gram = np.zeros((m, m))
            for i in range(m):
                for j in range(m):
                    gram[i][j] = train_data[i] * np.mat(train_data[j]).transpose()
    
            # print(gram)
            for i in range(100):
                for idx in range(m):
                    tmp = 0
                    for j in range(m):
                        tmp += alpha[j] * train_label[j] * gram[j, idx]
                    tmp += self.bias
                    if (train_label[idx] * tmp <= 0):
                        alpha[idx] = alpha[idx] + learning_rate
                        self.bias = self.bias + learning_rate * train_label[idx]
                # print(alpha, self.bias)
    
            for num in range(m):
                self.weight += alpha[num]*train_data[num]*train_label[num]
    
    
        def predict(self, input):
            out =self.weight[0]*input[0]+self.weight[1]*input[1] -3
            if out >=0 :
                return 1
            else:
                return -1
    
    
    if __name__ == "__main__":
        data = np.array([[3, 3, 1], [4, 3, 1], [1, 1, -1]])
    
        per = percePtron(data)
        per.train()
        result_ = per.predict(input=[1, 0])
        print(result_)
    
        per_d = percePtron_Dual(data)
        per_d.train()
        result = per_d.predict(input=[1, 0])
        print(result)

[f_x_ _ sign_w_x_b]: https://private.codecogs.com/gif.latex?f%28x%29%20%3D%20sign%28w*x&plus;b%29
[w_eta _y_x-_w]: https://private.codecogs.com/gif.latex?w&plus;%5Ceta%20*y*x-%3Ew
[b_eta _y-_b]: https://private.codecogs.com/gif.latex?b&plus;%5Ceta%20*y-%3Eb
[eta]: https://private.codecogs.com/gif.latex?%5Ceta
[https_zhuanlan.zhihu.com_p_105470826]: https://zhuanlan.zhihu.com/p/105470826
[y_i_ _sum_j_N_ _alpha _j_y_i_x_j_x_i_b_leq 0]: https://private.codecogs.com/gif.latex?y_%7Bi%7D%28%20%5Csum_%7Bj%7D%5E%7BN%7D%20%5Calpha%20_%7Bj%7Dy_%7Bi%7Dx_%7Bj%7Dx_%7Bi%7D&plus;b%29%5Cleq%200
[alpha _i_eta -_alpha _i]: https://private.codecogs.com/gif.latex?%5Calpha%20_%7Bi%7D&plus;%5Ceta%20-%3E%5Calpha%20_%7Bi%7D
[b_eta y_i_-_b]: https://private.codecogs.com/gif.latex?b&plus;%5Ceta%20y_%7Bi%7D-%3Eb
[alpha]: https://private.codecogs.com/gif.latex?%5Calpha