ML：基于自定义数据集利用Logistic、梯度下降算法GD、LoR逻辑回归、Perceptron感知器、SVM支持向量机、LDA线性判别分析算法进行二分类预测(决策边界可视化)

朱雀 2022-12-25 12:53 68阅读 0赞

ML：基于自定义数据集利用Logistic、梯度下降算法GD、LoR逻辑回归、Perceptron感知器、支持向量机(SVM\_Linear、SVM\_Rbf)、LDA线性判别分析算法进行二分类预测(决策边界可视化)

**目录**

基于自定义数据集利用Logistic、梯度下降算法GD、LoR逻辑回归、Perceptron感知器、支持向量机(SVM\_Linear、SVM\_Rbf)、LDA线性判别分析算法进行二分类预测(决策边界可视化)

设计思路

输出结果

核心代码

--------------------

**相关文章**  
[ML：基于自定义数据集利用Logistic、梯度下降算法GD、LoR逻辑回归、Perceptron感知器、SVM支持向量机、LDA线性判别分析算法进行二分类预测(决策边界可视化)][ML_Logistic_GD_LoR_Perceptron_SVM_LDA]  
[ML：基于自定义数据集利用Logistic、梯度下降算法GD、LoR逻辑回归、Perceptron感知器、SVM支持向量机、LDA线性判别分析算法进行二分类预测(决策边界可视化)实现][ML_Logistic_GD_LoR_Perceptron_SVM_LDA 1]

# 基于自定义数据集利用Logistic、梯度下降算法GD、LoR逻辑回归、Perceptron感知器、支持向量机(SVM\_Linear、SVM\_Rbf)、LDA线性判别分析算法进行二分类预测(决策边界可视化) #

## 设计思路 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70][]

## 输出结果 ##

w_target.shape:  (3,) [ 1.17881511 -5.13265596 -6.55556511]
    Pre_Logistic_function 
     <class 'function'>
    Product_x_function 
     [1.         0.10262954 0.43893794]
    data_x 
     (300, 3) [[ 1.         -0.15378708  0.9615284 ]
     [ 1.          0.36965948 -0.0381362 ]
     [ 1.         -0.21576496 -0.31364397]
     [ 1.          0.45809941 -0.12285551]
     [ 1.         -0.88064421 -0.20391149]]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 1][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 2][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 3][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 4][]

## 核心代码 ##

def gradient_descent(data_x, data_y, w_h=None, eta=1.0, max_iterations=10000, epsilon=0.001):
        if w_h == None:
            w_h = np.array([0.0 for i in range(data_x.shape[1])])
        w_h_i = [np.copy(w_h)]  
        
        for i in range(max_iterations):
            subset_indices = range(data_x.shape[0])
            grad_E_in = np.mean(np.tile(- data_y[subset_indices] /
                                            ( 1.0 + np.exp(data_y[subset_indices] * w_h.dot(data_x[subset_indices].T)) ),
                                        (data_x.shape[1], 1)).T *   data_x[subset_indices], axis=0)
            w_h -= eta * grad_E_in
            w_h_i.append(np.copy(w_h))
            if np.linalg.norm(grad_E_in) <= np.linalg.norm(w_h) * epsilon:
                break
        return np.array(w_h_i)
    
    
    LoR = linear_model.LogisticRegression()
    LoR.fit(data_x,data_y)
    y_train=LoR.predict(data_x)
    
    LoRpp_function = lambda z: LoR.predict_proba(z)[:,0]
    BG_Grid_BaseLoR = apply_to_fill(z_grid, LoRpp_function)
    
    full_N_fig = plot_dataset_and_hypothesis(3,data_x, data_y, xy_1, xy_2, BG_Grid_BaseLoR,title=r'LoR: Hypothesis, $N={:}$'.format(N))
    
    
    
    SVM_Linear = svm.SVC(kernel='linear')
    SVM_Linear.fit(data_x,data_y)
    
    SVM_LinearPre_function = lambda z: SVM_Linear.predict(z)
    BG_Grid_BaseSVM_Linear = apply_to_fill(z_grid, SVM_LinearPre_function)
    
    full_N_fig = plot_dataset_and_hypothesis(5,data_x, data_y, xy_1, xy_2, BG_Grid_BaseSVM_Linear, title=r'SVM_Linear: Hypothesis, $N={:}$'.format(N))

[ML_Logistic_GD_LoR_Perceptron_SVM_LDA]: https://yunyaniu.blog.csdn.net/article/details/110455073
[ML_Logistic_GD_LoR_Perceptron_SVM_LDA 1]: https://yunyaniu.blog.csdn.net/article/details/110455940
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70]: /images/20221120/6c786a4b2f4847f198b58c92599fdf7e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/2020120122231867.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20201201222356249.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20201201222434748.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20201201223010849.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4,size_16,color_FFFFFF,t_70