sklearn学习（二）

深碍√TFBOYSˉ_ 2022-05-14 02:20 237阅读 0赞

## 学习网站 ##

> [http://scikit-learn.org/stable/tutorial/statistical\_inference/index.html][http_scikit-learn.org_stable_tutorial_statistical_inference_index.html]

## Statistical learning: the setting and the estimator object in scikit-learn ##

通过下面代码：

from sklearn import datasets
    
    iris = datasets.load_iris()
    print(iris.DESCR)

可以获取到对应的数据描述：

Iris Plants Database
    ==================== 
    Notes
    ----- Data Set Characteristics:
        :Number of Instances: 150 (50 in each of three classes)
        :Number of Attributes: 4 numeric, predictive attributes and the class
        :Attribute Information:
            - sepal length in cm         - sepal width in cm         - petal length in cm         - petal width in cm         - class:                 - Iris-Setosa                 - Iris-Versicolour                 - Iris-Virginica     :Summary Statistics:
    
        ============== ==== ==== ======= ===== ====================                     Min  Max   Mean    SD   Class Correlation
        ============== ==== ==== ======= ===== ====================     sepal length:   4.3  7.9   5.84   0.83    0.7826
        sepal width:    2.0  4.4   3.05   0.43   -0.4194
        petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)
        petal width:    0.1  2.5   1.20  0.76     0.9565  (high!)
        ============== ==== ==== ======= ===== ==================== 
        :Missing Attribute Values: None
        :Class Distribution: 33.3% for each of 3 classes.
        :Creator: R.A. Fisher
        :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)
        :Date: July, 1988
    
    This is a copy of UCI ML iris datasets.
    http://archive.ics.uci.edu/ml/datasets/Iris
    
    The famous Iris database, first used by Sir R.A Fisher
    
    This is perhaps the best known database to be found in the
    pattern recognition literature.  Fisher's paper is a classic in the field and
    is referenced frequently to this day.  (See Duda & Hart, for example.)  The
    data set contains 3 classes of 50 instances each, where each class refers to a
    type of iris plant.  One class is linearly separable from the other 2; the
    latter are NOT linearly separable from each other.
    
    References
    ----------    - Fisher,R.A. "The use of multiple measurements in taxonomic problems"      Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to
         Mathematical Statistics" (John Wiley, NY, 1950).
       - Duda,R.O., & Hart,P.E. (1973) Pattern Classification and Scene Analysis.      (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.
       - Dasarathy, B.V. (1980) "Nosing Around the Neighborhood: A New System      Structure and Classification Rule for Recognition in Partially Exposed
         Environments".  IEEE Transactions on Pattern Analysis and Machine
         Intelligence, Vol. PAMI-2, No. 1, 67-71.
       - Gates, G.W. (1972) "The Reduced Nearest Neighbor Rule". IEEE Transactions      on Information Theory, May 1972, 431-433.
       - See also: 1988 MLC Proceedings, 54-64. Cheeseman et al"s AUTOCLASS II      conceptual clustering system finds 3 classes in the data.
       - Many, many more ...

## 数据长度处理 ##

一般来说，sklearn里面的数据都是 (`n_samples`, `n_features`)这样的样子。  
前面的那个数值表示的是有多少个样本。第二个表示有多少个特征。

from sklearn import datasets
    iris = datasets.load_iris()
    print(iris.data.shape)

**输出：**

(150, 4)

但是有些数据其实不是这样的格式的。那就需要处理一下。

比如，digits数据

from sklearn import datasets
    
    digits = datasets.load_digits()
    print(digits.data.shape)
    print(digits.images.shape)

输出是：

(1797, 64)
    (1797, 8, 8)

其实对于后者还有别的操作。

from sklearn import datasets
    
    digits = datasets.load_digits()
    data = digits.images.reshape((digits.images.shape[0], -1))
    print(data.shape)

输出：

(1797, 64)

`-1`表示这个自己计算出来，前面那个数据就是保证了样本数目不变的情况下。

## 监督学习：预测一个输出，通过高维的观察 ##

监督学习：包括学习一种在两个数据集之间。观测值 X和额外的一个变量y。 一般来说，y都是一个长度为`n_samples` 的一维数组。

## 最近邻和维度灾难 ##

**the curse of dimensionality（维度灾难）：**  
[https://zh.wikipedia.org/wiki/%E7%BB%B4%E6%95%B0%E7%81%BE%E9%9A%BE][https_zh.wikipedia.org_wiki_E7_BB_B4_E6_95_B0_E7_81_BE_E9_9A_BE]

简单来说，为了描述的一个低维的问题，需要保持一定的精度，然后就需要一定数量的采样点。但是随着维度的提升，那么需要的数据其实的指数增长的。

## Iris DataSet数据的处理 ##

根据之前的描述，可以看出。这里有三种iris（鸾尾花）。（Setosa, Versicolour, and Virginica）

![这里写图片描述][70]  
![这里写图片描述][70 1]

from sklearn import datasets
    import matplotlib.pyplot as plt
    from mpl_toolkits.mplot3d import Axes3D
    from sklearn.decomposition import PCA
    
    iris = datasets.load_iris()
    X = iris.data[:, :2]  # take the first two features
    y = iris.target
    
    x_min, x_max = X[:, 0].min() - 0.5, X[:, 0].max() + 0.5
    y_min, y_max = X[:, 1].min() - 0.5, X[:, 1].max() + 0.5
    
    fontsize = 15
    
    # 2D
    plt.figure(1, figsize=(8, 6))
    plt.clf()  # clear current figure
    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Set1, edgecolors='k')
    plt.xlabel('Sepal length', fontsize=fontsize)
    plt.ylabel('Sepal width', fontsize=fontsize)
    plt.xlim(x_min, x_max)
    plt.ylim(y_min, y_max)
    plt.xticks([])
    plt.yticks([])
    
    # 3D
    fig = plt.figure(2, figsize=(8, 6))
    ax = Axes3D(fig, elev=-150, azim=110)  # azim调整沿z轴旋转度数 elev表示的看的角度
    # 选前3个最相关
    X_reduced = PCA(n_components=3).fit_transform(iris.data)
    ax.scatter(X_reduced[:, 0], X_reduced[:, 1], X_reduced[:, 2], c=y, cmap=plt.cm.Set1, edgecolors='k', s=40)  # s表示散点大小
    ax.set_title("First three PCA directions", fontsize=fontsize)
    ax.set_xlabel("1st eigenvector", fontsize=fontsize)
    ax.w_xaxis.set_ticklabels([])
    ax.set_ylabel("2nd eigenvector", fontsize=fontsize)
    ax.w_yaxis.set_ticklabels([])
    ax.set_zlabel("3rd eigenvector", fontsize=fontsize)
    ax.w_zaxis.set_ticklabels([])
    
    plt.savefig('2.png')
    plt.show()

看起来上面的代码很复杂， 其实真实做了操作的，其实做的工作非常简单。甚至是特别的多的重复。

**说到最近邻，其实最简单的还是knn，也就是k最近邻算法：**  
最近邻，其实可以说是ML中最简单的一类算法了

**可能会需要的解释：**

[关于knn的weights][knn_weights]  
[关于np.c\_函数][np.c]

![这里写图片描述][70 2]

![这里写图片描述][70 3]

## 代码 ##

（copyfrom [http://scikit-learn.org/stable/auto\_examples/neighbors/plot\_classification.html][http_scikit-learn.org_stable_auto_examples_neighbors_plot_classification.html]）

解释一波：

*  关于那个权重的问题。如果是`uniform`表示的是，所有点在邻居之间是相等的。 `distance`表示关于距离的逆。[https://zhuanlan.zhihu.com/p/23191325][https_zhuanlan.zhihu.com_p_23191325]，最简单的解释就是假如采用的是5-nn，如果前面两个是A，后面三个是B，那么如果是等权的话，这里就应该是选B，但是如果是考虑距离的逆的话，越近，当然数值就会越大。那么很有可能就是选A了，因为A更近。
 *  meshgrid是构建网格数据，这个在matlab中也比较常见。
 *  np.c\_其实列项相连接的操作。比如说是A = \[1, 2\], B = \[3,4\], np.c\_(A, B) = \[\[1, 3\], \[2, 4\]\]

import numpy as np
    import matplotlib.pyplot as plt
    from matplotlib.colors import ListedColormap
    from sklearn import neighbors, datasets
    
    n_neighbors = 15
    
    # import some data to play with
    iris = datasets.load_iris()
    
    # we only take the first two features. We could avoid this ugly
    # slicing by using a two-dim dataset
    X = iris.data[:, :2]
    y = iris.target
    
    h = .02  # step size in the mesh
    
    # Create color maps
    cmap_light = ListedColormap(['#FFAAAA', '#AAFFAA', '#AAAAFF'])
    cmap_bold = ListedColormap(['#FF0000', '#00FF00', '#0000FF'])
    
    for weights in ['uniform', 'distance']:
        # we create an instance of Neighbours Classifier and fit the data.
        clf = neighbors.KNeighborsClassifier(n_neighbors, weights=weights)
        clf.fit(X, y)
    
        # Plot the decision boundary. For that, we will assign a color to each
        # point in the mesh [x_min, x_max]x[y_min, y_max].
        x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
        y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
        xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                             np.arange(y_min, y_max, h))
        Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
    
        # Put the result into a color plot
        Z = Z.reshape(xx.shape)
        plt.figure()
        # 背景染色
        plt.pcolormesh(xx, yy, Z, cmap=cmap_light)
    
        # Plot also the training points
        plt.scatter(X[:, 0], X[:, 1], c=y, cmap=cmap_bold,
                    edgecolor='k', s=20)
        plt.xlim(xx.min(), xx.max())
        plt.ylim(yy.min(), yy.max())
        plt.title("3-Class classification (k = %i, weights = '%s')"
                  % (n_neighbors, weights))
    
    plt.show()

KNN算法使用

选取除了后面的10个以外的作为训练集，后10个作为测试集合

import numpy as np
    from sklearn import neighbors, datasets
    
    n_neighbors = 15
    iris = datasets.load_iris()
    np.random.seed(0)
    iris_X = iris.data
    iris_y = iris.target
    indices = np.random.permutation(len(iris_X))
    iris_X_train = iris_X[indices[:-10]]
    iris_y_train = iris_y[indices[:-10]]
    iris_X_test = iris_X[indices[-10:]]
    iris_y_test = iris_y[indices[-10:]]
    
    knn =neighbors.KNeighborsClassifier(n_neighbors=n_neighbors)
    knn.fit(iris_X_train, iris_y_train)
    print(knn.predict(iris_X_test))
    print(iris_y_test)

输出：（预测结果和实际结果对比）

[1 2 1 0 0 0 2 1 2 0]
    [1 1 1 0 0 0 2 1 2 0]

当n的数目提高的之后。  
**n\_neighbors = 26时，**，所得到的结果就是完全正确的了。（k的数值提高，表明考虑的更全面，理论上讲是更好的。虽然也不能选太过了）

但是，当权重选为`distance`时候，却是在**n\_neighbors = 40时**，才完全正确。

## 线性模型 ##

**线性回归：** 符合线性模型。通过调整参数使得整个残差的平方和（the sum of the squared residuals）达到尽可能的小。

from sklearn import linear_model
    from sklearn import datasets
    import numpy as np
    
    diabetes = datasets.load_diabetes()
    
    N_test = 20
    diabetes_X_train = diabetes.data[:-N_test]
    diabetes_X_test = diabetes.data[-N_test:]
    diabetes_y_train = diabetes.target[:-N_test]
    diabetes_y_test = diabetes.target[-N_test:]
    
    regr = linear_model.LinearRegression()
    regr.fit(diabetes_X_train, diabetes_y_train)
    print(regr.coef_)
    
    mean_residual = np.mean((regr.predict(diabetes_X_test) - diabetes_y_test) ** 2)
    print(mean_residual)
    print(regr.score(diabetes_X_test, diabetes_y_test))

输出:

[ 3.03499549e-01 -2.37639315e+02  5.10530605e+02  3.27736980e+02
     -8.14131709e+02 4.92814588e+02 1.02848452e+02 1.84606489e+02   7.43519617e+02  7.60951722e+01]
    2004.5676026898218
    0.5850753022690572

**用线性回归来预测生理数据：**

由于数据维度较高，所以，就先用主成分分析法挖掘出主要成分中的两个。之后构建的一个平面来（三维的线性回归）

![这里写图片描述][70 4]

from sklearn import datasets
    import matplotlib.pyplot as plt
    from mpl_toolkits.mplot3d import Axes3D
    from sklearn.decomposition import PCA
    from sklearn import linear_model
    import numpy as np
    
    h = 100
    
    diabetes = datasets.load_diabetes()
    # 选前2个最相关(主成分分析)
    X_reduced = PCA(n_components=2).fit_transform(diabetes.data)
    regr = linear_model.LinearRegression()
    regr.fit(X_reduced, diabetes.target)
    
    x_min, x_max = min(X_reduced[:, 0]), max(X_reduced[:, 0])
    y_min, y_max = min(X_reduced[:, 1]), max(X_reduced[:, 1])
    xx, yy = np.meshgrid(np.linspace(x_min, x_max, h),
                         np.linspace(y_min, y_max, h))
    Z = regr.predict(np.c_[xx.ravel(), yy.ravel()])
    
    xx = xx.ravel()
    yy = yy.ravel()
    Z = Z.ravel()
    fig = plt.figure(1, figsize=(8, 6))
    ax = Axes3D(fig, elev=30, azim=-30)
    plt.xlim(x_min, x_max)
    plt.ylim(y_min, y_max)
    
    ax.scatter(X_reduced[:, 0], X_reduced[:, 1], diabetes.target, color='k')
    
    ax.plot(xx, yy, Z)
    plt.show()

> [http://scikit-learn.org/stable/auto\_examples/linear\_model/plot\_ols.html][http_scikit-learn.org_stable_auto_examples_linear_model_plot_ols.html]  
> 这里有一个二维平面的代码

**扰动，收缩变换：**  
![这里写图片描述][70 5]

其实就是，在0.5 和 1这两个参数点上，各添加一个扰动（这个扰动是服从正态分布的）。之后用这两个点上的数据，来做拟合（由于每次只考虑两个点，所以所得直线一定会穿过两点。）

**代码：**

import matplotlib.pyplot as plt
    from sklearn import linear_model
    import numpy as np
    X = np.c_[.5, 1].T
    y = [.5, 1]
    test = np.c_[0, 2].T
    regr = linear_model.LinearRegression()
    plt.figure()
    
    np.random.seed(0)
    for _ in range(6):
        this_X = .1 * np.random.normal(size=(2, 1)) + X
        regr.fit(this_X, y)
        # 描点 画线
        plt.plot(test, regr.predict(test))
        plt.scatter(this_X, y, s=3)
    
    plt.show()

## 线性模型—Ridge模型 ##

> This model solves a regression model where the loss function is the linear least squares function and regularization is given by the l2-norm. Also known as Ridge Regression or Tikhonov regularization. This estimator has built-in support for multi-variate regression (i.e., when y is a 2d-array of shape \[n\_samples, n\_targets\]).

在拟合的过程中，很容易出现过拟合的现象，所以，ridge模型，就提出添加一个l2正则化。在保证了线性残差平方较小的同时，也需要保证这个l2正则别太大。

总体来说，其实为了实现避免过拟合的问题。

**重写之前的三维模型：**  
其实就是该用rigde模型。

![这里写图片描述][70 6]

在保证其他都不变的情况下，可以看出这个会更向上翘起来了点。  
特别是在保证alpha的数值越来越高的时候，翘的会更明显。（不过，设置alpha值太高了的话，效果也不一定很好）

from sklearn import datasets
    import matplotlib.pyplot as plt
    from mpl_toolkits.mplot3d import Axes3D
    from sklearn.decomposition import PCA
    from sklearn import linear_model
    import numpy as np
    
    h = 100
    
    diabetes = datasets.load_diabetes()
    # 选前2个最相关(主成分分析)
    X_reduced = PCA(n_components=2).fit_transform(diabetes.data)
    reg = linear_model.Ridge(alpha=.3)
    reg.fit(X_reduced, diabetes.target)
    
    x_min, x_max = min(X_reduced[:, 0]), max(X_reduced[:, 0])
    y_min, y_max = min(X_reduced[:, 1]), max(X_reduced[:, 1])
    xx, yy = np.meshgrid(np.linspace(x_min, x_max, h),
                         np.linspace(y_min, y_max, h))
    Z = reg.predict(np.c_[xx.ravel(), yy.ravel()])
    
    xx = xx.ravel()
    yy = yy.ravel()
    Z = Z.ravel()
    fig = plt.figure(1, figsize=(8, 6))
    ax = Axes3D(fig, elev=30, azim=-30)
    plt.xlim(x_min, x_max)
    plt.ylim(y_min, y_max)
    
    ax.scatter(X_reduced[:, 0], X_reduced[:, 1], diabetes.target, color='k')
    
    ax.plot(xx, yy, Z)
    plt.show()

> Note Capturing in the fitted parameters noise that prevents the model to generalize to new data is called overfitting. The bias introduced by the ridge regression is called a regularization.

捕捉到了匹配的系数噪声，而阻止整个模型的推广到新数据，就是过拟合。  
做了一个偏差在ridge回归中，叫做正则化。

**Sparsity稀疏：**

为了调高问题的一些条件（比如环节维度诅咒），这会很有趣的去只选取一些有用的信息。另外的一种惩罚方式，叫做 Lasso方法（least absolute shrinkage and selection operator），会设置一些系数为0。

先尝试找到最好的alpha，之后，再用这样的方式，去拟合。最后给出在测试集合上的打分。

from sklearn import datasets
    import matplotlib.pyplot as plt
    from mpl_toolkits.mplot3d import Axes3D
    from sklearn.decomposition import PCA
    from sklearn import linear_model
    import numpy as np
    
    diabetes = datasets.load_diabetes()
    diabetes_X_train = diabetes.data[:-20]
    diabetes_X_test = diabetes.data[-20:]
    diabetes_y_train = diabetes.target[:-20]
    diabetes_y_test = diabetes.target[-20:]
    
    regr = linear_model.Lasso()
    
    alphas = np.logspace(-4, -1, 6)
    
    # choose the best alpha
    scores = [regr.set_params(alpha=alpha
                              ).fit(diabetes_X_train, diabetes_y_train
                                    ).score(diabetes_X_test, diabetes_y_test)
              for alpha in alphas]
    best_alpha = alphas[scores.index(max(scores))]
    
    # set the best alpha
    regr.alpha = best_alpha
    regr.fit(diabetes_X_train, diabetes_y_train)
    print(regr.coef_)
    print(regr.score(diabetes_X_test, diabetes_y_test))

输出结果是：

[   0.         -212.43764548  517.19478111  313.77959962 -160.8303982
       -0. -187.19554705 69.38229038 508.66011217 71.84239008] 0.5887622418309261

同样换作用Lasso来做下三维图。（最优化alpha之后）

![这里写图片描述][70 7]

from sklearn import datasets
    import matplotlib.pyplot as plt
    from mpl_toolkits.mplot3d import Axes3D
    from sklearn.decomposition import PCA
    from sklearn import linear_model
    import numpy as np
    
    h = 100
    
    diabetes = datasets.load_diabetes()
    # 选前2个最相关(主成分分析)
    X_reduced = PCA(n_components=2).fit_transform(diabetes.data)
    regr = linear_model.Lasso()
    regr.fit(X_reduced, diabetes.target)
    
    alphas = np.logspace(-4, -1, 6)
    
    # choose the best alpha
    scores = [regr.set_params(alpha=alpha
                              ).fit(X_reduced, diabetes.target
                                    ).score(X_reduced, diabetes.target)
              for alpha in alphas]
    best_alpha = alphas[scores.index(max(scores))]
    
    # set the best alpha
    regr.alpha = best_alpha
    
    
    
    x_min, x_max = min(X_reduced[:, 0]), max(X_reduced[:, 0])
    y_min, y_max = min(X_reduced[:, 1]), max(X_reduced[:, 1])
    xx, yy = np.meshgrid(np.linspace(x_min, x_max, h),
                         np.linspace(y_min, y_max, h))
    Z = regr.predict(np.c_[xx.ravel(), yy.ravel()])
    
    xx = xx.ravel()
    yy = yy.ravel()
    Z = Z.ravel()
    fig = plt.figure(1, figsize=(8, 6))
    ax = Axes3D(fig, elev=30, azim=-30)
    plt.xlim(x_min, x_max)
    plt.ylim(y_min, y_max)
    
    ax.scatter(X_reduced[:, 0], X_reduced[:, 1], diabetes.target, color='k')
    
    ax.plot(xx, yy, Z)
    plt.show()

**不同的算法来解决同一个问题：**  
不同的算法可以被用来解决同一个数学问题。例如，Lasso可以通过使用坐标下降的昂发来来解决Lasso算法问题，这在大数据上是非常有效的。但是LassoLars通过使用LARS算法，适合来处理权重向量是特别稀疏的情况。（观测点较少的情况）

## 分类 ##

为了分析，例如在标记过的鸾尾花iris，线性回归不是最好方法。因为，它会给套多权重给那些原理决策前言的数据。  
（比如，让线性回归去拟合逻辑回归方程）

逻辑回归的C还是为了避免过拟合而提出的一种系数问题。

import numpy as np
    from sklearn import linear_model, datasets
    
    iris = datasets.load_iris()
    np.random.seed(0)
    iris_X = iris.data
    iris_y = iris.target
    indices = np.random.permutation(len(iris_X))
    iris_X_train = iris_X[indices[:-10]]
    iris_y_train = iris_y[indices[:-10]]
    iris_X_test = iris_X[indices[-10:]]
    iris_y_test = iris_y[indices[-10:]]
    logistic = linear_model.LogisticRegression(C=1)
    logistic.fit(iris_X_train, iris_y_train)
    print(logistic.predict(iris_X_test))
    print(iris_y_test)

输出：

[1 2 1 0 0 0 2 1 2 0]
    [1 1 1 0 0 0 2 1 2 0]

画平面多色图：

![这里写图片描述][70 8]

> copy from  
> [http://scikit-learn.org/stable/auto\_examples/linear\_model/plot\_iris\_logistic.html][http_scikit-learn.org_stable_auto_examples_linear_model_plot_iris_logistic.html]

就是只用前两个变量作为特征来画图。  
其他的跟之前的那个没有太大的区别。

import numpy as np
    import matplotlib.pyplot as plt
    from sklearn import linear_model, datasets
    
    # import some data to play with
    iris = datasets.load_iris()
    X = iris.data[:, :2]  # we only take the first two features.
    Y = iris.target
    
    h = .02  # step size in the mesh
    
    logreg = linear_model.LogisticRegression(C=1e5)
    
    # we create an instance of Neighbours Classifier and fit the data.
    logreg.fit(X, Y)
    
    # Plot the decision boundary. For that, we will assign a color to each
    # point in the mesh [x_min, x_max]x[y_min, y_max].
    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
    Z = logreg.predict(np.c_[xx.ravel(), yy.ravel()])
    
    # Put the result into a color plot
    Z = Z.reshape(xx.shape)
    plt.figure(1, figsize=(4, 3))
    plt.pcolormesh(xx, yy, Z, cmap=plt.cm.Paired)
    
    # Plot also the training points
    plt.scatter(X[:, 0], X[:, 1], c=Y, edgecolors='k', cmap=plt.cm.Paired)
    plt.xlabel('Sepal length')
    plt.ylabel('Sepal width')
    
    plt.xlim(xx.min(), xx.max())
    plt.ylim(yy.min(), yy.max())
    plt.xticks(())
    plt.yticks(())
    
    plt.show()

多类分类，其实本质上还是先逐步分出一个类来。

> The C parameter controls the amount of regularization in the LogisticRegression object: a large value for C results in less regularization. penalty=”l2” gives Shrinkage (i.e. non-sparse coefficients), while penalty=”l1” gives Sparsity.

C参数，控制的是正则化，C越大正则化越小。

penalty是12，表示的是波动范围（非稀疏的系数）。  
11表示的，稀疏度。

## 支持向量机（SVM） ##

**线性SVM（Linear SVMs）**

支持向量机模型，属于判别式模型家族。他们尝试去找到样本之间的联系，去建立一个平面最大化两个类的边界。

正则化，是通过C参数来设置的。小的C值，意味着边界计算时候用更多观测数据（在分界线附近）。越大的C值，意味着边界的考量用更接近分界线上的点。

import numpy as np
    from sklearn import datasets
    from sklearn import svm
    
    iris = datasets.load_iris()
    np.random.seed(0)
    iris_X = iris.data
    iris_y = iris.target
    indices = np.random.permutation(len(iris_X))
    iris_X_train = iris_X[indices[:-10]]
    iris_y_train = iris_y[indices[:-10]]
    iris_X_test = iris_X[indices[-10:]]
    iris_y_test = iris_y[indices[-10:]]
    
    
    svc = svm.SVC(kernel='linear')
    svc.fit(iris_X_train, iris_y_train)
    print(svc.predict(iris_X_test))
    print(iris_y_test)
    
    print(svc.score(iris_X_test, iris_y_test))

**警告：**在很多评估器中，包括SVM的时候，需要讲过将数据集给标准化之后，才能更好的得到好的预测结果。

[http_scikit-learn.org_stable_tutorial_statistical_inference_index.html]: http://scikit-learn.org/stable/tutorial/statistical_inference/index.html
[https_zh.wikipedia.org_wiki_E7_BB_B4_E6_95_B0_E7_81_BE_E9_9A_BE]: https://zh.wikipedia.org/wiki/%E7%BB%B4%E6%95%B0%E7%81%BE%E9%9A%BE
[70]: /images/20220514/8060f5ed9815459296ea8c01015f46fe.png
[70 1]: /images/20220514/55ad89ae7beb46a2a2fc4af54f4c1b9f.png
[knn_weights]: http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html
[np.c]: https://blog.csdn.net/Together_CZ/article/details/79548217
[70 2]: /images/20220514/837b91ecd3b84b7087a5fd1119ddc71c.png
[70 3]: /images/20220514/be59c5aff19b4139b89c2ff76ec647a3.png
[http_scikit-learn.org_stable_auto_examples_neighbors_plot_classification.html]: http://scikit-learn.org/stable/auto_examples/neighbors/plot_classification.html
[https_zhuanlan.zhihu.com_p_23191325]: https://zhuanlan.zhihu.com/p/23191325
[70 4]: /images/20220514/621ef05c555a4853b5b121775acb5128.png
[http_scikit-learn.org_stable_auto_examples_linear_model_plot_ols.html]: http://scikit-learn.org/stable/auto_examples/linear_model/plot_ols.html
[70 5]: /images/20220514/2103dfcd06d745ee810829f8d3493ae6.png
[70 6]: /images/20220514/59b80ec2e1bf435f94c6d6d31dc0bc65.png
[70 7]: /images/20220514/9e932444692d45769c3eb0482f5e3e4a.png
[70 8]: /images/20220514/4e5b54b7e92842bab71b6d3905201be1.png
[http_scikit-learn.org_stable_auto_examples_linear_model_plot_iris_logistic.html]: http://scikit-learn.org/stable/auto_examples/linear_model/plot_iris_logistic.html