Scikit-learn：聚类clustering

待我称王封你为后i 2022-07-15 12:15 205阅读 0赞

http://[blog.csdn.net/pipisorry/article/details/53185758][blog.csdn.net_pipisorry_article_details_53185758]

## 不同聚类效果比较 ##

### sklearn不同聚类示例比较 ###

[![../\_images/sphx\_glr\_plot\_cluster\_comparison\_0011.png][.._images_sphx_glr_plot_cluster_comparison_0011.png]][.._images_sphx_glr_plot_cluster_comparison_0011.png_.._images_sphx_glr_plot_cluster_comparison_0011.png]

A comparison of the clustering algorithms in scikit-learn

### 不同聚类综述 ###

<table> 
 <thead> 
  <tr> 
   <th>Method name</th> 
   <th>Parameters</th> 
   <th>Scalability</th> 
   <th>Usecase</th> 
   <th>Geometry (metric used)</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#k-means" title="K-Means" rel="nofollow">K-Means</a></td> 
   <td>number of clusters</td> 
   <td>Very large <code>n_samples</code>, medium <code>n_clusters</code> with<a href="http://scikit-learn.org/stable/modules/clustering.html#mini-batch-kmeans" title="MiniBatch code" rel="nofollow">MiniBatch code</a></td> 
   <td>General-purpose, even cluster size, flat geometry, not too many clusters</td> 
   <td>Distances between points</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#affinity-propagation" title="Affinity propagation" rel="nofollow">Affinity propagation</a></td> 
   <td>damping, sample preference</td> 
   <td>Not scalable with n_samples</td> 
   <td>Many clusters, uneven cluster size, non-flat geometry</td> 
   <td>Graph distance (e.g. nearest-neighbor graph)</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#mean-shift" title="Mean-shift" rel="nofollow">Mean-shift</a></td> 
   <td>bandwidth</td> 
   <td>Not scalable with <code>n_samples</code></td> 
   <td>Many clusters, uneven cluster size, non-flat geometry</td> 
   <td>Distances between points</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#spectral-clustering" title="Spectral clustering" rel="nofollow">Spectral clustering</a></td> 
   <td>number of clusters</td> 
   <td>Medium <code>n_samples</code>, small <code>n_clusters</code></td> 
   <td>Few clusters, even cluster size, non-flat geometry</td> 
   <td>Graph distance (e.g. nearest-neighbor graph)</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#hierarchical-clustering" title="Ward hierarchical clustering" rel="nofollow">Ward hierarchical clustering</a></td> 
   <td>number of clusters</td> 
   <td>Large <code>n_samples</code> and <code>n_clusters</code></td> 
   <td>Many clusters, possibly connectivity constraints</td> 
   <td>Distances between points</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#hierarchical-clustering" title="Agglomerative clustering" rel="nofollow">Agglomerative clustering</a></td> 
   <td>number of clusters, linkage type, distance</td> 
   <td>Large <code>n_samples</code> and <code>n_clusters</code></td> 
   <td>Many clusters, possibly connectivity constraints, non Euclideandistances</td> 
   <td>Any pairwise distance</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#dbscan" title="DBSCAN" rel="nofollow">DBSCAN</a></td> 
   <td>neighborhood size</td> 
   <td>Very large <code>n_samples</code>, medium <code>n_clusters</code></td> 
   <td>Non-flat geometry, uneven cluster sizes</td> 
   <td>Distances between nearest points</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/mixture.html#mixture" title="Gaussian mixtures" rel="nofollow">Gaussian mixtures</a></td> 
   <td>many</td> 
   <td>Not scalable</td> 
   <td>Flat geometry, good for density estimation</td> 
   <td>Mahalanobis distances to centers</td> 
  </tr> 
  <tr> 
   <td><a href="http://scikit-learn.org/stable/modules/clustering.html#birch" title="Birch" rel="nofollow">Birch</a></td> 
   <td>branching factor, threshold, optional global clusterer.</td> 
   <td>Large <code>n_clusters</code> and <code>n_samples</code></td> 
   <td>Large dataset, outlier removal, data reduction.</td> 
   <td>Euclidean distance between points</td> 
  </tr> 
 </tbody> 
</table>

## kmeans原理 ##

sklearn kmeans原型：\[[sklearn.cluster][].KMeans[¶][Link 1]\]

*class *`sklearn.cluster.``KMeans`(*n\_clusters=8*, *\**, *init='k-means++'*, *n\_init=10*, *max\_iter=300*, *tol=0.0001*, *precompute\_distances='deprecated'*, *verbose=0*, *random\_state=None*, *copy\_x=True*, *n\_jobs='deprecated'*, *algorithm='auto'*)

Note: 默认的init方法实际是kmeans++。

**Attributes**

cluster\_centers\_ndarray of shape (n\_clusters, n\_features)  
Coordinates of cluster centers. If the algorithm stops before fully converging (see tol and max\_iter), these will not be consistent with labels\_.

labels\_ndarray of shape (n\_samples,)  
Labels of each point

inertia\_float  
Sum of squared distances of samples to their closest cluster center.

n\_iter\_int  
Number of iterations run.

**sklearn中kmeans的学习目标**

The K-means algorithm aims to choose centroids that minimise the inertia, or within-cluster sum-of-squares criterion:

![2021091819511224.png][]

\[[2.3. Clustering — scikit-learn 0.24.2 documentation][2.3. Clustering _ scikit-learn 0.24.2 documentation]\]

\[[聚类算法：kmeans][kmeans]\]

[皮皮blog][blog]

# sklearn实现聚类代码示例 #

## Kmeans聚类 ##

**kmeans代码1**

from sklearn.cluster import KMeans
    
    data = [[1, 2, 3], [4, 5, 6], [1, 3, 4], [5, 6, 7]]  # line_num * feature_num的二维数组
    num_clusters = 2
    km_cluster = KMeans(n_clusters=num_clusters, init='k-means++', n_init=10, verbose=1, tol=1e-6)
    #km_cluster = KMeans(n_clusters=num_clusters, init='random', n_init=1, verbose=1)
    km_cluster.fit(data)
    # print("labels:\n", km_cluster.labels_)
    print("cluster_centers:\n", km_cluster.cluster_centers_) #numpy.ndarray

cluster\_centers:  
 \[\[4.5 5.5 6.5\]  
 \[1.  2.5 3.5\]\]

**kmeans代码2**

\[[MachineLearning/Kmeans.py at master · pipilove/MachineLearning · GitHub][MachineLearning_Kmeans.py at master _ pipilove_MachineLearning _ GitHub]\]

## DBSCAN聚类 ##

def Dist(x, y):  
    from geopy import distance  
    return distance.vincenty(x, y).meters

import pickle, subprocess, pwd  
import pandas as pd  
import numpy as np  
import matplotlib.pyplot as plt  
from sklearn.cluster import DBSCAN

df = pd.read\_pickle(os.path.join(CWD, 'middlewares/df.pkl'))

ll = df\[\['longitude', 'latitude'\]\].values  
x, y = ll\[:, 0\], ll\[:, 1\]

print('starting dbsan...')  
dbscaner = DBSCAN(eps=DBSCAN\_R, min\_samples=DBSCAN\_MIN\_S, metric=Dist, n\_jobs=-1).fit(ll)  
pickle.dump(dbscaner, open(os.path.join(CWD, 'middlewares/dbscaner.pkl'), 'wb'))  
print('dbsan dumping end...')

dbscaner = pickle.load(open(os.path.join(CWD, 'middlewares/dbscaner.pkl'), 'rb'))  
labels = dbscaner.labels\_  
\# print(set(labels))  
colors = plt.cm.Spectral(np.linspace(0, 1, len(set(labels))))  
for k, col in zip(set(labels), colors):  
    marker = '.'  
    if k == -1:  
        col = 'k'  
        marker = 'x'  
    inds\_k = labels == k  
    plt.scatter(x\[inds\_k\], y\[inds\_k\], marker=marker, color=col)  
if pwd.getpwuid(os.geteuid()).pw\_name == 'pi':  
    plt.savefig('./1.png')  
elif pwd.getpwuid(os.geteuid()).pw\_name == 'pipi':  
    plt.show()

\[[DBSCAN][]\]

[皮皮blog][blog]

# 聚类模型评估 #

## **聚类趋势（聚类前）** ##

霍普金斯统计量（Hopkins Statistic）评估给定数据集是否存在有意义的可聚类的非随机结构。如果一个数据集是有随机的均匀的点生成的，虽然也可以产生聚类结果，但该结果没有意义。聚类的前提需要数据是**非均匀分布**的。该值在区间\[0, 1\]之间，\[0.01, 0.3\]表示数据结构regularly spaced，该值为0.5时数据是均匀分布的，\[0.7, 0.99\]表示聚类趋势很强。

\[[Python的Hopkins Statistic实现][Python_Hopkins Statistic]\]

## 无监督聚类评估方法（聚类后） ##

### 轮廓系数/Silhouette Coefficient[¶][Link 2] ###

较高的 Silhouette 系数得分和能够更好定义的聚类的模型相关联。Silhouette 系数 是依据每个样本进行定义，由两个得分组成:

*  a: 样本与同一类别中所有其他点之间的平均距离。
 *  b: 样本与 *下一个距离最近的簇* 中的所有其他点之间的平均距离。

然后将单个样本的 Silhouette 系数 *s* 给出为:

![s = \\frac\{b - a\}\{max(a, b)\}][s _ _frac_b - a_max_a_ b]

示例：

import numpy as np
    from sklearn.cluster import KMeans
    kmeans_model = KMeans(n_clusters=3, random_state=1).fit(X)
    labels = kmeans_model.labels_
    metrics.silhouette_score(X, labels, metric='euclidean')

### CH指数/Calinski-Harabasz Index[¶][Link 3] ###

亦称方差比准则(Variance Ratio Criterion)，较高的 Calinski-Harabaz 的得分和能够更好定义的聚类的模型相关联。

对于 k 簇，Calinski-Harabaz 得分是通过簇间色散平均值(between-clusters dispersion mean)与 簇内色散之间(within-cluster dispersion)的比值给出的

![s(k) = \\frac\{\\mathrm\{Tr\}(B\_k)\}\{\\mathrm\{Tr\}(W\_k)\} \\times \\frac\{N - k\}\{k - 1\}][s_k_ _ _frac_mathrm_Tr_B_k_mathrm_Tr_W_k_ _times _frac_N - k_k - 1]

其中 B\_k 是组间色散矩阵(between group dispersion matrix)， W\_k 是由以下定义的簇内色散矩阵(within-cluster dispersion matrix):

![W\_k = \\sum\_\{q=1\}^k \\sum\_\{x \\in C\_q\} (x - c\_q) (x - c\_q)^T][W_k _ _sum_q_1_k _sum_x _in C_q_ _x - c_q_ _x - c_q_T]

![B\_k = \\sum\_q n\_q (c\_q - c) (c\_q - c)^T][B_k _ _sum_q n_q _c_q - c_ _c_q - c_T]

示例：

import numpy as np
    from sklearn.cluster import KMeans
    kmeans_model = KMeans(n_clusters=3, random_state=1).fit(X)
    labels = kmeans_model.labels_
    metrics.calinski_harabasz_score(X, labels)

### Davies-Bouldin Index[¶][Link 4] ###

### Contingency Matrix[¶][Link 5] ###

\[[Clustering performance evaluation][]\]

\[[聚类 · sklearn 中文文档][_ sklearn]\]

\[[如何评价聚类结果的好坏？][Link 6]\]

from: [sklearn：聚类clustering][blog.csdn.net_pipisorry_article_details_53185758]

ref:

[blog.csdn.net_pipisorry_article_details_53185758]: http://blog.csdn.net/pipisorry/article/details/53185758
[.._images_sphx_glr_plot_cluster_comparison_0011.png]: https://img-blog.csdnimg.cn/img_convert/76430e56b301f04ebb256ab887ddf580.png
[.._images_sphx_glr_plot_cluster_comparison_0011.png_.._images_sphx_glr_plot_cluster_comparison_0011.png]: /images/20220715/ad86abb87a0145b6a4f7254d815fb33a.png
[sklearn.cluster]: https://scikit-learn.org/stable/modules/classes.html#module-sklearn.cluster
[Link 1]: https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn-cluster-kmeans
[2021091819511224.png]: /images/20220715/2c91e8ccbe644b2bbd79bc1591f72698.png
[2.3. Clustering _ scikit-learn 0.24.2 documentation]: https://scikit-learn.org/stable/modules/clustering.html#k-means
[kmeans]: https://youzipi.blog.csdn.net/article/details/41217279
[blog]: http://blog.csdn.net/pipisorry
[MachineLearning_Kmeans.py at master _ pipilove_MachineLearning _ GitHub]: https://github.com/pipilove/MachineLearning/blob/master/sim_cluster/Kmeans.py
[DBSCAN]: http://scikit-learn.org/stable/modules/clustering.html#dbscan
[Python_Hopkins Statistic]: https://link.zhihu.com/?target=https%3A//matevzkunaver.wordpress.com/2017/06/20/hopkins-test-for-cluster-tendency/
[Link 2]: https://scikit-learn.org/stable/modules/clustering.html#silhouette-coefficient
[s _ _frac_b - a_max_a_ b]: /images/20220715/4c798145a1c1445185abca32a08dbc84.png
[Link 3]: https://scikit-learn.org/stable/modules/clustering.html#calinski-harabasz-index
[s_k_ _ _frac_mathrm_Tr_B_k_mathrm_Tr_W_k_ _times _frac_N - k_k - 1]: /images/20220715/62465f7ea351490da9034ff0aca7a5f4.png
[W_k _ _sum_q_1_k _sum_x _in C_q_ _x - c_q_ _x - c_q_T]: /images/20220715/b8d4430c3ec047e88568156b02190352.png
[B_k _ _sum_q n_q _c_q - c_ _c_q - c_T]: /images/20220715/51604647f6884c3ca7b166a0dd8cdfbc.png
[Link 4]: https://scikit-learn.org/stable/modules/clustering.html#davies-bouldin-index
[Link 5]: https://scikit-learn.org/stable/modules/clustering.html#contingency-matrix
[Clustering performance evaluation]: https://scikit-learn.org/stable/modules/clustering.html#clustering-performance-evaluation
[_ sklearn]: https://sklearn.apachecn.org/docs/master/22.html
[Link 6]: https://www.zhihu.com/question/19635522