【one vs one】基于svm的二分类器实现多分类

你的名字 2023-01-21 15:23 368阅读 0赞

文章目录

  • 一、教案版本代码解析
    • 1、整体代码
    • 2、逐段分析
      • （1）输入可视化
      • （2） train_OvO分段解析
      • （3）test_OvO 分段解析
      • （4）score_svm函数解析
  • 二、20-21题，加入score_svm后的分类
    • 1、整体代码
    • 2、逐段解析
      • （1）test_OvO分析
      • （2）对我作出的更改进行测试

一、教案版本代码解析

1、整体代码

  1. import numpy as np
  2. from sklearn.svm import LinearSVC
  3. from sklearn.datasets import load_digits
  4. from sklearn.model_selection import train_test_split
  5. from sklearn.metrics import confusion_matrix
  6. import matplotlib.pyplot as plt
  7. import numpy as np
  8. import scipy as sp
  9. import warnings
  11. warnings.filterwarnings('ignore')
  14. # unit test utilities: you can ignore these function
  15. def is_approximately_equal(test, target, eps=1e-2):
  16.     return np.mean(np.fabs(np.array(test) - np.array(target))) < eps
  19. def assert_test_equality(test, target):
  20.     assert is_approximately_equal(test, target), 'Expected:\n %s \nbut got:\n %s' % (target, test)
  23. def train_svm(X_train, y_train, param):
  24.     est = LinearSVC(C=param).fit(X_train, y_train)
  25.     return est
  28. def test_svm(X_test, est):
  29.     return est.predict(X_test)
  32. def score_svm(X_test, est):
  33.     return est.decision_function(X_test)
  36. def train_OvO(X_train, y_train, train_func, param):
  37.     classes = sorted(set(y_train))
  38.     print("classes:", classes)
  39.     estimators = dict()
  40.     for i, ci in enumerate(classes):
  41.         print('-' * 20)
  42.         print("i, ci:", i, ci)
  43.         for j, cj in enumerate(classes):
  44.             print('-' * 7)
  45.             print("j, cj:", j, cj)
  46.             if j > i:
  47.                 X = X_train.copy()
  48.                 X = X[np.logical_or(y_train == ci, y_train == cj)]
  49.                 print("np.logical_or(y_train == ci, y_train == cj):", np.logical_or(y_train == ci, y_train == cj))
  50.                 print("X:", X)
  51.                 y = y_train.copy()
  52.                 y = y[np.logical_or(y_train == ci, y_train == cj)]
  53.                 print("y:", y)
  54.                 yp = y.copy()
  55.                 yp[y == ci] = 1
  56.                 yp[y == cj] = -1
  57.                 print("yp:", yp)
  58.                 est = train_func(X, yp, param)
  59.                 estimators[(i, j)] = est
  60.     return estimators
  63. def test_OvO(X_test, test_func, estimators):
  64.     print("len(estimators):", len(estimators))
  65.     all_scores = np.zeros((X_test.shape[0], len(estimators)))
  66.     print("all_scores.shape:", all_scores.shape)
  67.     for i, j in estimators:
  68.         est = estimators[(i, j)]
  69.         preds = test_func(X_test, est)
  70.         print("preds:", preds)
  71.         print("i:", i)
  72.         print("preds == 1:", preds == 1)
  73.         print('all_scores:', all_scores)
  74.         all_scores[:, i][preds == 1] += 1
  75.         print('all_scores:', all_scores)
  76.         print('j:', j)
  77.         print('preds == -1:', preds == -1)
  78.         all_scores[:, j][preds == -1] += 1
  79.         print('all_scores:', all_scores)
  80.     # 为什么这里是以横轴判断类别。说明每一样代表一个X_test。
  81.     preds = np.argmax(all_scores, axis=1)
  82.     return preds
  85. # This cell is reserved for the unit tests. Do not consider this cell.
  86. ### BEGIN TESTS
  87. X_train = np.array([[10, 10], [8, 10], [-5, 5.5], [-5.4, 5.5], [-20, -20], [-15, -20]])
  88. y_train = np.array([0, 0, 1, 1, 2, 2])
  89. X_test = np.array([[11, 11], [-5, 5], [-15, -15]])
  90. y_test = np.array([0, 1, 2])
  91. # plt.scatter(X_train[:, 0], X_train[:, 1])
  92. # plt.show()
  93. est = train_OvO(X_train, y_train, train_svm, param=1)
  94. # print(est)
  95. print('=' * 40)
  96. preds = test_OvO(X_test, test_svm, est)
  97. print("preds:", preds)
  98. test_cm = confusion_matrix(y_test, preds)
  99. # print(test_cm)
  100. target_cm = np.eye(3)
  101. assert_test_equality(target_cm, test_cm)
  102. ### END TESTS
  104. print('-' * 50)
  105. for i, j in est:
  106.     print(score_svm(X_test, est[(i, j)]))

  • 控制台输出

    classes: [0, 1, 2]

    i, ci: 0 0

    j, cj: 0 0

    j, cj: 1 1
    np.logical_or(y_train == ci, y_train == cj): [ True True True True False False]
    X: [[10. 10. ]
    [ 8. 10. ]
    [-5. 5.5]
    [-5.4 5.5]]
    y: [0 0 1 1]

    yp: [ 1 1 -1 -1]

    j, cj: 2 2
    np.logical_or(y_train == ci, y_train == cj): [ True True False False True True]
    X: [[ 10. 10.]
    [ 8. 10.]
    [-20. -20.]
    [-15. -20.]]
    y: [0 0 2 2]

    yp: [ 1 1 -1 -1]

    i, ci: 1 1

    j, cj: 0 0

    j, cj: 1 1

    j, cj: 2 2
    np.logical_or(y_train == ci, y_train == cj): [False False True True True True]
    X: [[ -5. 5.5]
    [ -5.4 5.5]
    [-20. -20. ]
    [-15. -20. ]]
    y: [1 1 2 2]

    yp: [ 1 1 -1 -1]

    i, ci: 2 2

    j, cj: 0 0

    j, cj: 1 1

    j, cj: 2 2

    len(estimators): 3
    all_scores.shape: (3, 3)
    preds: [ 1 -1 -1]
    i: 0
    preds == 1: [ True False False]
    all_scores: [[0. 0. 0.]
    [0. 0. 0.]
    [0. 0. 0.]]
    all_scores: [[1. 0. 0.]
    [0. 0. 0.]
    [0. 0. 0.]]
    j: 1
    preds == -1: [False True True]
    all_scores: [[1. 0. 0.]
    [0. 1. 0.]
    [0. 1. 0.]]
    preds: [ 1 1 -1]
    i: 0
    preds == 1: [ True True False]
    all_scores: [[1. 0. 0.]
    [0. 1. 0.]
    [0. 1. 0.]]
    all_scores: [[2. 0. 0.]
    [1. 1. 0.]
    [0. 1. 0.]]
    j: 2
    preds == -1: [False False True]
    all_scores: [[2. 0. 0.]
    [1. 1. 0.]
    [0. 1. 1.]]
    preds: [ 1 1 -1]
    i: 1
    preds == 1: [ True True False]
    all_scores: [[2. 0. 0.]
    [1. 1. 0.]
    [0. 1. 1.]]
    all_scores: [[2. 1. 0.]
    [1. 2. 0.]
    [0. 1. 1.]]
    j: 2
    preds == -1: [False False True]
    all_scores: [[2. 1. 0.]
    [1. 2. 0.]
    [0. 1. 2.]]

    preds: [0 1 2]

    [ 1.45548543 -0.97524791 -2.01288786]
    [ 1.20241692 0.06646526 -1.62537764]
    [ 0.57513497 0.93275666 -0.74550495]

    Process finished with exit code 0

2、逐段分析

（1）输入可视化

  1. # This cell is reserved for the unit tests. Do not consider this cell.
  2. ### BEGIN TESTS
  3. X_train = np.array([[10, 10], [8, 10], [-5, 5.5], [-5.4, 5.5], [-20, -20], [-15, -20]])
  4. y_train = np.array([0, 0, 1, 1, 2, 2])
  5. X_test = np.array([[11, 11], [-5, 5], [-15, -15]])
  6. y_test = np.array([0, 1, 2])
  7. plt.scatter(X_train[:, 0], X_train[:, 1], color="r")
  8. plt.scatter(X_test[:, 0], X_test[:, 1], color="b")
  9. plt.show()

在这里插入图片描述
- 从输入可以看出训练集和测试集不同标签分得很开，能够很好分类

（2） train_OvO分段解析

  1. def train_OvO(X_train, y_train, train_func, param):
  2.     classes = sorted(set(y_train))
  3.     print("classes:", classes)
  4.     estimators = dict()
  5.     for i, ci in enumerate(classes):
  6.         print('-' * 20)
  7.         print("i, ci:", i, ci)
  8.         for j, cj in enumerate(classes):
  9.             print('-' * 7)
  10.             print("j, cj:", j, cj)
  11.             if j > i:
  12.                 X = X_train.copy()
  13.                 X = X[np.logical_or(y_train == ci, y_train == cj)]
  14.                 print("np.logical_or(y_train == ci, y_train == cj):", np.logical_or(y_train == ci, y_train == cj))
  15.                 print("X:", X)
  16.                 y = y_train.copy()
  17.                 y = y[np.logical_or(y_train == ci, y_train == cj)]
  18.                 print("y:", y)
  19.                 yp = y.copy()
  20.                 yp[y == ci] = 1
  21.                 yp[y == cj] = -1
  22.                 print("yp:", yp)
  23.                 est = train_func(X, yp, param)
  24.                 estimators[(i, j)] = est
  25.     return estimators
  28. est = train_OvO(X_train, y_train, train_svm, param=1)

  • 输出

    classes: [0, 1, 2]

    i, ci: 0 0

    j, cj: 0 0

    j, cj: 1 1
    np.logical_or(y_train == ci, y_train == cj): [ True True True True False False]
    X: [[10. 10. ]
    [ 8. 10. ]
    [-5. 5.5]
    [-5.4 5.5]]
    y: [0 0 1 1]

    yp: [ 1 1 -1 -1]

    j, cj: 2 2
    np.logical_or(y_train == ci, y_train == cj): [ True True False False True True]
    X: [[ 10. 10.]
    [ 8. 10.]
    [-20. -20.]
    [-15. -20.]]
    y: [0 0 2 2]

    yp: [ 1 1 -1 -1]

    i, ci: 1 1

    j, cj: 0 0

    j, cj: 1 1

    j, cj: 2 2
    np.logical_or(y_train == ci, y_train == cj): [False False True True True True]
    X: [[ -5. 5.5]
    [ -5.4 5.5]
    [-20. -20. ]
    [-15. -20. ]]
    y: [1 1 2 2]

    yp: [ 1 1 -1 -1]

    i, ci: 2 2

    j, cj: 0 0

    j, cj: 1 1

    j, cj: 2 2

可以看出遍历了i，j的所有情况，每一个i和j都代表了一种类别，对i和j进行分类的过程就是一个二分类的过程。这样每一对（i，j）都产生了一个分类器 C i C_i Ci​。返回值estimators就是一个key为（i，j），value为 C i C_i Ci​的字典。

（3）test_OvO 分段解析

  1. def test_OvO(X_test, test_func, estimators):
  2.     print("len(estimators):", len(estimators))
  3.     all_scores = np.zeros((X_test.shape[0], len(estimators)))
  4.     print("all_scores.shape:", all_scores.shape)
  5.     for i, j in estimators:
  6.         est = estimators[(i, j)]
  7.         preds = test_func(X_test, est)
  8.         print("preds:", preds)
  9.         print("i:", i)
  10.         print("preds == 1:", preds == 1)
  11.         print('all_scores:', all_scores)
  12.         all_scores[:, i][preds == 1] += 1
  13.         print('all_scores:', all_scores)
  14.         print('j:', j)
  15.         print('preds == -1:', preds == -1)
  16.         all_scores[:, j][preds == -1] += 1
  17.         print('all_scores:', all_scores)
  18.     # 为什么这里是以横轴判断类别。说明每一样代表一个X_test。
  19.     preds = np.argmax(all_scores, axis=1)
  20.     return preds
  23. preds = test_OvO(X_test, test_svm, est)

  • 输出

    len(estimators): 3
    all_scores.shape: (3, 3)
    preds: [ 1 -1 -1]
    i: 0
    preds == 1: [ True False False]
    all_scores: [[0. 0. 0.]
    [0. 0. 0.]
    [0. 0. 0.]]
    all_scores: [[1. 0. 0.]
    [0. 0. 0.]
    [0. 0. 0.]]
    j: 1
    preds == -1: [False True True]
    all_scores: [[1. 0. 0.]
    [0. 1. 0.]
    [0. 1. 0.]]
    preds: [ 1 1 -1]
    i: 0
    preds == 1: [ True True False]
    all_scores: [[1. 0. 0.]
    [0. 1. 0.]
    [0. 1. 0.]]
    all_scores: [[2. 0. 0.]
    [1. 1. 0.]
    [0. 1. 0.]]
    j: 2
    preds == -1: [False False True]
    all_scores: [[2. 0. 0.]
    [1. 1. 0.]
    [0. 1. 1.]]
    preds: [ 1 1 -1]
    i: 1
    preds == 1: [ True True False]
    all_scores: [[2. 0. 0.]
    [1. 1. 0.]
    [0. 1. 1.]]
    all_scores: [[2. 1. 0.]
    [1. 2. 0.]
    [0. 1. 1.]]
    j: 2
    preds == -1: [False False True]
    all_scores: [[2. 1. 0.]
    [1. 2. 0.]
    [0. 1. 2.]]

    preds: [0 1 2]

    [ 1.45548543 -0.97524791 -2.01288786]
    [ 1.20241692 0.06646526 -1.62537764]
    [ 0.57513497 0.93275666 -0.74550495]

    Process finished with exit code 0

这一段解释了test_OvO是如何工作的，首先构造了all_scores为一个行数为输入测试集的个数，列数为分类器字典中，分类器的总数。这意味着，每一个分类器都会对每一个输入进行一次分类，对分类结果累加，最后得到一个分类矩阵。最后对分类矩阵按行求最大值的下标，就得到了每个输入的预测分类。

（4）score_svm函数解析

  1. print('-' * 50)
  2. for i, j in est:
  3.     print(score_svm(X_test, est[(i, j)]))

  • 输出

    [ 1.45548543 -0.97524791 -2.01288786]
    [ 1.20241692 0.06646526 -1.62537764]
    [ 0.57513497 0.93275666 -0.74550495]

此处score_svm的输出为（i，j）之间的距离，对于三种分类器

二、20-21题，加入score_svm后的分类

在这里插入图片描述
在这里插入图片描述
根据上面老师的解释，这一题的关键是要解决在使用test_svm函数得到得结果矩阵如果在一行中，出现了两个一样大得最大数值后，要使用score_svm来区分谁最优。这里就构造一个一样维度的结果矩阵

1、整体代码

  1. import numpy as np
  2. from sklearn.svm import LinearSVC
  3. from sklearn.datasets import load_digits
  4. from sklearn.model_selection import train_test_split
  5. from sklearn.metrics import confusion_matrix
  6. import matplotlib.pyplot as plt
  7. import numpy as np
  8. import scipy as sp
  9. import warnings
  11. warnings.filterwarnings('ignore')
  14. # unit test utilities: you can ignore these function
  15. def is_approximately_equal(test, target, eps=1e-2):
  16.     return np.mean(np.fabs(np.array(test) - np.array(target))) < eps
  19. def assert_test_equality(test, target):
  20.     assert is_approximately_equal(test, target), 'Expected:\n %s \nbut got:\n %s' % (target, test)
  23. def train_svm(X_train, y_train, param):
  24.     est = LinearSVC(C=param).fit(X_train, y_train)
  25.     return est
  28. def test_svm(X_test, est):
  29.     return est.predict(X_test)
  32. def score_svm(X_test, est):
  33.     return est.decision_function(X_test)
  36. def train_OvO(X_train, y_train, train_func, param):
  37.     classes = sorted(set(y_train))
  38.     estimators = dict()
  39.     for i, ci in enumerate(classes):
  40.         for j, cj in enumerate(classes):
  41.             if j > i:
  42.                 X = X_train.copy()
  43.                 X = X[np.logical_or(y_train == ci, y_train == cj)]
  44.                 y = y_train.copy()
  45.                 y = y[np.logical_or(y_train == ci, y_train == cj)]
  46.                 yp = y.copy()
  47.                 yp[y == ci] = 1
  48.                 yp[y == cj] = -1
  49.                 est = train_func(X, yp, param)
  50.                 estimators[(i, j)] = est
  51.     return estimators
  54. def test_OvO(X_test, test_func, score_func, estimators):
  55.     all_scores = np.zeros((X_test.shape[0], len(estimators)))
  56.     score_m = np.zeros((X_test.shape[0], len(estimators)))
  57.     for i, j in estimators:
  58.         est = estimators[(i, j)]
  59.         preds = test_func(X_test, est)
  60.         # the result of score_func
  61.         preds_s = score_func(X_test, est)
  63.         all_scores[:, i][preds == 1] += 1
  64.         score_m[:, i][preds > 0] += preds_s[preds_s > 0]
  65.         all_scores[:, j][preds == -1] += 1
  66.         score_m[:, j][preds < 0] += preds_s[preds_s < 0]
  67.     print("all_scores:\n", all_scores)
  68.     print("score_m:\n",score_m)
  70.     print("preds:\n", preds)
  71.     # When two or more classifiers end in a tie, use score_func to break the tie
  72.     preds = np.argmax(all_scores, axis=1)
  73.     # judge whether two or more classifiers end in a tie
  74.     for i in range(len(preds)):
  75.         all_scores_sort = sorted(all_scores[i])
  76.         # use score_func to break the tie
  77.         if all_scores_sort[-1] == all_scores_sort[-2]:
  78.             preds[i] = np.argmax(score_m, axis=1)[i]
  79.     print("preds:\n", preds)
  81.     return preds
  84. # This cell is reserved for the unit tests. Do not consider this cell.
  85. ### BEGIN TESTS
  86. X_train = np.array([[10, 10], [8, 10], [-5, 5.5], [-5.4, 5.5], [-20, -20], [-15, -20]])
  87. y_train = np.array([0, 0, 1, 1, 2, 2])
  88. X_test = np.array([[11, 11], [-5, 5], [-15, -15]])
  89. y_test = np.array([0, 1, 2])
  90. est = train_OvO(X_train, y_train, train_svm, param=1)
  91. preds = test_OvO(X_test, test_svm, score_svm, est)
  92. test_cm = confusion_matrix(y_test, preds)
  93. target_cm = np.eye(3)
  94. assert_test_equality(target_cm, test_cm)
  95. ### END TESTS

  • 输出

    all_scores:
    [[2. 1. 0.]
    [1. 2. 0.]
    [0. 1. 2.]]
    score_m:
    [[ 2.65790263 0.57513497 0. ]
    [ 0.06646526 -0.04249243 0. ]
    [ 0. -2.0128883 -2.3708826 ]]
    preds:
    [ 1 1 -1]
    preds:
    [0 1 2]

    Process finished with exit code 0

2、逐段解析

（1）test_OvO分析

  1. def test_OvO(X_test, test_func, score_func, estimators):
  2.     all_scores = np.zeros((X_test.shape[0], len(estimators)))
  3.     score_m = np.zeros((X_test.shape[0], len(estimators)))
  4.     for i, j in estimators:
  5.         est = estimators[(i, j)]
  6.         preds = test_func(X_test, est)
  7.         # the result of score_func
  8.         preds_s = score_func(X_test, est)
  10.         all_scores[:, i][preds == 1] += 1
  11.         score_m[:, i][preds > 0] += preds_s[preds_s > 0]
  12.         all_scores[:, j][preds == -1] += 1
  13.         score_m[:, j][preds < 0] += preds_s[preds_s < 0]
  14.     print("all_scores:\n", all_scores)
  15.     print("score_m:\n",score_m)
  17.     print("preds:\n", preds)
  18.     # When two or more classifiers end in a tie, use score_func to break the tie
  19.     preds = np.argmax(all_scores, axis=1)
  20.     # judge whether two or more classifiers end in a tie
  21.     for i in range(len(preds)):
  22.         all_scores_sort = sorted(all_scores[i])
  23.         # use score_func to break the tie
  24.         if all_scores_sort[-1] == all_scores_sort[-2]:
  25.             preds[i] = np.argmax(score_m, axis=1)[i]
  26.     print("preds:\n", preds)
  28.     return preds
  31. preds = test_OvO(X_test, test_svm, score_svm, est)

  • 输出

    all_scores:
    [[2. 1. 0.]
    [1. 2. 0.]
    [0. 1. 2.]]
    score_m:
    [[ 2.65790263 0.57513497 0. ]
    [ 0.06646526 -0.04249243 0. ]
    [ 0. -2.0128883 -2.3708826 ]]
    preds:
    [ 1 1 -1]
    preds:
    [0 1 2]

由于train_OvO没有改变故而这里不讲。接着本节开始的分析，对test_func, score_func分别构造了矩阵 all_scores，score_m用于存取分类器的结果。代码中score_m[:, i][preds > 0] += preds_s[preds_s > 0]意思是取出score_func输出的结果大于零的部分相加。矩阵 all_scores，score_m的构造过程相同。得到的结果如上面输出所示，可以发现score_m其实就是更加精细化输出的all_scores。代码的最后部分，判断了all_scores每一行中，最大的两个数是否相同，如果相同，就使用score_m矩阵得最优解。

（2）对我作出的更改进行测试

  1. import numpy as np
  3. all_scores = np.array([
  4.     [0, 2, 1],
  5.     [1, 1, 0],
  6.     [2, 0, 2]
  7. ])
  9. score_m = np.array([
  10.     [0, 2.1, 1.2],
  11.     [1.6, 1.3, 0],
  12.     [2.1, 0, 2.9]
  13. ])
  15. preds = np.argmax(all_scores, axis=1)
  16. print('preds:', preds)
  18. for i in range(len(preds)):
  19.     print('all_scores[i]:', all_scores[i])
  20.     all_scores_sort = sorted(all_scores[i])
  21.     print('all_scores_sort:', all_scores_sort)
  22.     if all_scores_sort[-1] == all_scores_sort[-2]:
  23.         preds[i] = np.argmax(score_m, axis=1)[i]
  25. print('preds:', preds)

  • 输出

    preds: [1 0 0]
    all_scores[i]: [0 2 1]
    all_scores_sort: [0, 1, 2]
    all_scores[i]: [1 1 0]
    all_scores_sort: [0, 1, 1]
    all_scores[i]: [2 0 2]
    all_scores_sort: [0, 2, 2]
    preds: [1 0 2]

分析我给出的矩阵 all_scores，score_m，all_scores中第二行和第三行有相同的值，得到的预测是preds: [1 0 0]，再判断all_scores中每一行，最大的两个数是否相同，如果相同，就使用score_m矩阵得最优解。最后矫正之后的预测为preds: [1 0 2]，可见，算法可行。

发表评论

表情：
评论列表 （有 0 条评论，368人围观）

还没有评论，来说两句吧...

相关阅读