《機器學習》周志華 習題答案3.5

編程實現判別分析，並給出西瓜數據集上的結果。

數據集如下

編號,色澤,根蒂,敲聲,紋理,臍部,觸感,密度,含糖率,好瓜
1,青綠,蜷縮,濁響,清晰,凹陷,硬滑,0.697,0.46,是
2,烏黑,蜷縮,沉悶,清晰,凹陷,硬滑,0.774,0.376,是
3,烏黑,蜷縮,濁響,清晰,凹陷,硬滑,0.634,0.264,是
4,青綠,蜷縮,沉悶,清晰,凹陷,硬滑,0.608,0.318,是
5,淺白,蜷縮,濁響,清晰,凹陷,硬滑,0.556,0.215,是
6,青綠,稍蜷,濁響,清晰,稍凹,軟粘,0.403,0.237,是
7,烏黑,稍蜷,濁響,稍糊,稍凹,軟粘,0.481,0.149,是
8,烏黑,稍蜷,濁響,清晰,稍凹,硬滑,0.437,0.211,是
9,烏黑,稍蜷,沉悶,稍糊,稍凹,硬滑,0.666,0.091,否
10,青綠,硬挺,清脆,清晰,平坦,軟粘,0.243,0.267,否
11,淺白,硬挺,清脆,模糊,平坦,硬滑,0.245,0.057,否
12,淺白,蜷縮,濁響,模糊,平坦,軟粘,0.343,0.099,否
13,青綠,稍蜷,濁響,稍糊,凹陷,硬滑,0.639,0.161,否
14,淺白,稍蜷,沉悶,稍糊,凹陷,硬滑,0.657,0.198,否
15,烏黑,稍蜷,濁響,清晰,稍凹,軟粘,0.36,0.37,否
16,淺白,蜷縮,濁響,模糊,平坦,硬滑,0.593,0.042,否
17,青綠,蜷縮,沉悶,稍糊,稍凹,硬滑,0.719,0.103,否

Python代碼實現方式如下：調用了sklearn中的線性判別分析模塊。

#!/usr/bin/python
# -*- coding:utf-8 -*-
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import colors

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

file1 = open('c:\quant\watermelon.csv','r')
data = [line.strip('\n').split(',') for line in file1]
X = [[float(raw[-3]), float(raw[-2])] for raw in data[1:]]
y = [1 if raw[-1]=='\xca\xc7' else 0 for raw in data[1:]]
X = np.array(X)
y = np.array(y)

#######################################################################以上是西瓜

# colormap
cmap = colors.LinearSegmentedColormap(
    'red_blue_classes',
    {'red': [(0, 1, 1), (1, 0.7, 0.7)],
     'green': [(0, 0.7, 0.7), (1, 0.7, 0.7)],
     'blue': [(0, 0.7, 0.7), (1, 1, 1)]})
plt.cm.register_cmap(cmap=cmap)

###############################################################################
# plot functions
def plot_data(lda, X, y, y_pred):
    plt.figure()
    plt.title('Linear Discriminant Analysis')
    plt.xlabel('Sugar Rate')
    plt.ylabel('Density')
    tp = (y == y_pred)  # True Positive //Boolean matrix

    tp0, tp1 = tp[y == 0], tp[y == 1]
    print tp
    X0, X1 = X[y == 0], X[y == 1]
    X0_tp, X0_fp = X0[tp0], X0[~tp0]
    X1_tp, X1_fp = X1[tp1], X1[~tp1]
    # class 0: dots
    plt.plot(X0_tp[:, 0], X0_tp[:, 1], 'o', color='red')
    plt.plot(X0_fp[:, 0], X0_fp[:, 1], '.', color='#990000')  # dark red

    # class 1: dots
    plt.plot(X1_tp[:, 0], X1_tp[:, 1], 'o', color='blue')
    plt.plot(X1_fp[:, 0], X1_fp[:, 1], '.', color='#000099')  # dark blue

    # class 0 and 1 : areas
    nx, ny = 200, 100
    x_min, x_max = plt.xlim()
    y_min, y_max = plt.ylim()
    xx, yy = np.meshgrid(np.linspace(x_min, x_max, nx),
                         np.linspace(y_min, y_max, ny))
    Z = lda.predict_proba(np.c_[xx.ravel(), yy.ravel()])
    Z = Z[:, 1].reshape(xx.shape)
    plt.pcolormesh(xx, yy, Z, cmap='red_blue_classes',
                   norm=colors.Normalize(0., 1.))
    plt.contour(xx, yy, Z, [0.5], linewidths=2., colors='k')

    # means
    plt.plot(lda.means_[0][0], lda.means_[0][1],
             'o', color='black', markersize=10)
    plt.plot(lda.means_[1][0], lda.means_[1][1],
             'o', color='black', markersize=10)

###############################################################################
# Linear Discriminant Analysis
lda = LinearDiscriminantAnalysis(solver="svd", store_covariance=True)
y_pred = lda.fit(X, y).predict(X)
plot_data(lda, X, y, y_pred)
plt.axis('tight')
plt.suptitle('Linear Discriminant Analysis of Watermelon')
plt.show()

結果如下：

其中紅色的藍色的分別是兩種西瓜。小紅色的點和小藍色的點表示區分錯誤。中間的橫線是分界線。