数据挖掘实战（二）—— 类不平衡问题_信用卡欺诈检测

写在jupyter里面比较漂亮：

https://douzujun.github.io/page/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/%E7%B1%BB%E4%B8%8D%E5%B9%B3%E8%A1%A1%E9%97%AE%E9%A2%98_%E4%BF%A1%E7%94%A8%E5%8D%A1%E6%AC%BA%E8%AF%88%E6%A3%80%E6%B5%8B.html

In [34]:
import pandas as pd import matplotlib.pylab as plt import numpy as np %matplotlib inline In [35]: data = pd.read_csv('creditcard.csv') data.head() # 0 - 正常的样本，1 - 有问题的数据

Out[35]:

	Time	V1	V2	V3	V4	V5	V6	V7	V8	V9	...	V21	V22	V23	V24	V25	V26	V27	V28	Amount	Class
0	0.0	-1.359807	-0.072781	2.536347	1.378155	-0.338321	0.462388	0.239599	0.098698	0.363787	...	-0.018307	0.277838	-0.110474	0.066928	0.128539	-0.189115	0.133558	-0.021053	149.62	0
1	0.0	1.191857	0.266151	0.166480	0.448154	0.060018	-0.082361	-0.078803	0.085102	-0.255425	...	-0.225775	-0.638672	0.101288	-0.339846	0.167170	0.125895	-0.008983	0.014724	2.69	0
2	1.0	-1.358354	-1.340163	1.773209	0.379780	-0.503198	1.800499	0.791461	0.247676	-1.514654	...	0.247998	0.771679	0.909412	-0.689281	-0.327642	-0.139097	-0.055353	-0.059752	378.66	0
3	1.0	-0.966272	-0.185226	1.792993	-0.863291	-0.010309	1.247203	0.237609	0.377436	-1.387024	...	-0.108300	0.005274	-0.190321	-1.175575	0.647376	-0.221929	0.062723	0.061458	123.50	0
4	2.0	-1.158233	0.877737	1.548718	0.403034	-0.407193	0.095921	0.592941	-0.270533	0.817739	...	-0.009431	0.798278	-0.137458	0.141267	-0.206010	0.502292	0.219422	0.215153	69.99	0

5 rows × 31 columns

In [36]:
count_classes = pd.value_counts(data['Class'], sort=True).sort_index() count_classes.plot(kind = 'bar', alpha=0.5) plt.title('Fraud class histogram') plt.xlabel('Class') plt.ylabel('Frequency') # 此时无缺陷的样本数非常多，有缺陷的样本非常的少，可以明显看到 样本分布非常不平衡 Out[36]: <matplotlib.text.Text at 0xfc64470>

 

 

样本不均衡解决方案

• 下采样 (两者一样小)

  • 从0的样本中选取 跟 1的样本一样小

• 过采样 (两者一样多)

  • 1的样本中采取生成策略，生成的数据和 0号样本一样多

标准化

• 比如Amount，数据量区间很大，会对模型有一个“数据大就重要的”误区; 所以要进行归一化，或者标准化
• 把他们区间放在 [-1,1] 或者[0, 1]区间上

In [37]:
# 这个sklearn库: 预处理操作 => 标准化的模块
from sklearn.preprocessing import StandardScaler # fit_transform(): 对数据进行一个变化了; 变化好的列成为一个新属性添加到 data data['normAmount'] = StandardScaler().fit_transform(data['Amount'].reshape(-1, 1)) data = data.drop(['Time', 'Amount'], axis=1) data.head()

Out[37]:

	V1	V2	V3	V4	V5	V6	V7	V8	V9	V10	...	V21	V22	V23	V24	V25	V26	V27	V28	Class	normAmount
0	-1.359807	-0.072781	2.536347	1.378155	-0.338321	0.462388	0.239599	0.098698	0.363787	0.090794	...	-0.018307	0.277838	-0.110474	0.066928	0.128539	-0.189115	0.133558	-0.021053	0	0.244964
1	1.191857	0.266151	0.166480	0.448154	0.060018	-0.082361	-0.078803	0.085102	-0.255425	-0.166974	...	-0.225775	-0.638672	0.101288	-0.339846	0.167170	0.125895	-0.008983	0.014724	0	-0.342475
2	-1.358354	-1.340163	1.773209	0.379780	-0.503198	1.800499	0.791461	0.247676	-1.514654	0.207643	...	0.247998	0.771679	0.909412	-0.689281	-0.327642	-0.139097	-0.055353	-0.059752	0	1.160686
3	-0.966272	-0.185226	1.792993	-0.863291	-0.010309	1.247203	0.237609	0.377436	-1.387024	-0.054952	...	-0.108300	0.005274	-0.190321	-1.175575	0.647376	-0.221929	0.062723	0.061458	0	0.140534
4	-1.158233	0.877737	1.548718	0.403034	-0.407193	0.095921	0.592941	-0.270533	0.817739	0.753074	...	-0.009431	0.798278	-0.137458	0.141267	-0.206010	0.502292	0.219422	0.215153	0	-0.073403

5 rows × 30 columns

下采样实现

In [38]:
X = data.ix[:, data.columns != 'Class'] y = data.ix[:, data.columns == 'Class'] # Number of data points in the minority class # 样本的个数 number_records_fraud = len(data[data.Class == 1]) # 样本的索引, 所有值为1的 索引 fraud_indices = np.array(data[data.Class == 1].index) # Picking the indices of the normal classes # 值为0 的 数据的索引 normal_indices = data[data.Class == 0].index # 使两个样本一样少 # Out of the indices we picked, randomly select 'x' number (number_records_fraud) # 随机选取样本中的数据, 随机选择: np.random.choice(Class为0的数据索引=>样本, 选择多少数据量, 是否选择代替=false) random_normal_indices = np.random.choice(normal_indices, number_records_fraud, replace=False) random_normal_indices = np.array(random_normal_indices) # Appending the 2 indices (连接缺陷数据索引 和 随机选取的Class=0的数据索引) under_sample_indices = np.concatenate([fraud_indices, random_normal_indices]) # Under sample dataset 下采样 (选取该索引下的数据) under_sample_data = data.iloc[under_sample_indices, :] X_undersample = under_sample_data.ix[:, under_sample_data.columns != 'Class'] y_undersample = under_sample_data.ix[:, under_sample_data.columns == 'Class'] # Showing Ratio print('Percentage oif normal transactions: ', len(under_sample_data[under_sample_data.Class == 0]) / len(under_sample_data)) print('Percentage oif fraud transactions: ', len(under_sample_data[under_sample_data.Class == 1]) / len(under_sample_data)) print('Total number of transactions in resampled data: ', len(under_sample_data)) Percentage oif normal transactions: 0.5 Percentage oif fraud transactions: 0.5 Total number of transactions in resampled data: 984 In [39]: # 交叉验证， train_test_split: 切分数据 from sklearn.cross_validation import train_test_split # Whole dataset, 将数据切割成训练集0.7 和测试集 0.3 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 0) print('Number transactions train dataset: ', len(X_train)) print('Number transactions test dataset: ', len(X_test)) print('Total number of transactions: ', len(X_train) + len(X_test)) # Undersampled dataset(对下采样数据集相同操作) # 只用下采样模型进行训练 # 最终用原始数据集进行测试 X_train_undersample, X_test_undersample, y_train_undersample, y_test_undersample = train_test_split(X_undersample, y_undersample, test_size = 0.3, random_state = 0) print('') print("Number transactions train dataset: ", len(X_train_undersample)) print("Number transactions test dataset: ", len(X_test_undersample)) print("Total number of transactions: ", len(X_train_undersample) + len(X_test_undersample)) Number transactions train dataset: 199364 Number transactions test dataset: 85443 Total number of transactions: 284807 Number transactions train dataset: 688 Number transactions test dataset: 296 Total number of transactions: 984 

模型评估标准

• 在类不平衡问题中，用精度来衡量指标是骗人的！没有意义(1000个人，全部预测为正常， 0个癌症) 精度 = TPTotalTPTotal

• 这里使用Recall(召回率, 查全率) = TPTP+FNTPTP+FN, 1000个人(990个正常,10个癌症),如果检测出0个病人 010=0,检测出2个病人210=0.2010=0,检测出2个病人210=0.2

• 检测任务上，通常用Recall作为模型的评估标准

In [40]:
# Recall = TP / (TP + FN)
from sklearn.linear_model import LogisticRegression # KFold->做几倍的交叉验证 from sklearn.cross_validation import KFold, cross_val_score # confusion_matrix: 混淆矩阵 from sklearn.metrics import confusion_matrix, recall_score, classification_report

正则化惩罚

• 设置正则化惩罚项
• 希望当前模型的泛化能力(更稳定一些)， 不仅满足训练数据，还要在测试数据上尽可能满足
• 浮动的差异小 ===> 过拟合的风险小
• 惩罚 θθ (一组分布大A，一组分布小B) => 大力度惩罚A模型，小力度惩罚B模型
• L2正则化L2正则化: loss函数(越低越好) + 12W2(惩罚项)12W2(惩罚项) => 计算哪个模型loss值小, 分析哪个模型更好
• L1正则化L1正则化: loss+|W|loss+|W|
• λL2:设置惩罚力度λλL2:设置惩罚力度λ

In [41]:
def printing_Kfold_score(x_train_data, y_train_data): # 切分成5部分，把原始训练集进行切分 fold = KFold(len(y_train_data), 5, shuffle=False) # Different C parameters (正则化惩罚项) # 希望当前模型的泛化能力(更稳定一些) ， 不仅满足训练数据，还要在测试数据上尽可能满足 # 浮动的差异小 ===> 过拟合的风险小 c_param_range = [0.01, 0.1, 1, 10, 100] # C_parameter => lambda results_table = pd.DataFrame(index = range(len(c_param_range), 2), columns = ['C_parameter', 'Mean recall score']) results_table['C_parameter'] = c_param_range # the k-fold will give 2 lists: train_indics = indices[0], test_indices = indices[1] j = 0 # 查看哪一个C值比较好 for c_param in c_param_range: print('-----------------------------------------') print('C parameter: ', c_param) print('-----------------------------------------') print('') recall_accs = [] # 每次交叉验证的结果 for iteration, indices in enumerate(fold, start=1): # Call the logistic regresstion model with a certain C parameter, "L1惩罚 or L2惩罚" lr = LogisticRegression(C = c_param, penalty='l1') # Use the training data to fit the model. In the case, we use the portion of the fold to train # with indices[0]. We then predict on the portion assigned as the test cross validation with indices. # 进行训练, 交叉验证里的数据(训练集)-->建立一个模型  lr.fit(x_train_data.iloc[indices[0], :], y_train_data.iloc[indices[0], :].values.ravel()) # Predict values using the test indices in the training data # 进行预测, 用交叉验证中的验证集 再进行一个验证(测试)的操作 y_pred_undersample = lr.predict(x_train_data.iloc[indices[1], :].values) # Calculate the recall score and append it to a list for recall scores representing the current xx # 计算当前模型的recall (indices[1]: 测试集) recall_acc = recall_score(y_train_data.iloc[indices[1], :].values, y_pred_undersample) recall_accs.append(recall_acc) print("Iterator ", iteration, ': recall score = ', recall_acc) # The mean value of those recall scores is the metric we want to save and get hold of. results_table.ix[j, 'Mean recall score'] = np.mean(recall_accs) j += 1 print('') print('Mean recall score ', np.mean(recall_accs)) print('') best_c = results_table.loc[results_table['Mean recall score'].idxmax()]['C_parameter'] # Finally, we can check which C parameter is the best amongst the chosen. print('*********************************************************************************') print('Best model to choose from cross validation is with C parameter = ', best_c) print('*********************************************************************************') return best_c In [42]: best_c = printing_Kfold_score(X_train_undersample, y_train_undersample)

 [out]:

-----------------------------------------
C parameter:  0.01
----------------------------------------- Iterator 1 : recall score = 0.931506849315 Iterator 2 : recall score = 0.917808219178 Iterator 3 : recall score = 1.0 Iterator 4 : recall score = 0.959459459459 Iterator 5 : recall score = 0.954545454545 Mean recall score 0.9526639965 ----------------------------------------- C parameter: 0.1 ----------------------------------------- Iterator 1 : recall score = 0.835616438356 Iterator 2 : recall score = 0.86301369863 Iterator 3 : recall score = 0.915254237288 Iterator 4 : recall score = 0.932432432432 Iterator 5 : recall score = 0.893939393939 Mean recall score 0.888051240129 ----------------------------------------- C parameter: 1 ----------------------------------------- Iterator 1 : recall score = 0.849315068493 Iterator 2 : recall score = 0.890410958904 Iterator 3 : recall score = 0.949152542373 Iterator 4 : recall score = 0.945945945946 Iterator 5 : recall score = 0.893939393939 Mean recall score 0.905752781931 ----------------------------------------- C parameter: 10 ----------------------------------------- Iterator 1 : recall score = 0.86301369863 Iterator 2 : recall score = 0.876712328767 Iterator 3 : recall score = 0.949152542373 Iterator 4 : recall score = 0.945945945946 Iterator 5 : recall score = 0.909090909091 Mean recall score 0.908783084961 ----------------------------------------- C parameter: 100 ----------------------------------------- Iterator 1 : recall score = 0.86301369863 Iterator 2 : recall score = 0.876712328767 Iterator 3 : recall score = 0.949152542373 Iterator 4 : recall score = 0.945945945946 Iterator 5 : recall score = 0.939393939394 Mean recall score 0.914843691022 ********************************************************************************* Best model to choose from cross validation is with C parameter = 0.01 ********************************************************************************* 

混淆矩阵

• 如何绘制混淆矩阵
• 混淆矩阵用途

In [43]:
def plot_confusion_matrix(cm, classes, title = 'Confusion matrix', cmap = plt.cm.Blues): ''' This function prints and plots the confusion matrix ''' plt.imshow(cm, interpolation='nearest', cmap=cmap) plt.title(title) plt.colorbar() tick_marks = np.arange(len(classes)) plt.xticks(tick_marks, classes, rotation=0) plt.yticks(tick_marks, classes) thresh = cm.max() / 2. for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])): plt.text(j, i, cm[i, j], horizontalalignment='center', color='white' if cm[i, j] > thresh else 'black') plt.tight_layout() plt.ylabel('True label') plt.xlabel('Perdicted label') In [44]: import itertools lr = LogisticRegression(C = best_c, penalty='l1') lr.fit(X_train_undersample, y_train_undersample.values.ravel()) y_pred_undersample = lr.predict(X_test_undersample.values) # Compulter confusion matrix cnf_matrix = confusion_matrix(y_test_undersample, y_pred_undersample) np.set_printoptions(precision=2) print("Recal metric in the testing dataset: ", cnf_matrix[1, 1]/(cnf_matrix[1, 0] + cnf_matrix[1, 1])) # Plot non-normalized confusion matrix class_names = [0, 1] plt.figure() plot_confusion_matrix(cnf_matrix, classes=class_names, title='Confusion matrix') plt.show() Recal metric in the testing dataset: 0.931972789116

 

In [45]:
lr = LogisticRegression(C = best_c, penalty='l1') lr.fit(X_train_undersample, y_train_undersample.values.ravel()) y_pred = lr.predict(X_test.values) # COmputer confusion matrix cnf_matrix = confusion_matrix(y_test, y_pred) np.set_printoptions(precision=2) print('Recall metric in the testing dataset: ', cnf_matrix[1, 1] / (cnf_matrix[1, 0] + cnf_matrix[1, 1])) # Plot non-normalized confusion matrix class_names = [0, 1] plt.figure() plot_confusion_matrix(cnf_matrix, classes=class_names, title='Confusion matrix') plt.show() # Recall = TP / (TP + FN) # 7101 ==> 误杀了那么多数据

Recall metric in the testing dataset:  0.918367346939

 

 

注: 下采样伴随着误杀的问题，精度不高

In [46]:
# 什么都不做的话
# best_c = printing_Kfold_score(X_train, y_train)
 

Logical 回归

• 阈值对结果的影响

In [47]:
lr = LogisticRegression(C = 0.01, penalty='l1') lr.fit(X_train_undersample, y_train_undersample.values.ravel()) y_pred_undersample_proba = lr.predict_proba(X_test_undersample.values) # 设置不同的阈值 thresholds = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] plt.figure(figsize=(10, 10)) j = 1 for i in thresholds: y_test_undersample_high_recall = y_pred_undersample_proba[:, 1] > i plt.subplot(3, 3, j) j += 1 # Compute confusion matrix cnf_matrix = confusion_matrix(y_test_undersample, y_test_undersample_high_recall) np.set_printoptions(precision=2) print('Recal metric in the testing dataset: ', cnf_matrix[1, 1] / (cnf_matrix[1, 0] + cnf_matrix[1, 1])) # Plot non-normalized confusion matrix class_names = [0, i] plot_confusion_matrix(cnf_matrix, classes=class_names, title='Threshold >= %s' %i)

 [out]:

Recal metric in the testing dataset:  1.0
Recal metric in the testing dataset:  1.0 Recal metric in the testing dataset: 1.0 Recal metric in the testing dataset: 0.972789115646 Recal metric in the testing dataset: 0.931972789116 Recal metric in the testing dataset: 0.87074829932 Recal metric in the testing dataset: 0.823129251701 Recal metric in the testing dataset: 0.761904761905 Recal metric in the testing dataset: 0.591836734694

 

过采样 — SMOTE样本生成策略

• 对于少数类中每一个样本x, 以欧氏距离为标准计算它到少数类样本集中所有样本的距离，得到其k近邻。
• 根据样本不平衡比例设置一个采样比例以确定采样倍率N，对于每一个少数类样本x, 从其k近邻中随机选择若干样本，假设选择的近邻为 xn。
• 对于每一个随机选出的近邻 xn, 分别与原样本按照如下的公式构建新的样本。
• Xnew=X+rand(0,1)×欧式距离Xnew=X+rand(0,1)×欧式距离

In [48]:
import pandas as pd from imblearn.over_sampling import SMOTE from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import confusion_matrix from sklearn.model_selection import train_test_split In [49]: credit_cards = pd.read_csv('creditcard.csv') columns = credit_cards.columns # The labels are in the last column ('Class'), Simply remove it to obtain features cloumns features_columns = columns.delete(len(columns) - 1) features = credit_cards[features_columns] labels = credit_cards['Class'] In [50]: features_train, features_test, labels_train, labels_test = train_test_split(features, labels, test_size=0.2, random_state=0) In [51]: oversampler = SMOTE(random_state=0) os_features, os_labels = oversampler.fit_sample(features_train, labels_train) In [ ]: len(os_labels[os_labels == 1])

Out[ ]:

227454

In [ ]:

os_features = pd.DataFrame(os_features)
os_labels = pd.DataFrame(os_labels) best_c = printing_Kfold_score(os_features, os_labels)

 Out[]:

-----------------------------------------
C parameter:  0.01
----------------------------------------- Iterator 1 : recall score = 0.890322580645 Iterator 2 : recall score = 0.894736842105 Iterator 3 : recall score = 0.968883479031

In [56]:

lr = LogisticRegression(C = best_c, penalty='l1')
lr.fit(os_features, os_labels.values.ravel())
y_pred = lr.predict(features_test.values) # Compute confusion matrix cnf_matrix = confusion_matrix(labels_test, y_pred) np.set_printoptions(precision=2) print("Recall metric in the testing dataset: ", cnf_matrix[1, 1] / (cnf_matrix[1, 0] + cnf_matrix[1, 1])) # Plot non-nonmalized confusion matrix class_names = [0, 1] plt.figure() plot_confusion_matrix(cnf_matrix, classes = class_names, title = 'Confusion matrix') plt.show()

 

Recall metric in the testing dataset:  0.910891089109

 

 

过采样--> Recal 低一些，模型整体的精度更高一些

• 精度 = (TP + TN) / Total

In [ ]:

 

In [ ]: