理解問題

本教程需要解決的問題是根據網站內評論的意見采取合適的行動。

可用的訓練數據集中，網站評論可能是有毒(toxic)(1)或者無毒(not toxic)(0)兩種類型。這種場景下，機器學習中的分類任務最為適合。

分類任務用於區分數據內的類別(category)，類型(type)或種類(class)。常見的例子有：

識別情感是正面或是負面
將郵件按照是否為垃圾郵件歸類
判定病人的實驗室樣本是否為癌症
按照客戶的偏好進行分類以響應銷售活動

分類任務可以是二元又或是多元的。這里面臨的是二元分類的問題。

准備數據

首先建立一個控制台應用程序，基於.NET Core。完成搭建后，添加Microsoft.ML類庫包。接着在工程下新建名為Data的文件夾。

之后，下載WikiPedia-detox-250-line-data.tsv與wikipedia-detox-250-line-test.tsv文件，並將它們放入Data文件夾，值得注意的是，這兩個文件的Copy to Output Directory屬性需要修改成Copy if newer。

加載數據

在Program.cs文件的Main方法里加入以下代碼：

MLContext mlContext = new MLContext(seed: 0);

_textLoader = mlContext.Data.TextReader(new TextLoader.Arguments()
{
    Separator = "tab",
    HasHeader = true,
    Column = new[]
                {
                    new TextLoader.Column("Label", DataKind.Bool, 0),
                    new TextLoader.Column("SentimentText", DataKind.Text, 1)
                }
});

其目的是通過使用TextLoader類為數據的加載作好准備。

Column屬性中構建了兩個對象，即對應數據集中的兩列數據。不過第一列這里必須使用Label而不是Sentiment。

提取特征

新建一個SentimentData.cs文件，其中加入SentimentData類與SentimentPrediction。

public class SentimentData
{
    [Column(ordinal: "0", name: "Label")]
    public float Sentiment;
    [Column(ordinal: "1")]
    public string SentimentText;
}

public class SentimentPrediction
{
    [ColumnName("PredictedLabel")]
    public bool Prediction { get; set; }

    [ColumnName("Probability")]
    public float Probability { get; set; }

    [ColumnName("Score")]
    public float Score { get; set; }
}

SentimentData類中的SentimentText為輸入數據集的特征，Sentiment則是數據集的標記(label)。

SentimentPrediction類用於模型被訓練后的預測。

訓練模型

在Program類中加入Train方法。首先它會讀取訓練數據集，接着將特征列中的文本型數據轉換為浮點型數組並設定了訓練時所使用的決策樹二元分類模型。之后，即是實際訓練模型。

public static ITransformer Train(MLContext mlContext, string dataPath)
{
    IDataView dataView = _textLoader.Read(dataPath);
    var pipeline = mlContext.Transforms.Text.FeaturizeText("SentimentText", "Features")
        .Append(mlContext.BinaryClassification.Trainers.FastTree(numLeaves: 50, numTrees: 50, minDatapointsInLeaves: 20));

    Console.WriteLine("=============== Create and Train the Model ===============");
    var model = pipeline.Fit(dataView);
    Console.WriteLine("=============== End of training ===============");
    Console.WriteLine();

    return model;
}

評估模型

加入Evaluate方法。到了這一步，需要讀取的是用於測試的數據集，且讀取后的數據仍然需要轉換成合適的數據類型。

public static void Evaluate(MLContext mlContext, ITransformer model)
{
    IDataView dataView = _textLoader.Read(_testDataPath);
    Console.WriteLine("=============== Evaluating Model accuracy with Test data===============");
    var predictions = model.Transform(dataView);

    var metrics = mlContext.BinaryClassification.Evaluate(predictions, "Label");
    Console.WriteLine();
    Console.WriteLine("Model quality metrics evaluation");
    Console.WriteLine("--------------------------------");
    Console.WriteLine($"Accuracy: {metrics.Accuracy:P2}");
    Console.WriteLine($"Auc: {metrics.Auc:P2}");
    Console.WriteLine($"F1Score: {metrics.F1Score:P2}");
    Console.WriteLine("=============== End of model evaluation ===============");
}

使用模型

訓練及評估模型完成后，就可以正式使用它了。這里需要建立一個用於預測的對象(PredictionFunction)，其預測方法的輸入參數是SentimentData類型，返回結果為SentimentPrediction類型。

private static void Predict(MLContext mlContext, ITransformer model)
{
    var predictionFunction = model.MakePredictionFunction<SentimentData, SentimentPrediction>(mlContext);
    SentimentData sampleStatement = new SentimentData
    {
        SentimentText = "This is a very rude movie"
    };

    var resultprediction = predictionFunction.Predict(sampleStatement);

    Console.WriteLine();
    Console.WriteLine("=============== Prediction Test of model with a single sample and test dataset ===============");

    Console.WriteLine();
    Console.WriteLine($"Sentiment: {sampleStatement.SentimentText} | Prediction: {(Convert.ToBoolean(resultprediction.Prediction) ? "Toxic" : "Not Toxic")} | Probability: {resultprediction.Probability} ");
    Console.WriteLine("=============== End of Predictions ===============");
    Console.WriteLine();
}

完整示例代碼

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Microsoft.ML;
using Microsoft.ML.Core.Data;
using Microsoft.ML.Runtime.Data;
using Microsoft.ML.Transforms.Text;

namespace SentimentAnalysis
{
    class Program
    {
        static readonly string _trainDataPath = Path.Combine(Environment.CurrentDirectory, "Data", "wikipedia-detox-250-line-data.tsv");
        static readonly string _testDataPath = Path.Combine(Environment.CurrentDirectory, "Data", "wikipedia-detox-250-line-test.tsv");
        static readonly string _modelPath = Path.Combine(Environment.CurrentDirectory, "Data", "Model.zip");
        static TextLoader _textLoader;

        static void Main(string[] args)
        {
            MLContext mlContext = new MLContext(seed: 0);

            _textLoader = mlContext.Data.TextReader(new TextLoader.Arguments()
            {
                Separator = "tab",
                HasHeader = true,
                Column = new[]
                            {
                                new TextLoader.Column("Label", DataKind.Bool, 0),
                                new TextLoader.Column("SentimentText", DataKind.Text, 1)
                            }
            });

            var model = Train(mlContext, _trainDataPath);

            Evaluate(mlContext, model);

            Predict(mlContext, model);

            Console.Read();
        }

        public static ITransformer Train(MLContext mlContext, string dataPath)
        {
            IDataView dataView = _textLoader.Read(dataPath);
            var pipeline = mlContext.Transforms.Text.FeaturizeText("SentimentText", "Features")
                .Append(mlContext.BinaryClassification.Trainers.FastTree(numLeaves: 50, numTrees: 50, minDatapointsInLeaves: 20));

            Console.WriteLine("=============== Create and Train the Model ===============");
            var model = pipeline.Fit(dataView);
            Console.WriteLine("=============== End of training ===============");
            Console.WriteLine();

            return model;
        }

        public static void Evaluate(MLContext mlContext, ITransformer model)
        {
            IDataView dataView = _textLoader.Read(_testDataPath);
            Console.WriteLine("=============== Evaluating Model accuracy with Test data===============");
            var predictions = model.Transform(dataView);

            var metrics = mlContext.BinaryClassification.Evaluate(predictions, "Label");
            Console.WriteLine();
            Console.WriteLine("Model quality metrics evaluation");
            Console.WriteLine("--------------------------------");
            Console.WriteLine($"Accuracy: {metrics.Accuracy:P2}");
            Console.WriteLine($"Auc: {metrics.Auc:P2}");
            Console.WriteLine($"F1Score: {metrics.F1Score:P2}");
            Console.WriteLine("=============== End of model evaluation ===============");
        }

        private static void Predict(MLContext mlContext, ITransformer model)
        {
            var predictionFunction = model.MakePredictionFunction<SentimentData, SentimentPrediction>(mlContext);
            SentimentData sampleStatement = new SentimentData
            {
                SentimentText = "This is a very rude movie"
            };

            var resultprediction = predictionFunction.Predict(sampleStatement);

            Console.WriteLine();
            Console.WriteLine("=============== Prediction Test of model with a single sample and test dataset ===============");

            Console.WriteLine();
            Console.WriteLine($"Sentiment: {sampleStatement.SentimentText} | Prediction: {(Convert.ToBoolean(resultprediction.Prediction) ? "Toxic" : "Not Toxic")} | Probability: {resultprediction.Probability} ");
            Console.WriteLine("=============== End of Predictions ===============");
            Console.WriteLine();
        }
    }
}

程序運行后顯示的結果：

=============== Create and Train the Model ===============
=============== End of training ===============

=============== Evaluating Model accuracy with Test data===============

Model quality metrics evaluation
--------------------------------
Accuracy: 83.33%
Auc: 98.77%
F1Score: 85.71%
=============== End of model evaluation ===============

=============== Prediction Test of model with a single sample and test dataset ===============

Sentiment: This is a very rude movie | Prediction: Toxic | Probability: 0.7387648
=============== End of Predictions ===============

可以看到在預測This is a very rude movie(這是一部粗制濫造的電影)這句評論時，模型判定其是有毒的:-)

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 ML.NET 示例：二元分類之垃圾短信檢測機器學習框架ML.NET學習筆記【2】入門之二元分類 ML.NET教程之客戶細分(聚類問題) 使用ML.NET實現情感分析[新手篇] ML.NET教程之出租車車費預測(回歸問題) 使用ML.NET實現情感分析[新手篇]后補 ML.NET 1 文本挖掘之情感分析（一）使用ML.NET實現基於RFM模型的客戶價值分析 ML.NET速覽