Pytorch-定義MLP&GPU加速&測試

Pytorch定義網絡結構識別手寫數字，可以對網絡中的參數w和b進行手動定義的（參考上一節），也可以直接用nn.Linear定義層的方式來定義，更加方便的方式是直接繼承nn.Module來定義自己的網絡結構。

1.nn.Linear方式

import torch 
import  torch.nn as nn
import  torch.nn.functional as F

#模擬一張28*28的圖片攤平
x = torch.randn(1, 784)            #shape=[1,784]

#定義三個全連接層
layer1=nn.Linear(784, 200)         #(in,out)
layer2=nn.Linear(200, 200)
layer3=nn.Linear(200, 10)

x=layer1(x)                        #shape=[1,200]
x=F.relu(x, inplace=True)          #inplace=True在原對象基礎上修改,可以節省內存

x=layer2(x)                        #shape=[1,200]
x=F.relu(x, inplace=True)

x=layer3(x)                        #shape=[1,10]
x=F.relu(x, inplace=True)

2.繼承nn.Module方式

import  torch
import  torch.nn as nn
import  torch.nn.functional as F
import  torch.optim as optim
from    torchvision import datasets, transforms

#超參數
batch_size=200
learning_rate=0.01
epochs=10

#獲取訓練數據
train_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=True, download=True,          #train=True則得到的是訓練集
                   transform=transforms.Compose([                 #transform進行數據預處理
                       transforms.ToTensor(),                     #轉成Tensor類型的數據
                       transforms.Normalize((0.1307,), (0.3081,)) #進行數據標准化(減去均值除以方差)
                   ])),
    batch_size=batch_size, shuffle=True)                          #按batch_size分出一個batch維度在最前面,shuffle=True打亂順序

#獲取測試數據
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=False, transform=transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize((0.1307,), (0.3081,))
    ])),
    batch_size=batch_size, shuffle=True)


class MLP(nn.Module):

    def __init__(self):
        super(MLP, self).__init__()
        
        self.model = nn.Sequential(         #定義網絡的每一層，nn.ReLU可以換成其他激活函數，比如nn.LeakyReLU()
            nn.Linear(784, 200),
            nn.ReLU(inplace=True),
            nn.Linear(200, 200),
            nn.ReLU(inplace=True),
            nn.Linear(200, 10),
            nn.ReLU(inplace=True),
        )

    def forward(self, x):
        x = self.model(x)
        return x    


net = MLP()
#定義sgd優化器,指明優化參數、學習率，net.parameters()得到這個類所定義的網絡的參數[[w1,b1,w2,b2,...]
optimizer = optim.SGD(net.parameters(), lr=learning_rate)
criteon = nn.CrossEntropyLoss()

for epoch in range(epochs):

    for batch_idx, (data, target) in enumerate(train_loader):
        data = data.view(-1, 28*28)          #將二維的圖片數據攤平[樣本數,784]

        logits = net(data)                   #前向傳播
        loss = criteon(logits, target)       #nn.CrossEntropyLoss()自帶Softmax

        optimizer.zero_grad()                #梯度信息清空   
        loss.backward()                      #反向傳播獲取梯度
        optimizer.step()                     #優化器更新

        if batch_idx % 100 == 0:             #每100個batch輸出一次信息
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader.dataset),
                       100. * batch_idx / len(train_loader), loss.item()))


    test_loss = 0
    correct = 0                                         #correct記錄正確分類的樣本數
    for data, target in test_loader:
        data = data.view(-1, 28 * 28)
        logits = net(data)
        test_loss += criteon(logits, target).item()     #其實就是criteon(logits, target)的值，標量
        
        pred = logits.data.max(dim=1)[1]                #也可以寫成pred=logits.argmax(dim=1)
        correct += pred.eq(target.data).sum()

    test_loss /= len(test_loader.dataset)
    print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
        test_loss, correct, len(test_loader.dataset),
        100. * correct / len(test_loader.dataset)))

Train Epoch: 0 [0/60000 (0%)] Loss: 2.307840
Train Epoch: 0 [20000/60000 (33%)] Loss: 2.022810
Train Epoch: 0 [40000/60000 (67%)] Loss: 1.342542

Test set: Average loss: 0.0038, Accuracy: 8374/10000 (84%)

Train Epoch: 1 [0/60000 (0%)] Loss: 0.802759
Train Epoch: 1 [20000/60000 (33%)] Loss: 0.627895
Train Epoch: 1 [40000/60000 (67%)] Loss: 0.482087

Test set: Average loss: 0.0020, Accuracy: 8926/10000 (89%)

Train Epoch: 2 [0/60000 (0%)] Loss: 0.496279
Train Epoch: 2 [20000/60000 (33%)] Loss: 0.420009
Train Epoch: 2 [40000/60000 (67%)] Loss: 0.429296

Test set: Average loss: 0.0017, Accuracy: 9069/10000 (91%)

Train Epoch: 3 [0/60000 (0%)] Loss: 0.304612
Train Epoch: 3 [20000/60000 (33%)] Loss: 0.356296
Train Epoch: 3 [40000/60000 (67%)] Loss: 0.405541

Test set: Average loss: 0.0015, Accuracy: 9149/10000 (91%)

Train Epoch: 4 [0/60000 (0%)] Loss: 0.304062
Train Epoch: 4 [20000/60000 (33%)] Loss: 0.406027
Train Epoch: 4 [40000/60000 (67%)] Loss: 0.385962

Test set: Average loss: 0.0014, Accuracy: 9201/10000 (92%)

Train Epoch: 5 [0/60000 (0%)] Loss: 0.186269
Train Epoch: 5 [20000/60000 (33%)] Loss: 0.196249
Train Epoch: 5 [40000/60000 (67%)] Loss: 0.228671

Test set: Average loss: 0.0013, Accuracy: 9248/10000 (92%)

Train Epoch: 6 [0/60000 (0%)] Loss: 0.364886
Train Epoch: 6 [20000/60000 (33%)] Loss: 0.295816
Train Epoch: 6 [40000/60000 (67%)] Loss: 0.244240

Test set: Average loss: 0.0012, Accuracy: 9290/10000 (93%)

Train Epoch: 7 [0/60000 (0%)] Loss: 0.228807
Train Epoch: 7 [20000/60000 (33%)] Loss: 0.192547
Train Epoch: 7 [40000/60000 (67%)] Loss: 0.223399

Test set: Average loss: 0.0012, Accuracy: 9329/10000 (93%)

Train Epoch: 8 [0/60000 (0%)] Loss: 0.176273
Train Epoch: 8 [20000/60000 (33%)] Loss: 0.346954
Train Epoch: 8 [40000/60000 (67%)] Loss: 0.253838

Test set: Average loss: 0.0011, Accuracy: 9359/10000 (94%)

Train Epoch: 9 [0/60000 (0%)] Loss: 0.246411
Train Epoch: 9 [20000/60000 (33%)] Loss: 0.201452
Train Epoch: 9 [40000/60000 (67%)] Loss: 0.162228

Test set: Average loss: 0.0011, Accuracy: 9377/10000 (94%)

區別nn.ReLU()和F.relu()

nn.ReLU()是類風格的API（大寫開頭，必須先實例化再調用，參數w、b是內部成員，通過.parameters來訪問）

F.relu()是函數風格的API（自己管理）

import  torch
import  torch.nn as nn
import  torch.nn.functional as F

x=torch.randn(1,10)
print(F.relu(x, inplace=True))   #tensor([[0.2846, 0.6158, 0.0000, 0.0000, 0.0000, 1.7980, 0.6466, 0.4263, 0.0000, 0.0000]])

layer=nn.ReLU()
print(layer(x))                  #tensor([[0.2846, 0.6158, 0.0000, 0.0000, 0.0000, 1.7980, 0.6466, 0.4263, 0.0000, 0.0000]])

3.GPU加速

Pytorch中使用torch.device()選取並返回抽象出的設備，然后在定義的網絡模塊或者Tensor后面加上.to(device變量)就可以將它們搬到設備上了。

device = torch.device('cpu:0')                     #使用第一張顯卡
net = MLP().to(device)                             #定義的網絡
criteon = nn.CrossEntropyLoss().to(device)         #損失函數

每次取出的訓練集和驗證集的batch數據放到GPU上：

data, target = data.to(device), target.cuda()     #兩種方式

應用上面的案例添加GPU加速，完整代碼如下：

import  torch
import  torch.nn as nn
import  torch.nn.functional as F
import  torch.optim as optim
from    torchvision import datasets, transforms

#超參數
batch_size=200
learning_rate=0.01
epochs=10

#獲取訓練數據
train_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=True, download=True,          #train=True則得到的是訓練集
                   transform=transforms.Compose([                 #transform進行數據預處理
                       transforms.ToTensor(),                     #轉成Tensor類型的數據
                       transforms.Normalize((0.1307,), (0.3081,)) #進行數據標准化(減去均值除以方差)
                   ])),
    batch_size=batch_size, shuffle=True)                          #按batch_size分出一個batch維度在最前面,shuffle=True打亂順序

#獲取測試數據
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=False, transform=transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize((0.1307,), (0.3081,))
    ])),
    batch_size=batch_size, shuffle=True)


class MLP(nn.Module):

    def __init__(self):
        super(MLP, self).__init__()

        self.model = nn.Sequential(         #定義網絡的每一層,
            nn.Linear(784, 200),
            nn.ReLU(inplace=True),
            nn.Linear(200, 200),
            nn.ReLU(inplace=True),
            nn.Linear(200, 10),
            nn.ReLU(inplace=True),
        )

    def forward(self, x):
        x = self.model(x)
        return x

device = torch.device('cuda:0')
net = MLP().to(device)
#定義sgd優化器,指明優化參數、學習率，net.parameters()得到這個類所定義的網絡的參數[[w1,b1,w2,b2,...]
optimizer = optim.SGD(net.parameters(), lr=learning_rate)
criteon = nn.CrossEntropyLoss().to(device)


for epoch in range(epochs):

    for batch_idx, (data, target) in enumerate(train_loader):
        data = data.view(-1, 28*28)          #將二維的圖片數據攤平[樣本數,784]
        data, target = data.to(device), target.cuda()

        logits = net(data)               #前向傳播
        loss = criteon(logits, target)       #nn.CrossEntropyLoss()自帶Softmax

        optimizer.zero_grad()                #梯度信息清空
        loss.backward()                      #反向傳播獲取梯度
        optimizer.step()                     #優化器更新

        if batch_idx % 100 == 0:             #每100個batch輸出一次信息
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader.dataset),
                       100. * batch_idx / len(train_loader), loss.item()))


    test_loss = 0
    correct = 0                                         #correct記錄正確分類的樣本數
    for data, target in test_loader:
        data = data.view(-1, 28 * 28)
        data, target = data.to(device), target.cuda()

        logits = net(data)
        test_loss += criteon(logits, target).item()     #其實就是criteon(logits, target)的值，標量

        pred = logits.data.max(dim=1)[1]                #也可以寫成pred=logits.argmax(dim=1)
        correct += pred.eq(target.data).sum()

    test_loss /= len(test_loader.dataset)
    print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
        test_loss, correct, len(test_loader.dataset),
        100. * correct / len(test_loader.dataset)))

4.多分類測試 

下面的例子中logits是一個4*10的Tensor，可以理解成4張圖片，每張圖片有10維向量的預測，然后對每一張照片的輸出值執行softmax和argmax（dim=1），得出預測標簽，與真實標簽比較，得出准確率。

import  torch
import  torch.nn.functional as F

logits = torch.rand(4, 10)
pred = F.softmax(logits, dim=1)

pred_label = pred.argmax(dim=1)                     #tensor([5, 8, 4, 7]) 和logits.argmax(dim=1)結果一樣

label = torch.tensor([5, 3, 2, 7])
correct = torch.eq(pred_label, label)               #tensor([ True, False, False,  True]) 和pred_label.eq(label)結果一樣

print(correct.sum().float().item() / len(logits))   #0.5