神經網絡架構PYTORCH－前饋神經網絡

　　　首先要熟悉一下怎么使用PyTorch來實現前饋神經網絡吧．為了方便理解，我們這里只拿只有一個隱藏層的前饋神經網絡來舉例：

一個前饋神經網絡的源碼和注釋如下：比較簡單，這里就不多介紹了．

class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(NeuralNet, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size) //輸入層
        self.relu = nn.ReLU()　//隱藏網絡：elu的功能是將輸入的feature的tensor所有的元素中如果小於零的就取零。
        self.fc2 = nn.Linear(hidden_size, num_classes) //輸出層

    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        return out

　　下面要看一下怎么調用和使用前饋神經網絡的：為了提高運算效率，要把該網絡優先使用GPU來進行運算．這里的輸入尺寸和隱藏尺寸要和訓練的圖片保持一致的．

# Device configuration
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model = NeuralNet(input_size, hidden_size, num_classes).to(device)

　　為了訓練網絡，都需要定義一個loss function來描述模型對問題的求解精度。loss越小，代表模型的結果和真實值偏差越小，這里使用CrossEntropyLoss()來計算．Adam，這是一種基於一階梯度來優化隨機目標函數的算法。詳細的概念和推導我們后續再專門做分析．

criterion = nn.CrossEntropyLoss()　//針對單目標分類問題, 結合了 nn.LogSoftmax() 和 nn.NLLLoss() 來計算 loss.
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate) //優化器，設置學習的速度和使用的模型

　　接下來就是訓練模型了，訓練模型這部分是有點繞的，首先我們來看代碼，后面再針對各個函數做說明：

total_step = len(train_loader)
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        # Move tensors to the configured device
        images = images.reshape(-1, 28*28).to(device)
        labels = labels.to(device)

        # Forward pass
        outputs = model(images)
        loss = criterion(outputs, labels)　

        # Backward and optimize
        optimizer.zero_grad() //把梯度置零，也就是把loss關於weight的導數變成0.
        loss.backward()
        optimizer.step()

　　訓練模型，首先把圖片矩陣變換成25*25的矩陣單元．其次，把運算參數綁定到特定設備上．

　　然后就是網絡的前向傳播了：

outputs = model(inputs)

　　然后將輸出的outputs和原來導入的labels作為loss函數的輸入就可以得到損失了：

loss = criterion(outputs, labels)

　　計算得到loss后就要回傳損失。要注意的是這是在訓練的時候才會有的操作，測試時候只有forward過程。

loss.backward()

　　回傳損失過程中會計算梯度，然后需要根據這些梯度更新參數，optimizer.step()就是用來更新參數的。optimizer.step()后，你就可以從optimizer.param_groups[0][‘params’]里面看到各個層的梯度和權值信息。

optimizer.step()

　　測試這個模型，沒有梯度的模型，這樣就大大大額減少了內存的使用量和運算效率，這個測試模型，其實只有一個關鍵的語句就可以預測模型了，那就是：_, predicted = torch.max(outputs.data, 1)．

with torch.no_grad():
correct = 0
total = 0
for images, labels in test_loader:
images = images.reshape(-1, 28*28).to(device)
labels = labels.to(device)
outputs = model(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
print(labels.size(0))
correct += (predicted == labels).sum().item()

　　這里有個問題．訓練好的數據怎么和預測聯系起來呢？
訓練輸出的outputs也是torch.autograd.Variable格式，得到輸出后（網絡的全連接層的輸出）還希望能到到模型預測該樣本屬於哪個類別的信息，這里采用torch.max。torch.max()的第一個輸入是tensor格式，所以用outputs.data而不是outputs作為輸入；第二個參數1是代表dim的意思，也就是取每一行的最大值，其實就是我們常見的取概率最大的那個index；第三個參數loss也是torch.autograd.Variable格式。
　　總體源碼：

import torch
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms


# Device configuration
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# Hyper-parameters 
input_size = 784
hidden_size = 500
num_classes = 10
#input_size = 84
#hidden_size = 50
#num_classes = 2
num_epochs = 5
batch_size = 100
learning_rate = 0.001

# MNIST dataset 
train_dataset = torchvision.datasets.MNIST(root='../../data',
                                           train=True,
                                           transform=transforms.ToTensor(),
                                           download=True)

test_dataset = torchvision.datasets.MNIST(root='../../data',
                                          train=False,
                                          transform=transforms.ToTensor())

# Data loader
train_loader = torch.utils.data.DataLoader(dataset=train_dataset,
                                           batch_size=batch_size,
                                           shuffle=True)

test_loader = torch.utils.data.DataLoader(dataset=test_dataset,
                                          batch_size=batch_size,
                                          shuffle=False)

# Fully connected neural network with one hidden layer
class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(NeuralNet, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        return out

model = NeuralNet(input_size, hidden_size, num_classes).to(device)

# Loss and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

# Train the model
total_step = len(train_loader)
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        # Move tensors to the configured device
        images = images.reshape(-1, 28*28).to(device)
        labels = labels.to(device)

        # Forward pass
        outputs = model(images)
        loss = criterion(outputs, labels)

        # Backward and optimize
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if (i+1) % 100 == 0:
            print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
                   .format(epoch+1, num_epochs, i+1, total_step, loss.item()))
# Test the model
# In test phase, we don't need to compute gradients (for memory efficiency)
with torch.no_grad():
    correct = 0
    total = 0
    for images, labels in test_loader:
        images = images.reshape(-1, 28*28).to(device)
        labels = labels.to(device)
        outputs = model(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        #print(predicted)
        correct += (predicted == labels).sum().item()

    print('Accuracy of the network on the 10000 test images: {} %'.format(100 * correct / total))

# Save the model checkpoint
torch.save(model.state_dict(), 'model.ckpt')

　　每日一言：人之所畏，不可不畏。

　　參考文檔：

１　https://blog.csdn.net/fireflychh/article/details/75516165

２　https://blog.csdn.net/kgzhang/article/details/77479737

 

３　https://github.com/pytorch/tutorials