pytorch_5.11 殘差網絡--RestNet

了解殘差網絡

• ResNet是何凱明在2015年提出的一種網絡結構
• ResNet又名殘差神經網絡，指的是在傳統卷積神經網絡中加入殘差學習（residual learning）的思想，解決了深層網絡中梯度彌散和精度下降（訓練集）的問題，使網絡能夠越來越深，既保證了精度，又控制了速度
• ResNet網絡是參考了VGG19網絡，在其基礎上進行了修改，並通過短路機制加入了殘差單元

殘差塊

• 輸入的x分成兩個路線，一條路保持原始值，另一條路進行卷積
• 殘差塊中有兩個3x3的卷積層
• 輸出兩個結果相加
  

不同的殘差網絡

詳細的網絡過程

創建殘差塊

import torch
import time
from torch import nn,optim
import torch.nn.functional as F
import pytorch_deep as pyd

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
# 創建殘差塊 rest block
class Residual(nn.Module):
    def __init__(self, in_channels,out_channels,use_1x1conv = False,stride = 1):
        super(Residual,self).__init__()
        self.conv1 = nn.Conv2d(in_channels,out_channels,kernel_size=3,padding = 1, stride = stride)
        self.conv2 = nn.Conv2d(out_channels,out_channels,kernel_size=3,padding=1)
        if use_1x1conv:
            self.conv3 = nn.Conv2d(in_channels,out_channels,kernel_size=3,padding = 1, stride = stride)
        else:self.conv3 = None
        self.bn1 = nn.BatchNorm2d(out_channels)
        self.bn2 = nn.BatchNorm2d(out_channels)
        
    def forward(self,X):
        Y = F.relu(self.bn1(self.conv1(X)))
        Y = self.bn2(self.conv2(Y))
        if self.conv3:
            X = self.conv3(X)
        return F.relu(X+Y)

blk = Residual(3,3)
X = torch.rand((4,3,6,6))
print(blk(X).shape)

torch.Size([4, 3, 6, 6])

RestNet-18 模型

前兩層網絡

net = nn.Sequential(
                    nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3),
                    nn.BatchNorm2d(64),
                    nn.ReLU(),
                    nn.MaxPool2d(kernel_size=3, stride=2, padding=1))

def resnet_block(in_channels, out_channels, num_residuals,first_block=False):
    if first_block:
        assert in_channels == out_channels # 第⼀個模塊的通道數同輸⼊通道數⼀致
    blk = []
    for i in range(num_residuals):
        if i == 0 and not first_block:
            blk.append(Residual(in_channels, out_channels,use_1x1conv=True, stride=2))
        else:
            blk.append(Residual(out_channels, out_channels))
    return nn.Sequential(*blk)

net.add_module("resnet_block1", resnet_block(64, 64, 2,first_block=True))
net.add_module("resnet_block2", resnet_block(64, 128, 2))
net.add_module("resnet_block3", resnet_block(128, 256, 2))
net.add_module("resnet_block4", resnet_block(256, 512, 2))

net.add_module("global_avg_pool", pyd.GlobalAvgPool2d()) #GlobalAvgPool2d的輸出: (Batch, 512, 1, 1)
net.add_module("fc", nn.Sequential(pyd.FlattenLayer(),
nn.Linear(512, 10)))

X = torch.rand((1, 1, 224, 224))
for name, layer in net.named_children():
    X = layer(X)
    print(name, ' output shape:\t', X.shape)

0  output shape:	 torch.Size([1, 64, 112, 112])
1  output shape:	 torch.Size([1, 64, 112, 112])
2  output shape:	 torch.Size([1, 64, 112, 112])
3  output shape:	 torch.Size([1, 64, 56, 56])
resnet_block1  output shape:	 torch.Size([1, 64, 56, 56])
resnet_block2  output shape:	 torch.Size([1, 128, 28, 28])
resnet_block3  output shape:	 torch.Size([1, 256, 14, 14])
resnet_block4  output shape:	 torch.Size([1, 512, 7, 7])
global_avg_pool  output shape:	 torch.Size([1, 512, 1, 1])
fc  output shape:	 torch.Size([1, 10])

獲取數據集並訓練

batch_size = 256
# 如出現“out of memory”的報錯信息，可減⼩batch_size或resize
train_iter, test_iter = pyd.load_data_fashion_mnist(batch_size,resize=96)
lr, num_epochs = 0.001, 5
optimizer = torch.optim.Adam(net.parameters(), lr=lr)
pyd.train_ch5(net, train_iter, test_iter, batch_size, optimizer,device, num_epochs)

training on  cuda
epoch 1, loss 0.4026, train acc 0.850, test acc 0.896,time 39.2 sec
epoch 2, loss 0.1229, train acc 0.908, test acc 0.890,time 39.0 sec
epoch 3, loss 0.0685, train acc 0.925, test acc 0.897,time 39.2 sec
epoch 4, loss 0.0449, train acc 0.933, test acc 0.921,time 39.2 sec
epoch 5, loss 0.0304, train acc 0.944, test acc 0.912,time 39.3 sec