Pytorch自定義參數層

本文轉載自查看原文 2019-07-18 16:02 1321 PyTorch

注意，一般官方接口都帶有可導功能，如果你實現的層不具有可導功能，就需要自己實現梯度的反向傳遞。

 1 官方Linear層：
 2 class Linear(Module):
 3     def __init__(self, in_features, out_features, bias=True):
 4         super(Linear, self).__init__()
 5         self.in_features = in_features
 6         self.out_features = out_features
 7         self.weight = Parameter(torch.Tensor(out_features, in_features))
 8         if bias:
 9             self.bias = Parameter(torch.Tensor(out_features))
10         else:
11             self.register_parameter('bias', None)
12         self.reset_parameters()
13 
14     def reset_parameters(self):
15         stdv = 1. / math.sqrt(self.weight.size(1))
16         self.weight.data.uniform_(-stdv, stdv)
17         if self.bias is not None:
18             self.bias.data.uniform_(-stdv, stdv)
19 
20     def forward(self, input):
21         return F.linear(input, self.weight, self.bias)
22 
23     def extra_repr(self):
24         return 'in_features={}, out_features={}, bias={}'.format(
25             self.in_features, self.out_features, self.bias is not None
26         )

1 實現view層
2 class Reshape(nn.Module):
3     def __init__(self, *args):
4         super(Reshape, self).__init__()
5         self.shape = args
6 
7     def forward(self, x):
8         return x.view((x.size(0),)+self.shape)

 1 實現LinearWise層
 2 class LinearWise(nn.Module):
 3     def __init__(self, in_features, bias=True):
 4         super(LinearWise, self).__init__()
 5         self.in_features = in_features
 6 
 7         self.weight = nn.Parameter(torch.Tensor(self.in_features))
 8         if bias:
 9             self.bias = nn.Parameter(torch.Tensor(self.in_features))
10         else:
11             self.register_parameter('bias', None)
12         self.reset_parameters()
13 
14     def reset_parameters(self):
15         stdv = 1. / math.sqrt(self.weight.size(0))
16         self.weight.data.uniform_(-stdv, stdv)
17         if self.bias is not None:
18             self.bias.data.uniform_(-stdv, stdv)
19 
20     def forward(self, input):
21         x = input * self.weight
22         if self.bias is not None:
23             x = x + self.bias
24         return x

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 springboot 接口層參數校驗自定義參數校驗 pytorch自定義算子 keras自定義層 PyTorch(二)——搭建和自定義網絡自定義 div 彈窗，遮罩層 angular 自定義指令參數詳解 Jenkins 傳遞自定義的參數自定義注解實現參數校驗 iview @change添加自定義參數 springmvc自定義參數轉換