WassersteinGAN源碼
作者的代碼包括兩部分：models包下包含dcgan.py和mlp.py, 這兩個py文件是兩種不同的網絡結構，在dcgan.py中判別器和生成器都含有卷積網絡，而mlp.py中判別器和生成器都只是全連接。 此外main.py為主函數，通過引入import models中的生成器和判別器來完成訓練與迭代。
參數說明(main.py中)：
parser = argparse.ArgumentParser() parser.add_argument('--dataset', required=True, help='cifar10 | lsun | imagenet | folder | lfw ') parser.add_argument('--dataroot', required=True, help='path to dataset') parser.add_argument('--workers', type=int, help='number of data loading workers', default=2) parser.add_argument('--batchSize', type=int, default=64, help='input batch size') parser.add_argument('--imageSize', type=int, default=64, help='the height / width of the input image to network') parser.add_argument('--nc', type=int, default=3, help='input image channels') parser.add_argument('--nz', type=int, default=100, help='size of the latent z vector') parser.add_argument('--ngf', type=int, default=64) parser.add_argument('--ndf', type=int, default=64) parser.add_argument('--niter', type=int, default=25, help='number of epochs to train for') parser.add_argument('--lrD', type=float, default=0.00005, help='learning rate for Critic, default=0.00005') parser.add_argument('--lrG', type=float, default=0.00005, help='learning rate for Generator, default=0.00005') parser.add_argument('--beta1', type=float, default=0.5, help='beta1 for adam. default=0.5') parser.add_argument('--cuda' , action='store_true', help='enables cuda') parser.add_argument('--ngpu' , type=int, default=1, help='number of GPUs to use') parser.add_argument('--netG', default='', help="path to netG (to continue training)") parser.add_argument('--netD', default='', help="path to netD (to continue training)") parser.add_argument('--clamp_lower', type=float, default=-0.01) parser.add_argument('--clamp_upper', type=float, default=0.01) parser.add_argument('--Diters', type=int, default=5, help='number of D iters per each G iter') parser.add_argument('--noBN', action='store_true', help='use batchnorm or not (only for DCGAN)') parser.add_argument('--mlp_G', action='store_true', help='use MLP for G') parser.add_argument('--mlp_D', action='store_true', help='use MLP for D') parser.add_argument('--n_extra_layers', type=int, default=0, help='Number of extra layers on gen and disc') parser.add_argument('--experiment', default=None, help='Where to store samples and models') parser.add_argument('--adam', action='store_true', help='Whether to use adam (default is rmsprop)')
１．models包中的mlp.py:
from __future__ import absolute_import from __future__ import division from __future__ import print_function from __future__ import unicode_literals import torch import torch.nn as nn class MLP_G(nn.Module): def __init__(self, isize, nz, nc, ngf, ngpu): super(MLP_G, self).__init__() self.ngpu = ngpu main = nn.Sequential( # Z goes into a linear of size: ngf nn.Linear(nz, ngf), nn.ReLU(True), nn.Linear(ngf, ngf), nn.ReLU(True), nn.Linear(ngf, ngf), nn.ReLU(True), nn.Linear(ngf, nc * isize * isize), ) self.main = main self.nc = nc self.isize = isize self.nz = nz def forward(self, input): input = input.view(input.size(0), input.size(1)) if isinstance(input.data, torch.cuda.FloatTensor) and self.ngpu > 1: output = nn.parallel.data_parallel(self.main, input, range(self.ngpu)) else: output = self.main(input) return output.view(output.size(0), self.nc, self.isize, self.isize) class MLP_D(nn.Module): def __init__(self, isize, nz, nc, ndf, ngpu): super(MLP_D, self).__init__() self.ngpu = ngpu main = nn.Sequential( # Z goes into a linear of size: ndf nn.Linear(nc * isize * isize, ndf), nn.ReLU(True), nn.Linear(ndf, ndf), nn.ReLU(True), nn.Linear(ndf, ndf), nn.ReLU(True), nn.Linear(ndf, 1), ) self.main = main self.nc = nc self.isize = isize self.nz = nz def forward(self, input): input = input.view(input.size(0), input.size(1) * input.size(2) * input.size(3)) if isinstance(input.data, torch.cuda.FloatTensor) and self.ngpu > 1: output = nn.parallel.data_parallel(self.main, input, range(self.ngpu)) else: output = self.main(input) output = output.mean(0) return output.view(1)
在利用全連接實現的網絡中，生成器的結構為四層全連接，伴有4個ReLU激活函數。噪聲即生成器的輸入，其維度為 nz＝100維。所以生成器的輸入維度為(batch_size, nz), 輸出為圖像的尺寸(batch_size, nc, isize, isize)。注意的是torch.nn只支持mini_batch，若想輸入單個樣本，可利用input.unsqueeze(0)將batch_size設為１。WGAN的判別器與GAN不同之處是最后一層取消了sigmoid，其結構也為４層全連接。判別器的輸入為圖像的尺寸，同時判別器的輸入為生成器的輸出，而輸出為１維，即batch_size大小的向量，求mean得到一個數。
此外代碼中還對 ngpu>1 的情形下使用Multi-GPU layers： class torch.nn.DataParallel(module, device_ids=None, output_device=None, dim=0) 此容器通過將mini-batch划分到不同的設備上來實現給定module的並行。在forward過程中，module會在每個設備上都復制一遍，每個副本都會處理部分輸入。在backward過程中，副本上的梯度會累加到原始module上。
batch的大小應該大於所使用的GPU的數量。還應當是GPU個數的整數倍，這樣划分出來的每一塊都會有相同的樣本數量。
２．models包中的dcgan.py
import torch import torch.nn as nn import torch.nn.parallel class DCGAN_D(nn.Module): def __init__(self, isize, nz, nc, ndf, ngpu, n_extra_layers=0): super(DCGAN_D, self).__init__() self.ngpu = ngpu assert isize % 16 == 0, "isize has to be a multiple of 16" main = nn.Sequential() # input is nc x isize x isize main.add_module('initial.conv.{0}-{1}'.format(nc, ndf), nn.Conv2d(nc, ndf, 4, 2, 1, bias=False)) main.add_module('initial.relu.{0}'.format(ndf), nn.LeakyReLU(0.2, inplace=True)) csize, cndf = isize / 2, ndf # Extra layers for t in range(n_extra_layers): main.add_module('extra-layers-{0}.{1}.conv'.format(t, cndf), nn.Conv2d(cndf, cndf, 3, 1, 1, bias=False)) main.add_module('extra-layers-{0}.{1}.batchnorm'.format(t, cndf), nn.BatchNorm2d(cndf)) main.add_module('extra-layers-{0}.{1}.relu'.format(t, cndf), nn.LeakyReLU(0.2, inplace=True)) while csize > 4: in_feat = cndf out_feat = cndf * 2 main.add_module('pyramid.{0}-{1}.conv'.format(in_feat, out_feat), nn.Conv2d(in_feat, out_feat, 4, 2, 1, bias=False)) main.add_module('pyramid.{0}.batchnorm'.format(out_feat), nn.BatchNorm2d(out_feat)) main.add_module('pyramid.{0}.relu'.format(out_feat), nn.LeakyReLU(0.2, inplace=True)) cndf = cndf * 2 csize = csize / 2 # state size. K x 4 x 4 main.add_module('final.{0}-{1}.conv'.format(cndf, 1), nn.Conv2d(cndf, 1, 4, 1, 0, bias=False)) self.main = main def forward(self, input): if isinstance(input.data, torch.cuda.FloatTensor) and self.ngpu > 1: output = nn.parallel.data_parallel(self.main, input, range(self.ngpu)) else: output = self.main(input) output = output.mean(0) return output.view(1) class DCGAN_G(nn.Module): def __init__(self, isize, nz, nc, ngf, ngpu, n_extra_layers=0): super(DCGAN_G, self).__init__() self.ngpu = ngpu assert isize % 16 == 0, "isize has to be a multiple of 16" cngf, tisize = ngf//2, 4 while tisize != isize: cngf = cngf * 2 tisize = tisize * 2 main = nn.Sequential() # input is Z, going into a convolution main.add_module('initial.{0}-{1}.convt'.format(nz, cngf), nn.ConvTranspose2d(nz, cngf, 4, 1, 0, bias=False)) main.add_module('initial.{0}.batchnorm'.format(cngf), nn.BatchNorm2d(cngf)) main.add_module('initial.{0}.relu'.format(cngf), nn.ReLU(True)) csize, cndf = 4, cngf while csize < isize//2: main.add_module('pyramid.{0}-{1}.convt'.format(cngf, cngf//2), nn.ConvTranspose2d(cngf, cngf//2, 4, 2, 1, bias=False)) main.add_module('pyramid.{0}.batchnorm'.format(cngf//2), nn.BatchNorm2d(cngf//2)) main.add_module('pyramid.{0}.relu'.format(cngf//2), nn.ReLU(True)) cngf = cngf // 2 csize = csize * 2 # Extra layers for t in range(n_extra_layers): main.add_module('extra-layers-{0}.{1}.conv'.format(t, cngf), nn.Conv2d(cngf, cngf, 3, 1, 1, bias=False)) main.add_module('extra-layers-{0}.{1}.batchnorm'.format(t, cngf), nn.BatchNorm2d(cngf)) main.add_module('extra-layers-{0}.{1}.relu'.format(t, cngf), nn.ReLU(True)) main.add_module('final.{0}-{1}.convt'.format(cngf, nc), nn.ConvTranspose2d(cngf, nc, 4, 2, 1, bias=False)) main.add_module('final.{0}.tanh'.format(nc), nn.Tanh()) self.main = main def forward(self, input): if isinstance(input.data, torch.cuda.FloatTensor) and self.ngpu > 1: output = nn.parallel.data_parallel(self.main, input, range(self.ngpu)) else: output = self.main(input) return output ############################################################################### class DCGAN_D_nobn(nn.Module): def __init__(self, isize, nz, nc, ndf, ngpu, n_extra_layers=0): super(DCGAN_D_nobn, self).__init__() self.ngpu = ngpu assert isize % 16 == 0, "isize has to be a multiple of 16" main = nn.Sequential() # input is nc x isize x isize # input is nc x isize x isize main.add_module('initial.conv.{0}-{1}'.format(nc, ndf), nn.Conv2d(nc, ndf, 4, 2, 1, bias=False)) main.add_module('initial.relu.{0}'.format(ndf), nn.LeakyReLU(0.2, inplace=True)) csize, cndf = isize / 2, ndf # Extra layers for t in range(n_extra_layers): main.add_module('extra-layers-{0}.{1}.conv'.format(t, cndf), nn.Conv2d(cndf, cndf, 3, 1, 1, bias=False)) main.add_module('extra-layers-{0}.{1}.relu'.format(t, cndf), nn.LeakyReLU(0.2, inplace=True)) while csize > 4: in_feat = cndf out_feat = cndf * 2 main.add_module('pyramid.{0}-{1}.conv'.format(in_feat, out_feat), nn.Conv2d(in_feat, out_feat, 4, 2, 1, bias=False)) main.add_module('pyramid.{0}.relu'.format(out_feat), nn.LeakyReLU(0.2, inplace=True)) cndf = cndf * 2 csize = csize / 2 # state size. K x 4 x 4 main.add_module('final.{0}-{1}.conv'.format(cndf, 1), nn.Conv2d(cndf, 1, 4, 1, 0, bias=False)) self.main = main def forward(self, input): if isinstance(input.data, torch.cuda.FloatTensor) and self.ngpu > 1: output = nn.parallel.data_parallel(self.main, input, range(self.ngpu)) else: output = self.main(input) output = output.mean(0) return output.view(1) class DCGAN_G_nobn(nn.Module): def __init__(self, isize, nz, nc, ngf, ngpu, n_extra_layers=0): super(DCGAN_G_nobn, self).__init__() self.ngpu = ngpu assert isize % 16 == 0, "isize has to be a multiple of 16" cngf, tisize = ngf//2, 4 while tisize != isize: cngf = cngf * 2 tisize = tisize * 2 main = nn.Sequential() main.add_module('initial.{0}-{1}.convt'.format(nz, cngf), nn.ConvTranspose2d(nz, cngf, 4, 1, 0, bias=False)) main.add_module('initial.{0}.relu'.format(cngf), nn.ReLU(True)) csize, cndf = 4, cngf while csize < isize//2: main.add_module('pyramid.{0}-{1}.convt'.format(cngf, cngf//2), nn.ConvTranspose2d(cngf, cngf//2, 4, 2, 1, bias=False)) main.add_module('pyramid.{0}.relu'.format(cngf//2), nn.ReLU(True)) cngf = cngf // 2 csize = csize * 2 # Extra layers for t in range(n_extra_layers): main.add_module('extra-layers-{0}.{1}.conv'.format(t, cngf), nn.Conv2d(cngf, cngf, 3, 1, 1, bias=False)) main.add_module('extra-layers-{0}.{1}.relu'.format(t, cngf), nn.ReLU(True)) main.add_module('final.{0}-{1}.convt'.format(cngf, nc), nn.ConvTranspose2d(cngf, nc, 4, 2, 1, bias=False)) main.add_module('final.{0}.tanh'.format(nc), nn.Tanh()) self.main = main def forward(self, input): if isinstance(input.data, torch.cuda.FloatTensor) and self.ngpu > 1: output = nn.parallel.data_parallel(self.main, input, range(self.ngpu)) else: output = self.main(input) return output
此文件中共４個類，分為兩組。第一組是DCGAN_D和DCGAN_G, 這兩個類都使用了Batch normalization。而另一組是DCGAN_D_nobn和DCGAN_G_nobn, 這兩個類都沒有使用Batch normalization。首先看判別器，設定了image的尺寸為１６的倍數，然后經過一個卷積層和一個LeakyReLU后來到Extra layers, 在這個其他層中當參數 n_extra_layers 為n時, 將Conv-BN-LeakyReLU重復n次，此時判斷如果特征圖大小 >4, 則再次進行Conv-BN-LeakyReLU操作直到特征圖大小 ＝4，然后進行最后一次卷積核大小為４的卷積，此時輸出為１維向量，求均值后得到一個數。
然后看生成器，生成器用到了反卷積，因為其輸入為100維噪聲數據（類似向量），輸出為圖像（類似矩陣）。首先經過ConvTranspose2d-BN-ReLU, 將100維的噪聲反卷積為512維。然后經過一系列（３次）ConvTranspose2d-BN-ReLU將特征圖維度改為了64通道。此時又來到了Extra layers, 在這個其他層中當參數 n_extra_layers 為n時, 將ConvTranspose2d-BN-ReLU重復n次，注意此時n次反卷積設置為通道數不變的反卷積，所以若經過這ｎ次操作，通道數仍為64維。最后經過ConvTranspose2d-Tanh后，將通道數將為了３，數值大小都在-1至１之間。
對於兩組文件不同之處只有BN的使用與否，所以不必贅述。
３．main.py
from __future__ import print_function import argparse import random import torch import torch.nn as nn import torch.nn.parallel import torch.backends.cudnn as cudnn import torch.optim as optim import torch.utils.data import torchvision.datasets as dset import torchvision.transforms as transforms import torchvision.utils as vutils from torch.autograd import Variable import os import models.dcgan as dcgan import models.mlp as mlp parser = argparse.ArgumentParser() parser.add_argument('--dataset', required=True, help='cifar10 | lsun | imagenet | folder | lfw ') parser.add_argument('--dataroot', required=True, help='path to dataset') parser.add_argument('--workers', type=int, help='number of data loading workers', default=2) parser.add_argument('--batchSize', type=int, default=64, help='input batch size') parser.add_argument('--imageSize', type=int, default=64, help='the height / width of the input image to network') parser.add_argument('--nc', type=int, default=3, help='input image channels') parser.add_argument('--nz', type=int, default=100, help='size of the latent z vector') parser.add_argument('--ngf', type=int, default=64) parser.add_argument('--ndf', type=int, default=64) parser.add_argument('--niter', type=int, default=25, help='number of epochs to train for') parser.add_argument('--lrD', type=float, default=0.00005, help='learning rate for Critic, default=0.00005') parser.add_argument('--lrG', type=float, default=0.00005, help='learning rate for Generator, default=0.00005') parser.add_argument('--beta1', type=float, default=0.5, help='beta1 for adam. default=0.5') parser.add_argument('--cuda' , action='store_true', help='enables cuda') parser.add_argument('--ngpu' , type=int, default=1, help='number of GPUs to use') parser.add_argument('--netG', default='', help="path to netG (to continue training)") parser.add_argument('--netD', default='', help="path to netD (to continue training)") parser.add_argument('--clamp_lower', type=float, default=-0.01) parser.add_argument('--clamp_upper', type=float, default=0.01) parser.add_argument('--Diters', type=int, default=5, help='number of D iters per each G iter') parser.add_argument('--noBN', action='store_true', help='use batchnorm or not (only for DCGAN)') parser.add_argument('--mlp_G', action='store_true', help='use MLP for G') parser.add_argument('--mlp_D', action='store_true', help='use MLP for D') parser.add_argument('--n_extra_layers', type=int, default=0, help='Number of extra layers on gen and disc') parser.add_argument('--experiment', default=None, help='Where to store samples and models') parser.add_argument('--adam', action='store_true', help='Whether to use adam (default is rmsprop)') opt = parser.parse_args() print(opt) if opt.experiment is None: opt.experiment = 'samples' os.system('mkdir {0}'.format(opt.experiment)) opt.manualSeed = random.randint(1, 10000) # fix seed print("Random Seed: ", opt.manualSeed) random.seed(opt.manualSeed) torch.manual_seed(opt.manualSeed) cudnn.benchmark = True if torch.cuda.is_available() and not opt.cuda: print("WARNING: You have a CUDA device, so you should probably run with --cuda") if opt.dataset in ['imagenet', 'folder', 'lfw']: # folder dataset dataset = dset.ImageFolder(root=opt.dataroot, transform=transforms.Compose([ transforms.Scale(opt.imageSize), transforms.CenterCrop(opt.imageSize), transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), ])) elif opt.dataset == 'lsun': dataset = dset.LSUN(db_path=opt.dataroot, classes=['bedroom_train'], transform=transforms.Compose([ transforms.Scale(opt.imageSize), transforms.CenterCrop(opt.imageSize), transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), ])) elif opt.dataset == 'cifar10': dataset = dset.CIFAR10(root=opt.dataroot, download=True, transform=transforms.Compose([ transforms.Scale(opt.imageSize), transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), ]) ) assert dataset dataloader = torch.utils.data.DataLoader(dataset, batch_size=opt.batchSize, shuffle=True, num_workers=int(opt.workers)) ngpu = int(opt.ngpu) nz = int(opt.nz) ngf = int(opt.ngf) ndf = int(opt.ndf) nc = int(opt.nc) n_extra_layers = int(opt.n_extra_layers) # custom weights initialization called on netG and netD def weights_init(m): classname = m.__class__.__name__ if classname.find('Conv') != -1: m.weight.data.normal_(0.0, 0.02) elif classname.find('BatchNorm') != -1: m.weight.data.normal_(1.0, 0.02) m.bias.data.fill_(0) if opt.noBN: netG = dcgan.DCGAN_G_nobn(opt.imageSize, nz, nc, ngf, ngpu, n_extra_layers) elif opt.mlp_G: netG = mlp.MLP_G(opt.imageSize, nz, nc, ngf, ngpu) else: netG = dcgan.DCGAN_G(opt.imageSize, nz, nc, ngf, ngpu, n_extra_layers) netG.apply(weights_init) if opt.netG != '': # load checkpoint if needed netG.load_state_dict(torch.load(opt.netG)) print(netG) if opt.mlp_D: netD = mlp.MLP_D(opt.imageSize, nz, nc, ndf, ngpu) else: netD = dcgan.DCGAN_D(opt.imageSize, nz, nc, ndf, ngpu, n_extra_layers) netD.apply(weights_init) if opt.netD != '': netD.load_state_dict(torch.load(opt.netD)) print(netD) input = torch.FloatTensor(opt.batchSize, 3, opt.imageSize, opt.imageSize) noise = torch.FloatTensor(opt.batchSize, nz, 1, 1) fixed_noise = torch.FloatTensor(opt.batchSize, nz, 1, 1).normal_(0, 1) one = torch.FloatTensor([1]) mone = one * -1 if opt.cuda: netD.cuda() netG.cuda() input = input.cuda() one, mone = one.cuda(), mone.cuda() noise, fixed_noise = noise.cuda(), fixed_noise.cuda() # setup optimizer if opt.adam: optimizerD = optim.Adam(netD.parameters(), lr=opt.lrD, betas=(opt.beta1, 0.999)) optimizerG = optim.Adam(netG.parameters(), lr=opt.lrG, betas=(opt.beta1, 0.999)) else: optimizerD = optim.RMSprop(netD.parameters(), lr = opt.lrD) optimizerG = optim.RMSprop(netG.parameters(), lr = opt.lrG) gen_iterations = 0 for epoch in range(opt.niter): data_iter = iter(dataloader) i = 0 while i < len(dataloader): ############################ # (1) Update D network ########################### for p in netD.parameters(): # reset requires_grad p.requires_grad = True # they are set to False below in netG update # train the discriminator Diters times if gen_iterations < 25 or gen_iterations % 500 == 0: Diters = 100 else: Diters = opt.Diters j = 0 while j < Diters and i < len(dataloader): j += 1 # clamp parameters to a cube for p in netD.parameters(): p.data.clamp_(opt.clamp_lower, opt.clamp_upper) data = data_iter.next() i += 1 # train with real real_cpu, _ = data netD.zero_grad() batch_size = real_cpu.size(0) if opt.cuda: real_cpu = real_cpu.cuda() input.resize_as_(real_cpu).copy_(real_cpu) inputv = Variable(input) errD_real = netD(inputv) errD_real.backward(one) # train with fake noise.resize_(opt.batchSize, nz, 1, 1).normal_(0, 1) noisev = Variable(noise, volatile = True) # totally freeze netG fake = Variable(netG(noisev).data) inputv = fake errD_fake = netD(inputv) errD_fake.backward(mone) errD = errD_real - errD_fake optimizerD.step() ############################ # (2) Update G network ########################### for p in netD.parameters(): p.requires_grad = False # to avoid computation netG.zero_grad() # in case our last batch was the tail batch of the dataloader, # make sure we feed a full batch of noise noise.resize_(opt.batchSize, nz, 1, 1).normal_(0, 1) noisev = Variable(noise) fake = netG(noisev) errG = netD(fake) errG.backward(one) optimizerG.step() gen_iterations += 1 print('[%d/%d][%d/%d][%d] Loss_D: %f Loss_G: %f Loss_D_real: %f Loss_D_fake %f' % (epoch, opt.niter, i, len(dataloader), gen_iterations, errD.data[0], errG.data[0], errD_real.data[0], errD_fake.data[0])) if gen_iterations % 500 == 0: real_cpu = real_cpu.mul(0.5).add(0.5) vutils.save_image(real_cpu, '{0}/real_samples.png'.format(opt.experiment)) fake = netG(Variable(fixed_noise, volatile=True)) fake.data = fake.data.mul(0.5).add(0.5) vutils.save_image(fake.data, '{0}/fake_samples_{1}.png'.format(opt.experiment, gen_iterations)) # do checkpointing torch.save(netG.state_dict(), '{0}/netG_epoch_{1}.pth'.format(opt.experiment, epoch)) torch.save(netD.state_dict(), '{0}/netD_epoch_{1}.pth'.format(opt.experiment, epoch))
首先利用parser = argparse.ArgumentParser()命令行解析工具設置了一堆參數，如文章最開始處。
然后利用net.apply(weighs_init)遞歸進行權重初始化：
def weights_init(m): classname = m.__class__.__name__ if classname.find('Conv') != -1: m.weight.data.normal_(0.0, 0.02) elif classname.find('BatchNorm') != -1: m.weight.data.normal_(1.0, 0.02) m.bias.data.fill_(0)
接着選擇dcgan結構或者mlp結構，盡量不要在 D 中使用 batch normalization，即不選DCGAN_D_nobn。可選的優化器為Adam和RMSprop。在ＷGAN中盡量使用RMSProp 或 SGD 。
迭代訓練的epoch為25，注意參數 Diters默認為５,是指每迭代generator 1 次，迭代descriminator 5 次。為什么迭代判別器多於生成器呢？因為如果判別器太弱，那么生成器的質量就會受到影響導致生成的圖片質量太低。在代碼中我們看可以看到：
if gen_iterations < 25 or gen_iterations % 500 == 0: Diters = 100 else: Diters = opt.Diters
gen_iterations（生成器迭代次數）在epoch循環外被初始化為０，也即是說當在第一個epoch中，將判別器迭代100次，將生成器迭代１次。然后當gen_iteration>=25時，即生成器迭代了２５次以上時，生成器每迭代一次，判別器迭代默認的５次。此外還有一些細節例如：
one = torch.FloatTensor([1]) mone = one * -1 ... errD_real.backward(one) ... errD_fake.backward(mone) ... errG.backward(one)
為啥反向傳播參數有的為１，有的為－１？先看WGAN的損失函數：
即對於判別器：maxmize LD
對於生成器：maxmize LG
最大化判ＬＤ可以看做最大化E(D(x)), 最小化Ｅ(D(G(z)))。所以第一項求梯度后系數為１，即梯度上升求最大值，而第二項最小化得利用梯度下降法，所以反向傳播時得加個負號變為負梯度。
最大化ＬＧ可以看做最大化Ｅ(D(G(z)))。即利用梯度上升法，所以反向傳播時系數為１，即梯度方向就是最速上升方向。
附：DCGAN、ＷGAN實現。