（原）torch的apply函數

轉載請注明出處：

http://www.cnblogs.com/darkknightzh/p/6221633.html

 

torch中的apply函數通過可以不斷遍歷model的各個模塊。實際上其使用的是深度優先算法。

其具體代碼如下所示（代碼見torch/install/share/lua/5.1/nn/Module.lua）：

-- Run a callback (called with the module as an argument) in preorder over this
-- module and its children.
--
function Module:apply(callback)
    callback(self)

    if self.modules then
        for _, module in ipairs(self.modules) do
            module:apply(callback)
        end
    end
end

可見，apply遞歸調用自身，直到不存在模塊為止（這樣說不太合理）。

 

如下所示的測試代碼：

require "dpnn"

function createModel()
   local net = nn.Sequential()

   net:add(nn.SpatialConvolutionMM(3, 64, 7, 7, 2, 2, 3, 3))
   net:add(nn.SpatialBatchNormalization(64))
   net:add(nn.ReLU())
   net:add(nn.SpatialMaxPooling(3, 3, 2, 2, 1, 1))

   net:add(nn.Inception{
     inputSize = 192,
     kernelSize = {3, 5},
     kernelStride = {1, 1},
     outputSize = {128, 32},
     reduceSize = {96, 16, 32, 64},
     pool = nn.SpatialMaxPooling(3, 3, 1, 1, 1, 1),
     batchNorm = true
   })

   net:add(nn.Inception{
     inputSize = 256,
     kernelSize = {3, 5},
     kernelStride = {1, 1},
     outputSize = {128, 64},
     reduceSize = {96, 32, 64, 64},
     pool = nn.SpatialLPPooling(256, 2, 3, 3, 1, 1),
     batchNorm = false
   })

   net:add(nn.SpatialAveragePooling(7, 7))
   net:add(nn.View(320))
   net:add(nn.Linear(320, 128))
   net:add(nn.Normalize(2))

   return net
end


torch.setdefaulttensortype('torch.FloatTensor')

local model = createModel()

--print(model)
tt = 0
model:apply(function(module)
    tt = tt + 1
    print(tt, module)
end)

其輸出結果為：

1	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> (7) -> (8) -> (9) -> (10) -> output]
  (1): nn.SpatialConvolutionMM(3 -> 64, 7x7, 2,2, 3,3)
  (2): nn.SpatialBatchNormalization
  (3): nn.ReLU
  (4): nn.SpatialMaxPooling(3x3, 2,2, 1,1)
  (5): nn.Inception @ nn.DepthConcat {
    input
      |`-> (1): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
      |      (1): nn.SpatialConvolution(192 -> 96, 1x1)
      |      (2): nn.SpatialBatchNormalization
      |      (3): nn.ReLU
      |      (4): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
      |      (5): nn.SpatialBatchNormalization
      |      (6): nn.ReLU
      |    }
      |`-> (2): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
      |      (1): nn.SpatialConvolution(192 -> 16, 1x1)
      |      (2): nn.SpatialBatchNormalization
      |      (3): nn.ReLU
      |      (4): nn.SpatialConvolution(16 -> 32, 5x5, 1,1, 2,2)
      |      (5): nn.SpatialBatchNormalization
      |      (6): nn.ReLU
      |    }
      |`-> (3): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> (4) -> output]
      |      (1): nn.SpatialMaxPooling(3x3, 1,1, 1,1)
      |      (2): nn.SpatialConvolution(192 -> 32, 1x1)
      |      (3): nn.SpatialBatchNormalization
      |      (4): nn.ReLU
      |    }
      |`-> (4): nn.Sequential {
             [input -> (1) -> (2) -> (3) -> output]
             (1): nn.SpatialConvolution(192 -> 64, 1x1)
             (2): nn.SpatialBatchNormalization
             (3): nn.ReLU
           }
       ... -> output
  }
  (6): nn.Inception @ nn.DepthConcat {
    input
      |`-> (1): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> (4) -> output]
      |      (1): nn.SpatialConvolution(256 -> 96, 1x1)
      |      (2): nn.ReLU
      |      (3): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
      |      (4): nn.ReLU
      |    }
      |`-> (2): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> (4) -> output]
      |      (1): nn.SpatialConvolution(256 -> 32, 1x1)
      |      (2): nn.ReLU
      |      (3): nn.SpatialConvolution(32 -> 64, 5x5, 1,1, 2,2)
      |      (4): nn.ReLU
      |    }
      |`-> (3): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> output]
      |      (1): nn.Sequential {
      |        [input -> (1) -> (2) -> (3) -> (4) -> output]
      |        (1): nn.Square
      |        (2): nn.SpatialAveragePooling(3x3, 1,1)
      |        (3): nn.MulConstant
      |        (4): nn.Sqrt
      |      }
      |      (2): nn.SpatialConvolution(256 -> 64, 1x1)
      |      (3): nn.ReLU
      |    }
      |`-> (4): nn.Sequential {
             [input -> (1) -> (2) -> output]
             (1): nn.SpatialConvolution(256 -> 64, 1x1)
             (2): nn.ReLU
           }
       ... -> output
  }
  (7): nn.SpatialAveragePooling(7x7, 1,1)
  (8): nn.View(320)
  (9): nn.Linear(320 -> 128)
  (10): nn.Normalize(2)
}
2	nn.SpatialConvolutionMM(3 -> 64, 7x7, 2,2, 3,3)
3	nn.SpatialBatchNormalization
4	nn.ReLU
5	nn.SpatialMaxPooling(3x3, 2,2, 1,1)
6	nn.Inception @ nn.DepthConcat {
  input
    |`-> (1): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
    |      (1): nn.SpatialConvolution(192 -> 96, 1x1)
    |      (2): nn.SpatialBatchNormalization
    |      (3): nn.ReLU
    |      (4): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
    |      (5): nn.SpatialBatchNormalization
    |      (6): nn.ReLU
    |    }
    |`-> (2): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
    |      (1): nn.SpatialConvolution(192 -> 16, 1x1)
    |      (2): nn.SpatialBatchNormalization
    |      (3): nn.ReLU
    |      (4): nn.SpatialConvolution(16 -> 32, 5x5, 1,1, 2,2)
    |      (5): nn.SpatialBatchNormalization
    |      (6): nn.ReLU
    |    }
    |`-> (3): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> output]
    |      (1): nn.SpatialMaxPooling(3x3, 1,1, 1,1)
    |      (2): nn.SpatialConvolution(192 -> 32, 1x1)
    |      (3): nn.SpatialBatchNormalization
    |      (4): nn.ReLU
    |    }
    |`-> (4): nn.Sequential {
           [input -> (1) -> (2) -> (3) -> output]
           (1): nn.SpatialConvolution(192 -> 64, 1x1)
           (2): nn.SpatialBatchNormalization
           (3): nn.ReLU
         }
     ... -> output
}
7	nn.DepthConcat {
  input
    |`-> (1): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
    |      (1): nn.SpatialConvolution(192 -> 96, 1x1)
    |      (2): nn.SpatialBatchNormalization
    |      (3): nn.ReLU
    |      (4): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
    |      (5): nn.SpatialBatchNormalization
    |      (6): nn.ReLU
    |    }
    |`-> (2): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
    |      (1): nn.SpatialConvolution(192 -> 16, 1x1)
    |      (2): nn.SpatialBatchNormalization
    |      (3): nn.ReLU
    |      (4): nn.SpatialConvolution(16 -> 32, 5x5, 1,1, 2,2)
    |      (5): nn.SpatialBatchNormalization
    |      (6): nn.ReLU
    |    }
    |`-> (3): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> output]
    |      (1): nn.SpatialMaxPooling(3x3, 1,1, 1,1)
    |      (2): nn.SpatialConvolution(192 -> 32, 1x1)
    |      (3): nn.SpatialBatchNormalization
    |      (4): nn.ReLU
    |    }
    |`-> (4): nn.Sequential {
           [input -> (1) -> (2) -> (3) -> output]
           (1): nn.SpatialConvolution(192 -> 64, 1x1)
           (2): nn.SpatialBatchNormalization
           (3): nn.ReLU
         }
     ... -> output
}
8	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
  (1): nn.SpatialConvolution(192 -> 96, 1x1)
  (2): nn.SpatialBatchNormalization
  (3): nn.ReLU
  (4): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
  (5): nn.SpatialBatchNormalization
  (6): nn.ReLU
}
9	nn.SpatialConvolution(192 -> 96, 1x1)
10	nn.SpatialBatchNormalization
11	nn.ReLU
12	nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
13	nn.SpatialBatchNormalization
14	nn.ReLU
15	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
  (1): nn.SpatialConvolution(192 -> 16, 1x1)
  (2): nn.SpatialBatchNormalization
  (3): nn.ReLU
  (4): nn.SpatialConvolution(16 -> 32, 5x5, 1,1, 2,2)
  (5): nn.SpatialBatchNormalization
  (6): nn.ReLU
}
16	nn.SpatialConvolution(192 -> 16, 1x1)
17	nn.SpatialBatchNormalization
18	nn.ReLU
19	nn.SpatialConvolution(16 -> 32, 5x5, 1,1, 2,2)
20	nn.SpatialBatchNormalization
21	nn.ReLU
22	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> output]
  (1): nn.SpatialMaxPooling(3x3, 1,1, 1,1)
  (2): nn.SpatialConvolution(192 -> 32, 1x1)
  (3): nn.SpatialBatchNormalization
  (4): nn.ReLU
}
23	nn.SpatialMaxPooling(3x3, 1,1, 1,1)
24	nn.SpatialConvolution(192 -> 32, 1x1)
25	nn.SpatialBatchNormalization
26	nn.ReLU
27	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> output]
  (1): nn.SpatialConvolution(192 -> 64, 1x1)
  (2): nn.SpatialBatchNormalization
  (3): nn.ReLU
}
28	nn.SpatialConvolution(192 -> 64, 1x1)
29	nn.SpatialBatchNormalization
30	nn.ReLU
31	nn.Inception @ nn.DepthConcat {
  input
    |`-> (1): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> output]
    |      (1): nn.SpatialConvolution(256 -> 96, 1x1)
    |      (2): nn.ReLU
    |      (3): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
    |      (4): nn.ReLU
    |    }
    |`-> (2): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> output]
    |      (1): nn.SpatialConvolution(256 -> 32, 1x1)
    |      (2): nn.ReLU
    |      (3): nn.SpatialConvolution(32 -> 64, 5x5, 1,1, 2,2)
    |      (4): nn.ReLU
    |    }
    |`-> (3): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> output]
    |      (1): nn.Sequential {
    |        [input -> (1) -> (2) -> (3) -> (4) -> output]
    |        (1): nn.Square
    |        (2): nn.SpatialAveragePooling(3x3, 1,1)
    |        (3): nn.MulConstant
    |        (4): nn.Sqrt
    |      }
    |      (2): nn.SpatialConvolution(256 -> 64, 1x1)
    |      (3): nn.ReLU
    |    }
    |`-> (4): nn.Sequential {
           [input -> (1) -> (2) -> output]
           (1): nn.SpatialConvolution(256 -> 64, 1x1)
           (2): nn.ReLU
         }
     ... -> output
}
32	nn.DepthConcat {
  input
    |`-> (1): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> output]
    |      (1): nn.SpatialConvolution(256 -> 96, 1x1)
    |      (2): nn.ReLU
    |      (3): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
    |      (4): nn.ReLU
    |    }
    |`-> (2): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> (4) -> output]
    |      (1): nn.SpatialConvolution(256 -> 32, 1x1)
    |      (2): nn.ReLU
    |      (3): nn.SpatialConvolution(32 -> 64, 5x5, 1,1, 2,2)
    |      (4): nn.ReLU
    |    }
    |`-> (3): nn.Sequential {
    |      [input -> (1) -> (2) -> (3) -> output]
    |      (1): nn.Sequential {
    |        [input -> (1) -> (2) -> (3) -> (4) -> output]
    |        (1): nn.Square
    |        (2): nn.SpatialAveragePooling(3x3, 1,1)
    |        (3): nn.MulConstant
    |        (4): nn.Sqrt
    |      }
    |      (2): nn.SpatialConvolution(256 -> 64, 1x1)
    |      (3): nn.ReLU
    |    }
    |`-> (4): nn.Sequential {
           [input -> (1) -> (2) -> output]
           (1): nn.SpatialConvolution(256 -> 64, 1x1)
           (2): nn.ReLU
         }
     ... -> output
}
33	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> output]
  (1): nn.SpatialConvolution(256 -> 96, 1x1)
  (2): nn.ReLU
  (3): nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
  (4): nn.ReLU
}
34	nn.SpatialConvolution(256 -> 96, 1x1)
35	nn.ReLU
36	nn.SpatialConvolution(96 -> 128, 3x3, 1,1, 1,1)
37	nn.ReLU
38	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> output]
  (1): nn.SpatialConvolution(256 -> 32, 1x1)
  (2): nn.ReLU
  (3): nn.SpatialConvolution(32 -> 64, 5x5, 1,1, 2,2)
  (4): nn.ReLU
}
39	nn.SpatialConvolution(256 -> 32, 1x1)
40	nn.ReLU
41	nn.SpatialConvolution(32 -> 64, 5x5, 1,1, 2,2)
42	nn.ReLU
43	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> output]
  (1): nn.Sequential {
    [input -> (1) -> (2) -> (3) -> (4) -> output]
    (1): nn.Square
    (2): nn.SpatialAveragePooling(3x3, 1,1)
    (3): nn.MulConstant
    (4): nn.Sqrt
  }
  (2): nn.SpatialConvolution(256 -> 64, 1x1)
  (3): nn.ReLU
}
44	nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> output]
  (1): nn.Square
  (2): nn.SpatialAveragePooling(3x3, 1,1)
  (3): nn.MulConstant
  (4): nn.Sqrt
}
45	nn.Square
46	nn.SpatialAveragePooling(3x3, 1,1)
47	nn.MulConstant
48	nn.Sqrt
49	nn.SpatialConvolution(256 -> 64, 1x1)
50	nn.ReLU
51	nn.Sequential {
  [input -> (1) -> (2) -> output]
  (1): nn.SpatialConvolution(256 -> 64, 1x1)
  (2): nn.ReLU
}
52	nn.SpatialConvolution(256 -> 64, 1x1)
53	nn.ReLU
54	nn.SpatialAveragePooling(7x7, 1,1)
55	nn.View(320)
56	nn.Linear(320 -> 128)
57	nn.Normalize(2)

由上述結果可以看出，使用apply后，第1次輸出整個模型，此處為最頂層的。

 

第2-5次輸出：

2       nn.SpatialConvolutionMM(3 -> 64, 7x7, 2,2, 3,3)

3       nn.SpatialBatchNormalization

4       nn.ReLU

5       nn.SpatialMaxPooling(3x3, 2,2, 1,1)

為Inception之前的幾個層。

 

第6次為nn.Inception @ nn.DepthConcat，第7次為nn.DepthConcat。此處是第一個Inceptioin層。

第8次為Inception的第一個nn.Sequential，第9-14次為該層的具體層。此時已經到了第一個最底層。

第15次為Inception的第二個nn.Sequential，第16-21次為該層的具體層。此時已經到了第二個最底層。

第22次為Inception的第三個nn.Sequential，第23-26次為該層的具體層。此時已經到了第三個最底層。

第27次為Inception的第四個nn.Sequential，第28-30次為該層的具體層。此時已經到了第四個最底層。

至此，第一個Inception層通過深度優先的方式遍歷完畢。

 

第31次為nn.Inception @ nn.DepthConcat，第32次為nn.DepthConcat。此處是第二個Inceptioin層（注意，為了區分第一個Inception和第二個Inception層，這兩個層具體結構不完全一樣）。

第33次為Inception的第一個nn.Sequential，第34-37次為該層的具體層。此時已經到了第一個最底層。

第38次為Inception的第二個nn.Sequential，第39-42次為該層的具體層。此時已經到了第二個最底層。

第43次為Inception的第三個nn.Sequential。

第44次為第三個nn.Sequential的第一個小module（也是一個nn.Sequential）。第45-48依次遍歷此nn.Sequential。到了最底層后遍歷完畢。

第49-50為第三個nn.Sequential的最后兩層。

第51次為Inception的第四個nn.Sequential，第52-53次為該層的具體層。此時已經到了第四個最底層。

至此，第二個Inception層通過深度優先的方式遍歷完畢。

 

第54-57為最后的兩個層。

 

由上面可以看出，apply采用的是深度優先的方式進行遍歷。