caffe筆記之例程學習

學習notebook自帶例程Classification with HDF5 data時遇到了一些問題，認真把模型文件看了一遍。

模型定義中有一點比較容易被誤解，信號在有向圖中是自下而上流動的，並不是自上而下。

層的結構定義如下：

 name:層名稱 2 type:層類型 3 top:出口 4 bottom:入口 

Each layer type defines three critical computations: setup, forward, and backward.

• Setup: initialize the layer and its connections once at model initialization.
• Forward: given input from bottom compute the output and send to the top.
• Backward: given the gradient w.r.t. the top output compute the gradient w.r.t. to the input and send to the bottom. A layer with parameters computes the gradient w.r.t. to its parameters and stores it internally.

From: caffe_root/examples/hdf5_classification/train_val.prototxt

#From: caffe_root/examples/hdf5_classification/train_val.prototxt
#計算Logistic函數
name: "LogisticRegressionNet" 
#訓練數據層，put in HDF5Data
layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "examples/hdf5_classification/data/train.txt"
    batch_size: 10
  }
}
#測試數據層，put in HDF5Data
layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  hdf5_data_param {
    source: "examples/hdf5_classification/data/test.txt"
    batch_size: 10
  }
}
#全連接層
layer {
  name: "fc1"
  type: "InnerProduct"
  bottom: "data"
  top: "fc1"
  param {
    lr_mult: 1        # learning rate multiplier for the filters
    decay_mult: 1    # learning rate multiplier for the biases
  }
  param {
    lr_mult: 2        # weight decay multiplier for the filters
    decay_mult: 0    # weight decay multiplier for the biases
  }
  inner_product_param {
    num_output: 2
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
#loss函數層
layer {
  name: "loss"
  type: "SoftmaxWithLoss"    #計算loss函數的方法
  bottom: "fc1"
  bottom: "label"
  top: "loss"
}
#ACCURACY層
layer {
  name: "accuracy"
  type: "Accuracy"
  bottom: "fc1"
  bottom: "label"
  top: "accuracy"
  include {
    phase: TEST
  }
}

模型詳解（實在懶得翻譯了）：

1.HD5 Input:

HDF5 Input

    LayerType: HDF5_DATA
    Parameters
        Required
            source: the name of the file to read from
            batch_size

2.Inner Product(全連接層，計算向量內積):

Inner Product

    LayerType: INNER_PRODUCT
    CPU implementation: ./src/caffe/layers/inner_product_layer.cpp
    CUDA GPU implementation: ./src/caffe/layers/inner_product_layer.cu
    Parameters (InnerProductParameter inner_product_param)
        Required
            num_output (c_o): the number of filters
        Strongly recommended
            weight_filler [default type: 'constant' value: 0]
        Optional
            bias_filler [default type: 'constant' value: 0]
            bias_term [default true]: specifies whether to learn and apply a set of additive biases to the filter outputs
    Input
        n * c_i * h_i * w_i
    Output
        n * c_o * 1 * 1

    Sample

    layers {
      name: "fc8"
      type: INNER_PRODUCT
      blobs_lr: 1          # learning rate multiplier for the filters
      blobs_lr: 2          # learning rate multiplier for the biases
      weight_decay: 1      # weight decay multiplier for the filters
      weight_decay: 0      # weight decay multiplier for the biases
      inner_product_param {
        num_output: 1000
        weight_filler {
          type: "gaussian"
          std: 0.01
        }
        bias_filler {
          type: "constant"
          value: 0
        }
      }
      bottom: "fc7"
      top: "fc8"
    }

The INNER_PRODUCT layer (also usually referred to as the fully connected layer) treats the input as a simple vector and produces an output in the form of a single vector (with the blob’s height and width set to 1).

3.Loss(最基本的loss函數):

Loss

In Caffe, as in most of machine learning, learning is driven by a loss function (also known as an error, cost, or objective function). A loss function specifies the goal of learning by mapping parameter settings (i.e., the current network weights) to a scalar value specifying the “badness” of these parameter settings. Hence, the goal of learning is to find a setting of the weights that minimizes the loss function.

The loss in Caffe is computed by the Forward pass of the network. Each layer takes a set of input (bottom) blobs and produces a set of output (top) blobs. Some of these layers’ outputs may be used in the loss function. A typical choice of loss function for one-versus-all classification tasks is the SOFTMAX_LOSS function, used in a network definition as follows, for example:
/***********************************************************************/
layers {
  name: "loss"
  type: SOFTMAX_LOSS
  bottom: "pred"
  bottom: "label"
  top: "loss"
}

In a SOFTMAX_LOSS function, the top blob is a scalar (dimensions 1×1×1×1) which averages the loss (computed from predicted labels pred and actuals labels label) over the entire mini-batch.

4.Accuracy(和loss功能類似，但只是簡單做差，並不求和):

Accuracy and Top-k
 
ACCURACY scores the output as the accuracy of output with respect to target – it is not actually a loss and has no backward step.
 Activation / Neuron Layers
 
In general, activation / Neuron layers are element-wise operators, taking one bottom blob and producing one top blob of the same size. In the layers below, we will ignore the input and out sizes as they are identical:
 
    Input
        n * c * h * w
    Output
        n * c * h * w

嗯。這是所有例程中最簡單的一個模型。天要亮了，洗把臉去。

References:

1.caffe.berkeleyvision.org/tutorial/forward_backward.html

2.caffe.berkeleyvision.org/tutorial/layers.html

3.nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/hdf5_classification.ipynb