TensorFlow中權重的隨機初始化

　　一開始沒看懂stddev是什么參數，找了一下，在tensorflow/python/ops里有random_ops，其中是這么寫的：

def random_normal(shape, mean=0.0, stddev=1.0, dtype=types.float32,
                  seed=None, name=None):
  """Outputs random values from a normal distribution.

  Args:
    shape: A 1-D integer Tensor or Python array. The shape of the output tensor.
    mean: A 0-D Tensor or Python value of type `dtype`. The mean of the normal
      distribution.
    stddev: A 0-D Tensor or Python value of type `dtype`. The standard deviation
      of the normal distribution.
    dtype: The type of the output.
    seed: A Python integer. Used to create a random seed for the distribution.
      See
      [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)
      for behavior.
    name: A name for the operation (optional).

  Returns:
    A tensor of the specified shape filled with random normal values.
  """

　　也就是按照正態分布初始化權重，mean是正態分布的平均值，stddev是正態分布的標准差（standard deviation），seed是作為分布的random seed（隨機種子，我百度了一下，跟什么偽隨機數發生器還有關，就是產生隨機數的），在mnist/concolutional中seed賦值為66478，挺有意思，不知道是什么原理。

 

　　后面還有truncated_normal的定義：

def truncated_normal(shape, mean=0.0, stddev=1.0, dtype=types.float32,
                     seed=None, name=None):
  """Outputs random values from a truncated normal distribution.

  The generated values follow a normal distribution with specified mean and
  standard deviation, except that values whose magnitude is more than 2 standard
  deviations from the mean are dropped and re-picked.

  Args:
    shape: A 1-D integer Tensor or Python array. The shape of the output tensor.
    mean: A 0-D Tensor or Python value of type `dtype`. The mean of the
      truncated normal distribution.
    stddev: A 0-D Tensor or Python value of type `dtype`. The standard deviation
      of the truncated normal distribution.
    dtype: The type of the output.
    seed: A Python integer. Used to create a random seed for the distribution.
      See
      [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)
      for behavior.
    name: A name for the operation (optional).

  Returns:
    A tensor of the specified shape filled with random truncated normal values.
  """

　　截斷正態分布，以前都沒聽說過。

 

　　TensorFlow還提供了平均分布等。

 

參考：

1.https://tensorflow.googlesource.com/tensorflow/+/refs/heads/master/tensorflow/g3doc/api_docs/python

2.隨機種子：http://baike.baidu.com/link?url=bjDp9u9pkEg2oWOffMep1RW6B1U-0AX2FNmykTtCAa8L_7xzA0ygq6AyLBf8pv7XW8b4gwUKlvMWiCsp32Nu8K