Tensorflow學習筆記（2）：變量，常用類和一些操作

變量是用來存儲和更新參數的，也就是網絡中的W或b。變量會被放在內存中。當模型訓練結束后，他們需要被存在硬盤上，以便將來使用或分析模型。

一.變量

創建和初始化

　　當創建一個變量的時候，需要將一個Tensor作為初始值傳入構造函數Variable()。這個初始值可以是隨機值也可以是常量。Tensor的初始值需要指定shape，這個shape通常都是固定的，但是也可以通過一些高級方法重新調整。

　　只是創建了變量還是不夠的，需要在定義一個初始化的操作，並且在使用任何變量之前，運行初始化的操作。例：

  1 import tensorflow as tf
  2 
  3 #Create two variables
  4 weights = tf.Variable(tf.random_normal([20,10], stddev=0.35), name="weights")
  5 biases = tf.Variable(tf.zeros([10]), name="biases")
  6 
  7 #Add other net structure...
  8 #...
  9 
 10 #Add an op to initialize the variables.
 11 init = tf.initialize_all_variables()
 12 
 13 #Later, when launching the model
 14 with tf.Session() as sess:
 15     sess.run(init)
 16     print weights.eval()
 17     print biases.eval()

輸出：weights是[20,10]的矩陣，biases是[10]的向量。

注意：

　　tf.initialize_all_variables()是並行的初始化所有變量，所以如果需要用一個變量的值給另一個變量初始化的時候，一定要小心。雖然直接初始化不一定會出現問題，但是如果出現問題是很難找到這個原因的。

　　這時應該用如下方式進行初始化：

　

# Create a variable with a random value.
weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35),
                      name="weights")
# Create another variable with the same value as 'weights'.
w2 = tf.Variable(weights.initialized_value(), name="w2")
# Create another variable with twice the value of 'weights'
w_twice = tf.Variable(weights.initialized_value() * 0.2, name="w_twice")

另外還有自定義初始化，因為目前還沒用到，先挖個坑，以后再填，詳見Variables Documentation

變量可以被初始化成常量，或者是隨機數，這個跟初始化的策略有關，具體什么情況下使用什么方法進行初始化也挖個坑，以后學到了再講。

初始化成常量的方法：

tf.zeros(shape, dtype=tf.float32, name=None)　　

　　全部初始化為0

tf.zeros_like(tensor, dtype=None, name=None, optimize=True) 

　　創建一個shape和指定tensor相同的變量，但全部元素都為零。例如‘tensor’ =[[1,2,3], [4,5,6]]，那么tf.zeros_like(tensor) ==>[[0,0,0],[0,0,0]]

tf.ones(shape, dtype=tf.float32, name=None)

　　全部初始化為1

tf.ones_like()同上

tf.fill(dims, value, name=None)

　　對指定好的shape初始化為value值。tf.fill([2,3], 9) ==> [[9,9,9]　[9,9,9,]]　

tf.constant(value, dtype=None, shape=None, name='Const', verify_shape=False)

　　For example:

　　```python # Constant 1-D Tensor populated with value list. tensor = tf.constant([1, 2, 3, 4, 5, 6, 7]) => [1 2 3 4 5 6 7]

　　# Constant 2-D tensor populated with scalar value -1. tensor = tf.constant(-1.0, shape=[2, 3]) => [[-1. -1. -1.] [-1. -1. -1.]] ```

初始化成序列

tf.linspace()

tf.range()

初始化為隨機數

tf.random_normal(shape, mean=0.0, stddev=1.0. dtype=tf.float32,  seed=None, name=None)

　　按照正太分布初始化

tf.truncated_normal()

　　同上，但超過兩個標准差的數據被丟棄，並重新隨機選擇數據。即產生的數據都在正太分布的兩個標准差內。

　　

保存和加載

　　得到了訓練好的模型之后，我們需要將這個模型保存下來，之后可以再次讀取這個模型進行進一步的使用。最簡單的方法是使用托tf.train.Saver。下面是例子：

首先是保存變量：

import tensorflow as tf

#Create some variables.
v1 = tf.Variable([1,2,3,4,5], name="v1")
v2 = tf.Variable([11,12,13,14], name="v2")

#Add an op to initialize the variables
init = tf.initialize_all_variables()

#Add an op to save and restore all the variables
saver = tf.train.Saver()

with tf.Session() as sess:
    sess.run(init)
    #Do something with the model

    print v1.eval()
    print v2.eval()
    #save the variables to disk
    save_path = saver.save(sess,"model/model.ckpt")
    print "Model saved in file:", save_path

接下來是加載：

import tensorflow as tf

v3 = tf.Variable([0,0,0,0,0], name='v1')
v4 = tf.Variable([0,0,0,0], name='v2')

saver = tf.train.Saver()

with tf.Session() as sess:
    saver.restore(sess,"model/model.ckpt")
    print "Model restored."
    print v3.eval()
    print v4.eval()

在從文件中恢復變量時，不需要事先進行初始化。注意：在回復變量時，tf.Variable()里的name參數一定要與原來的變量名稱一致，這樣才能恢復到對應的變量。

變量的函數

__init__(initial_value=None, trainable=True, collections=None, validate_shape=True, caching_device=None, name=None, variable_def=None, dtype=None, expected_shape=None, import_scope=None)

Creates a new variable with value initial_value.

The new variable is added to the graph collections listed in collections, which defaults to [GraphKeys.GLOBAL_VARIABLES].

If trainable is True the variable is also added to the graph collectionGraphKeys.TRAINABLE_VARIABLES.

This constructor creates both a variable Op and an assign Op to set the variable to its initial value.

Args:

• initial_value: A Tensor, or Python object convertible to a Tensor, which is the initial value for the Variable. The initial value must have a shape specified unless validate_shape is set to False. Can also be a callable with no argument that returns the initial value when called. In that case, dtype must be specified. (Note that initializer functions from init_ops.py must first be bound to a shape before being used here.)
• trainable: If True, the default, also adds the variable to the graph collection GraphKeys.TRAINABLE_VARIABLES. This collection is used as the default list of variables to use by the Optimizer classes.應該是會在做最優化的時候使用得到
• collections: List of graph collections keys. The new variable is added to these collections. Defaults to [GraphKeys.GLOBAL_VARIABLES].
• validate_shape: If False, allows the variable to be initialized with a value of unknown shape. If True, the default, the shape ofinitial_value must be known.
• caching_device: Optional device string describing where the Variable should be cached for reading. Defaults to the Variable's device. If not None, caches on another device. Typical use is to cache on the device where the Ops using the Variable reside, to deduplicate copying through Switch and other conditional statements.
• name: Optional name for the variable. Defaults to 'Variable' and gets uniquified automatically.
• variable_def: VariableDef protocol buffer. If not None, recreates the Variable object with its contents. variable_def and the other arguments are mutually exclusive.
• dtype: If set, initial_value will be converted to the given type. If None, either the datatype will be kept (if initial_value is a Tensor), or convert_to_tensor will decide.
• expected_shape: A TensorShape. If set, initial_value is expected to have this shape.
• import_scope: Optional string. Name scope to add to the Variable.Only used when initializing from protocol buffer.

Raises:

• ValueError: If both variable_def and initial_value are specified.
• ValueError: If the initial value is not specified, or does not have a shape and validate_shape is True.

 

eval(session=None)

In a session, computes and returns the value of this variable.

This is not a graph construction method, it does not add ops to the graph.

This convenience method requires a session where the graph containing this variable has been launched. If no session is passed, the default session is used. See tf.Session for more information on launching a graph and on sessions.

 

v = tf.Variable([1, 2])
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    # Usage passing the session explicitly.
    print(v.eval(sess))
    # Usage with the default session.  The 'with' block
    # above makes 'sess' the default session.
    print(v.eval())

Args:

• session: The session to use to evaluate this variable. If none, the default session is used.

Returns:

A numpy ndarray with a copy of the value of this variable.

二.常見類

Tensor

　　Tensor類是最核心的數據結構。Tensor是一個處理操作輸出的符號，它並不保存操作輸出的值，但是提供了在Session中計算這些值的方法。

This class has two primary purposes:

1. A Tensor can be passed as an input to another Operation. This builds a dataflow connection between operations, which enables TensorFlow to execute an entire Graph that represents a large, multi-step computation.

2. After the graph has been launched in a session, the value of the Tensor can be computed by passing it toSession.run(). t.eval() is a shortcut for calling tf.get_default_session().run(t).

 Operation

Operation是tensorflow中的節點，使用Tensor作為輸入，並輸出一個Tensor。其實就是運算操作，例如tf.matmul(a,b），就是a×b。當圖在session中啟動之后，operation拒可以通過tf.Session.run()這種操作執行，或者op.run()。