（原）tensorflow中函數執行完畢，顯存不自動釋放

轉載請注明出處：

http://www.cnblogs.com/darkknightzh/p/7608916.html

參考網址：

https://stackoverflow.com/questions/39758094/clearing-tensorflow-gpu-memory-after-model-execution

https://github.com/tensorflow/tensorflow/issues/1727#issuecomment-285815312s

tensorflow中，在一個函數內配置完GPU，tf分配了顯存，等函數執行完，顯存不會釋放（貌似torch7中也一樣。。。）。第二個參考網址指出：

As for the original problem, currently the Allocator in the GPUDevice belongs to the ProcessState, which is essentially a global singleton. The first session using GPU initializes it, and frees itself when the process shuts down. Even if a second session chooses a different GPUOptions, it would not take effect.

第一個session對GPU初始化后，即便釋放了顯存，第二個sess使用不同的GPU選項來初始化GPU，也不會起效。

第一個網址Oli Blum指出，use processes and shut them down after the computation才能釋放顯存。具體代碼如下（可以參考第一個網址）：

import tensorflow as tf
import multiprocessing
import numpy as np

def run_tensorflow():

    n_input = 10000
    n_classes = 1000

    # Create model
    def multilayer_perceptron(x, weight):
        # Hidden layer with RELU activation
        layer_1 = tf.matmul(x, weight)
        return layer_1

    # Store layers weight & bias
    weights = tf.Variable(tf.random_normal([n_input, n_classes]))


    x = tf.placeholder("float", [None, n_input])
    y = tf.placeholder("float", [None, n_classes])
    pred = multilayer_perceptron(x, weights)

    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
    optimizer = tf.train.AdamOptimizer(learning_rate=0.001).minimize(cost)

    init = tf.global_variables_initializer()

    with tf.Session() as sess:
        sess.run(init)

        for i in range(100):
            batch_x = np.random.rand(10, 10000)
            batch_y = np.random.rand(10, 1000)
            sess.run([optimizer, cost], feed_dict={x: batch_x, y: batch_y})

    print "finished doing stuff with tensorflow!"


if __name__ == "__main__":

    # option 1: execute code with extra process
    p = multiprocessing.Process(target=run_tensorflow)
    p.start()
    p.join()

    # wait until user presses enter key
    raw_input()

    # option 2: just execute the function
    run_tensorflow()

    # wait until user presses enter key
    raw_input()

使用multiprocessing.Process運行run_tensorflow后，顯存會自動釋放，但是如果直接執行run_tensorflow，顯存不會自動釋放。當然，該函數計算量較小，如果顯卡太好，可能看不到運行multiprocessing.Process后，顯存分配、計算並釋放的過程，感覺就像沒有運行一樣。。。