如果拋開Keras，TensorLayer，tfLearn，tensroflow 能否寫出簡介的代碼？ 可以！slim這個模塊是在16年新推出的，其主要目的是來做所謂的“代碼瘦身”
一.簡介
slim被放在tensorflow.contrib這個庫下面，導入的方法如下：
import tensorflow.contrib.slim as slim
眾所周知 tensorflow.contrib這個庫，tensorflow官方對它的描述是：此目錄中的任何代碼未經官方支持，可能會隨時更改或刪除。每個目錄下都有指定的所有者。它旨在包含額外功能和貢獻，最終會合並到核心TensorFlow中，但其接口可能仍然會發生變化，或者需要進行一些測試，看是否可以獲得更廣泛的接受。所以slim依然不屬於原生tensorflow。
slim是一個使構建，訓練，評估神經網絡變得簡單的庫。它可以消除原生tensorflow里面很多重復的模板性的代碼，讓代碼更緊湊，更具備可讀性。另外slim提供了很多計算機視覺方面的著名模型（VGG, AlexNet等），我們不僅可以直接使用，甚至能以各種方式進行擴展。
slim的子模塊及功能介紹：
arg_scope: provides a new scope named arg_scope that allows a user to define default arguments for specific operations within that scope.
除了基本的namescope，variabelscope外，又加了argscope，它是用來控制每一層的默認超參數的。（后面會詳細說）
data: contains TF-slim's dataset definition, data providers, parallel_reader, and decoding utilities.
貌似slim里面還有一套自己的數據定義，這個跳過，我們用的不多。
evaluation: contains routines for evaluating models.
評估模型的一些方法，用的也不多
layers: contains high level layers for building models using tensorflow.
這個比較重要，slim的核心和精髓，一些復雜層的定義
learning: contains routines for training models.
一些訓練規則
losses: contains commonly used loss functions.
一些loss
metrics: contains popular evaluation metrics.
評估模型的度量標准
nets: contains popular network definitions such as VGG and AlexNet models.
包含一些經典網絡，VGG等，用的也比較多
queues: provides a context manager for easily and safely starting and closing QueueRunners.
文本隊列管理，比較有用。
regularizers: contains weight regularizers.
包含一些正則規則
variables: provides convenience wrappers for variable creation and manipulation.
slim管理變量的機制
二.slim定義模型
slim中定義一個變量的示例：
# Model Variables
weights = slim.model_variable(
'weights'
,
shape=[
10
,
10
,
3
,
3
],
initializer=tf.truncated_normal_initializer(stddev=
0.1
),
regularizer=slim.l2_regularizer(
0.05
),
device=
'/CPU:0'
)
model_variables = slim.get_model_variables()
# Regular variables
my_var = slim.variable(
'my_var'
,
shape=[
20
,
1
],
initializer=tf.zeros_initializer())
regular_variables_and_model_variables = slim.get_variables()
slim中實現一個層：
首先讓我們看看tensorflow怎么實現一個層，例如卷積層：
input = ...
with tf.name_scope(
'conv1_1'
) as scope:
kernel = tf.Variable(tf.truncated_normal([
3
,
3
,
64
,
128
], dtype=tf.float32,
stddev=1e-
1
), name=
'weights'
conv = tf.nn.conv2d(input, kernel, [
1
,
1
,
1
,
1
], padding=
'SAME'
)
biases = tf.Variable(tf.constant(
0.0
, shape=[
128
], dtype=tf.float32),
trainable=True, name=
'biases'
)
bias = tf.nn.bias_add(conv, biases)
conv1 = tf.nn.relu(bias, name=scope)
input = ...
net = slim.conv2d(input,
128
, [
3
,
3
], scope=
'conv1_1'
)
net = ...
net = slim.conv2d(net,
256
, [
3
,
3
], scope=
'conv3_1'
)
net = slim.conv2d(net,
256
, [
3
,
3
], scope=
'conv3_2'
)
net = slim.conv2d(net,
256
, [
3
,
3
], scope=
'conv3_3'
)
net = slim.max_pool2d(net, [
2
,
2
], scope=
'pool2'
)
net = slim.repeat(net,
3
, slim.conv2d,
256
, [
3
,
3
], scope=
'conv3'
)
net = slim.max_pool2d(net, [
2
,
2
], scope=
'pool2'
)
假設定義三層FC：
# Verbose way:
x = slim.fully_connected(x,
32
, scope=
'fc/fc_1'
)
x = slim.fully_connected(x,
64
, scope=
'fc/fc_2'
)
x = slim.fully_connected(x,
128
, scope=
'fc/fc_3'
)
32
,
64
,
128
], scope=
'fc'
)
# 普通方法:
x = slim.conv2d(x,
32
, [
3
,
3
], scope=
'core/core_1'
)
x = slim.conv2d(x,
32
, [
1
,
1
], scope=
'core/core_2'
)
x = slim.conv2d(x,
64
, [
3
,
3
], scope=
'core/core_3'
)
x = slim.conv2d(x,
64
, [
1
,
1
], scope=
'core/core_4'
)
# 簡便方法:
slim.stack(x, slim.conv2d, [(
32
, [
3
,
3
]), (
32
, [
1
,
1
]), (
64
, [
3
,
3
]), (
64
, [
1
,
1
])], scope=
'core'
)
slim中的argscope：
如果你的網絡有大量相同的參數，如下：
net = slim.conv2d(inputs, 64, [11, 11], 4, padding='SAME',
weights_initializer=tf.truncated_normal_initializer(stddev=
0.01
),
weights_regularizer=slim.l2_regularizer(
0.0005
), scope=
'conv1'
)
net = slim.conv2d(net,
128
, [
11
,
11
], padding=
'VALID'
,
weights_initializer=tf.truncated_normal_initializer(stddev=
0.01
),
weights_regularizer=slim.l2_regularizer(
0.0005
), scope=
'conv2'
)
net = slim.conv2d(net,
256
, [
11
,
11
], padding=
'SAME'
,
weights_initializer=tf.truncated_normal_initializer(stddev=
0.01
),
weights_regularizer=slim.l2_regularizer(
0.0005
), scope=
'conv3'
)
with slim.arg_scope([slim.conv2d], padding=
'SAME'
,
weights_initializer=tf.truncated_normal_initializer(stddev=
0.01
)
weights_regularizer=slim.l2_regularizer(
0.0005
)):
net = slim.conv2d(inputs,
64
, [
11
,
11
], scope=
'conv1'
)
net = slim.conv2d(net,
128
, [
11
,
11
], padding=
'VALID'
, scope=
'conv2'
)
net = slim.conv2d(net,
256
, [
11
,
11
], scope=
'conv3'
)
with slim.arg_scope([slim.conv2d, slim.fully_connected],
activation_fn=tf.nn.relu,
weights_initializer=tf.truncated_normal_initializer(stddev=
0.01
),
weights_regularizer=slim.l2_regularizer(
0.0005
)):
with slim.arg_scope([slim.conv2d], stride=
1
, padding=
'SAME'
):
net = slim.conv2d(inputs,
64
, [
11
,
11
],
4
, padding=
'VALID'
, scope=
'conv1'
)
net = slim.conv2d(net,
256
, [
5
,
5
],
weights_initializer=tf.truncated_normal_initializer(stddev=
0.03
),
scope=
'conv2'
)
net = slim.fully_connected(net,
1000
, activation_fn=None, scope=
'fc'
)
VGG：
def vgg16(inputs):
with slim.arg_scope([slim.conv2d, slim.fully_connected],
activation_fn=tf.nn.relu,
weights_initializer=tf.truncated_normal_initializer(
0.0
,
0.01
),
weights_regularizer=slim.l2_regularizer(
0.0005
)):
net = slim.repeat(inputs,
2
, slim.conv2d,
64
, [
3
,
3
], scope=
'conv1'
)
net = slim.max_pool2d(net, [
2
,
2
], scope=
'pool1'
)
net = slim.repeat(net,
2
, slim.conv2d,
128
, [
3
,
3
], scope=
'conv2'
)
net = slim.max_pool2d(net, [
2
,
2
], scope=
'pool2'
)
net = slim.repeat(net,
3
, slim.conv2d,
256
, [
3
,
3
], scope=
'conv3'
)
net = slim.max_pool2d(net, [
2
,
2
], scope=
'pool3'
)
net = slim.repeat(net,
3
, slim.conv2d,
512
, [
3
,
3
], scope=
'conv4'
)
net = slim.max_pool2d(net, [
2
,
2
], scope=
'pool4'
)
net = slim.repeat(net,
3
, slim.conv2d,
512
, [
3
,
3
], scope=
'conv5'
)
net = slim.max_pool2d(net, [
2
,
2
], scope=
'pool5'
)
net = slim.fully_connected(net,
4096
, scope=
'fc6'
)
net = slim.dropout(net,
0.5
, scope=
'dropout6'
)
net = slim.fully_connected(net,
4096
, scope=
'fc7'
)
net = slim.dropout(net,
0.5
, scope=
'dropout7'
)
net = slim.fully_connected(net,
1000
, activation_fn=None, scope=
'fc8'
)
return
net
三.訓練模型
import
tensorflow as tf
vgg = tf.contrib.slim.nets.vgg
# Load the images and labels.
images, labels = ...
# Create the model.
predictions, _ = vgg.vgg_16(images)
# Define the loss functions and get the total loss.
loss = slim.losses.softmax_cross_entropy(predictions, labels)
關於loss,要說一下定義自己的loss的方法，以及注意不要忘記加入到slim中讓slim看到你的loss。
還有正則項也是需要手動添加進loss當中的，不然最后計算的時候就不優化正則目標了。
# Load the images and labels.
images, scene_labels, depth_labels, pose_labels = ...
# Create the model.
scene_predictions, depth_predictions, pose_predictions = CreateMultiTaskModel(images)
# Define the loss functions and get the total loss.
classification_loss = slim.losses.softmax_cross_entropy(scene_predictions, scene_labels)
sum_of_squares_loss = slim.losses.sum_of_squares(depth_predictions, depth_labels)
pose_loss = MyCustomLossFunction(pose_predictions, pose_labels)
slim.losses.add_loss(pose_loss) # Letting TF-Slim know about the additional loss.
# The following two ways to compute the total loss are equivalent:
regularization_loss = tf.add_n(slim.losses.get_regularization_losses())
total_loss1 = classification_loss + sum_of_squares_loss + pose_loss + regularization_loss
# (Regularization Loss is included in the total loss by
default
).
total_loss2 = slim.losses.get_total_loss()
四.讀取保存模型變量
通過以下功能我們可以載入模型的部分變量：
# Create some variables.
v1 = slim.variable(name=
"v1"
, ...)
v2 = slim.variable(name=
"nested/v2"
, ...)
...
# Get list of variables to restore (which contains only
'v2'
).
variables_to_restore = slim.get_variables_by_name("v2")
# Create the saver which will be used to restore the variables.
restorer = tf.train.Saver(variables_to_restore)
with tf.Session() as sess:
# Restore variables from disk.
restorer.restore(sess,
"/tmp/model.ckpt"
)
print(
"Model restored."
)
假設我們定義的網絡變量是conv1/weights，而從VGG加載的變量名為vgg16/conv1/weights，正常load肯定會報錯（找不到變量名），但是可以這樣：
def name_in_checkpoint(var):
return
'vgg16/'
+ var.op.name
variables_to_restore = slim.get_model_variables()
variables_to_restore = {name_in_checkpoint(var):var for var in variables_to_restore}
restorer = tf.train.Saver(variables_to_restore)
with tf.Session() as sess:
# Restore variables from disk.
restorer.restore(sess,
"/tmp/model.ckpt"
)
通過這種方式我們可以加載不同變量名的變量