tensorflow代碼升級

目錄：

一、自動轉換腳本

二、高級別behavioral changes

三、讓代碼2.0本土化

四、改變模型

五、訓練

This doc for users of low level TensorFlow APIs. If you are using the high level APIs (tf.keras) there may be little or no action you need to take to make your code fully TensorFlow 2.0 compatible:

該文檔是為低級的Tensorflow API。如果你使用的上高級API（tf.keras），可能只有一點或者根本不用做任何改變。

• Check your optimizer's default learning rate.
• Note that the "name" that metrics are logged to may have changed.

It is still possible to run 1.X code, unmodified (except for contrib), in TensorFlow 2.0:

當然，在tensorflow2.0你可以直接使用tensorflow1.0的代碼而不做任何改變（除了contrib）：

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

 

However, this does not let you take advantage of many of the improvements made in TensorFlow 2.0. This guide will help you upgrade your code, making it simpler, more performant, and easier to maintain.

 但是，這並不能很好的兼容tensorflow2.0的運行環境，例如GPU版本？

 

一、自動轉換腳本 

The first step, before attempting to implement the changes described in this doc, is to try running the upgrade script.

第一步，嘗試一下自動轉換腳本。

This will do an initial pass at upgrading your code to TensorFlow 2.0. But it can't make your code idiomatic to 2.0. Your code may still make use of tf.compat.v1 endpoints to access placeholders, sessions, collections, and other 1.x-style functionality.

這僅僅是初始化的步驟。並不符合2.0的習慣。你的代碼可能依然是在使用tf.compat.v1來調用placeholders,sessions,collections,和其他1.x風格的函數功能。

二、高級別behavioral changes

If your code works in TensorFlow 2.0 using tf.compat.v1.disable_v2_behavior(), there are still global behavioral changes you may need to address. The major changes are:

如果你的代碼是使用tf.compat.v1.disable_v2_behavior()在tensorflow2.0中工作，那么依然有一些全局的行為需要改變。這些主要改變是：

• Eager execution, v1.enable_eager_execution() : Any code that implicitly uses a tf.Graph will fail. Be sure to wrap this code in a with tf.Graph().as_default() context.

          飢餓執行，v1.enable_eager_execution()：任何代碼使用tf.Graph都會失敗。確保使用tf.Graph().as_default()上下文將代碼包裹。

• Resource variables, v1.enable_resource_variables(): Some code may depends on non-deterministic behaviors enabled by TF reference variables. Resource variables are locked while being written to, and so provide more intuitive consistency guarantees.

          資源變量，v1.enable_resource_variables()：有些代碼可能依賴於TF引用變量啟用的非確定性行為。資源變量在寫入時被鎖定，因此提供了更直接的一致性保證。

             This may change behavior in edge cases.

             This may create extra copies and can have higher memory usage.

             This can be disabled by passing use_resource=False to the tf.Variable constructor.

• Tensor shapes, v1.enable_v2_tensorshape(): TF 2.0 simplifies the behavior of tensor shapes. Instead of t.shape[0].value you can say t.shape[0]. These changes should be small, and it makes sense to fix them right away. See TensorShape for examples.

          張量的形狀，v1.enable_v2_tensorshape()：TF2.0簡化了張量形狀的行為。替代t.shape[0].value，你可以說t.shape[0]。這些改變應該很小，而且修正它們是好懂的。

• Control flow, v1.enable_control_flow_v2(): The TF 2.0 control flow implementation has been simplified, and so produces different graph representations. Please file bugs for any issues.

          控制流，v1.enable_control_flow_v2()：TF2.0控制流已經簡化了，並且因此產生了不同的圖形表示。

三、讓代碼2.0本土化

This guide will walk through several examples of converting TensorFlow 1.x code to TensorFlow 2.0. These changes will let your code take advantage of performance optimizations and simplified API calls.

 該指導將通過幾個例子，展示從tensorflow1.x到tensorflow2.0的轉換。這些改變將會使你的代碼充分利用性能優化和簡化的API調用。

In each case, the pattern is:

在每種情況下，模式是：

1. Replace v1.Session.run calls

    取代v1.Session

Every v1.Session.run call should be replaced by a Python function.

每一個v1.Session.run命令都應該更換為Python函數。

  The feed_dict and v1.placeholders become function arguments.

  feed_dict和v1.placeholders變成函數參數。
  The fetches become the function's return value.

  fetches變成函數的返回值。
  During conversion eager execution allows easy debugging with standard Python tools like pdb.

  飢餓模式允許輕松的調試，例如pdb。

 After that add a tf.function decorator to make it run efficiently in graph. See the Autograph Guide for more on how this works.

在那之后添加一個tf.function修飾詞，使它在graph里有效地運行。查看Autograph Guide查看更多。

 Note that:注意：

Unlike v1.Session.run a tf.function has a fixed return signature, and always returns all outputs. If this causes performance problems, create two separate functions.

不像v1.Session.run，tf.function有固定的返回標志，而且總是返回所有的輸出。如果這引起表現問題，創建兩個分開的函數。

There is no need for a tf.control_dependencies or similar operations: A tf.function behaves as if it were run in the order written. tf.Variable assignments and tf.asserts, for example, are executed automatically.

 對tf.control_dependencies或類似的操作沒有必要：tf.function表現得就像它運行在寫定的順序里。例如，tf.Variable賦值和tf.asserts，自動執行。

2. Use Python objects to track variables and losses

     使用Python對象跟蹤變量和損失

All name-based variable tracking is strongly discouraged in TF 2.0. Use Python objects to to track variables.

在tf2.0中，強烈不推薦所有基於名稱的變量跟蹤。使用Python對象跟蹤變量。

Use tf.Variable instead of v1.get_variable.

使用tf.Variable而不是v1.get_variable。

Every v1.variable_scope should be converted to a Python object. Typically this will be one of:

每個v1.variable_scope都應轉換為Python對象。通常這將是：

• tf.keras.layers.Layer
• tf.keras.Model
• tf.Module

If you need to aggregate lists of variables (like tf.Graph.get_collection(tf.GraphKeys.VARIABLES)), use the .variables and .trainable_variables attributes of the Layer and Model objects.

如果需要聚合變量列表（比如tf.Graph.get_collection(tf.GraphKeys.VARIABLES))，使用層和模型對象的.variables和.trainable_variables屬性。

These Layer and Model classes implement several other properties that remove the need for global collections. Their .losses property can be a replacement for using the tf.GraphKeys.LOSSES collection.

這些Layer和Model類實現了幾個其它的屬性，移除了全局collection的需要。它們的.losses屬性可以取代tf.GraphKeys.LOSSES集合的使用需要。

有關詳細信息，請參見keras指南。

3. Upgrade your training loops

   升級你的訓練環

Use the highest level API that works for your use case. Prefer tf.keras.Model.fit over building your own training loops.

使用適用於您的用例的最高級別的API。更喜歡tf.keras.Model.fit建立自己的訓練循環。

These high level functions manage a lot of the low-level details that might be easy to miss if you write your own training loop. For example, they automatically collect the regularization losses, and set the training=True argument when calling the model.

這些高級函數管理許多低級的細節，如果您編寫自己的訓練循環，這些細節可能很容易遺漏。例如，它們自動收集正則化損失，並在調用模型時設置training=True參數。

4. Upgrade your data input pipelines

    升級數據輸入管道

Use tf.data datasets for data input. These objects are efficient, expressive, and integrate well with tensorflow.

使用tf.data用於數據輸入的數據集。這些對象效率高、表現力強，並且與tensorflow很好地集成在一起。

They can be passed directly to the tf.keras.Model.fit method.

它們可以直接傳遞給tf.keras.Model.fit方法。

 model.fit(dataset, epochs=5)

它們可以直接在標准Python代碼上迭代：

for example_batch, label_batch in dataset:
    break

5. Migrate off compat.v1 symbols

    遷移除compat.v1符號

The tf.compat.v1 module contains the complete TensorFlow 1.x API, with its original semantics.

這個tf.compat.v1模塊包含完整的TensorFlow 1.x API及其原始語義。

The TF2 upgrade script will convert symbols to their 2.0 equivalents if such a conversion is safe, i.e., if it can determine that the behavior of the 2.0 version is exactly equivalent (for instance, it will rename v1.arg_max to tf.argmax, since those are the same function).

如果這種轉換是安全的，即如果它能夠確定2.0版本的行為完全等效（例如，它將把v1.arg_max重命名為tf.argmax公司，因為它們是相同的函數）。

After the upgrade script is done with a piece of code, it is likely there are many mentions of compat.v1. It is worth going through the code and converting these manually to the 2.0 equivalent (it should be mentioned in the log if there is one).

在用一段代碼完成升級腳本后，很可能會有很多次提到compat.v1。仔細檢查代碼並將其手動轉換為2.0等效版本是值得的（如果有，應該在日志中提到）。

四、改變模型

1.Setup

   設置

import tensorflow as tf
import tensorflow_datasets as tfds

 2.Low-level variables & operator execution

Examples of low-level API use include:

低級API的使用示例包括：

• using variable scopes to control reuse
• creating variables with v1.get_variable.
• accessing collections explicitly

• accessing collections implicitly with methods like :

  • v1.global_variables
  • v1.losses.get_regularization_loss

• using v1.placeholder to set up graph inputs

• executing graphs with Session.run

• initializing variables manually

轉換之前：

這里是Tensorflow 1.x里可能看上去的：

in_a = tf.placeholder(dtype=tf.float32, shape=(2))
in_b = tf.placeholder(dtype=tf.float32, shape=(2))

def forward(x):
  with tf.variable_scope("matmul", reuse=tf.AUTO_REUSE):
    W = tf.get_variable("W", initializer=tf.ones(shape=(2,2)),
                        regularizer=tf.contrib.layers.l2_regularizer(0.04))
    b = tf.get_variable("b", initializer=tf.zeros(shape=(2)))
    return W * x + b

out_a = forward(in_a)
out_b = forward(in_b)

reg_loss=tf.losses.get_regularization_loss(scope="matmul")

with tf.Session() as sess:
  sess.run(tf.global_variables_initializer())
  outs = sess.run([out_a, out_b, reg_loss],
                feed_dict={in_a: [1, 0], in_b: [0, 1]})

轉換之后： 

轉換后的代碼：

• The variables are local Python objects.
• The forward function still defines the calculation.
• The Session.run call is replaced with a call to forward
• The optional tf.function decorator can be added for performance.
• The regularizations are calculated manually, without referring to any global collection.
• No sessions or placeholders.

W = tf.Variable(tf.ones(shape=(2,2)), name="W")
b = tf.Variable(tf.zeros(shape=(2)), name="b")

@tf.function
def forward(x):
  return W * x + b

out_a = forward([1,0])
print(out_a)

 

tf.Tensor(
[[1. 0.]
 [1. 0.]], shape=(2, 2), dtype=float32)

out_b = forward([0,1])

regularizer = tf.keras.regularizers.l2(0.04)
reg_loss=regularizer(W)

3. Models based on tf.layers

    基於tf.layers的模型

The v1.layers module is used to contain layer-functions that relied on v1.variable_scope to define and reuse variables.

v1.layers模塊用於包含依賴v1.variable_作用域定義和重用變量的層函數。

轉換之前

def model(x, training, scope='model'):
  with tf.variable_scope(scope, reuse=tf.AUTO_REUSE):
    x = tf.layers.conv2d(x, 32, 3, activation=tf.nn.relu,
          kernel_regularizer=tf.contrib.layers.l2_regularizer(0.04))
    x = tf.layers.max_pooling2d(x, (2, 2), 1)
    x = tf.layers.flatten(x)
    x = tf.layers.dropout(x, 0.1, training=training)
    x = tf.layers.dense(x, 64, activation=tf.nn.relu)
    x = tf.layers.batch_normalization(x, training=training)
    x = tf.layers.dense(x, 10)
    return x

train_out = model(train_data, training=True)
test_out = model(test_data, training=False)

轉換之后

• The simple stack of layers fits neatly into tf.keras.Sequential. (For more complex models see custom layers and models, and the functional API.)
• The model tracks the variables, and regularization losses.
• The conversion was one-to-one because there is a direct mapping from v1.layers to tf.keras.layers.

大多數參數沒有改變。但請注意不同之處：

• The training argument is passed to each layer by the model when it runs.
• The first argument to the original model function (the input x) is gone. This is because object layers separate building the model from calling the model.

還請注意：

• If you were using regularizers of initializers from tf.contrib, these have more argument changes than others.
• The code no longer writes to collections, so functions like v1.losses.get_regularization_loss will no longer return these values, potentially breaking your training loops.

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu',
                           kernel_regularizer=tf.keras.regularizers.l2(0.04),
                           input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10)
])

train_data = tf.ones(shape=(1, 28, 28, 1))
test_data = tf.ones(shape=(1, 28, 28, 1))

train_out = model(train_data, training=True)
print(train_out)

tf.Tensor([[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]], shape=(1, 10), dtype=float32)

test_out = model(test_data, training=False)
print(test_out)

tf.Tensor(
[[-0.11456077 -0.3126101   0.13154565 -0.50197905 -0.02416557  0.36460522
  -0.24887308 -0.37784547  0.05524942  0.01696768]], shape=(1, 10), dtype=float32)

# Here are all the trainable variables.
len(model.trainable_variables)

8

# Here is the regularization loss.
model.losses

[<tf.Tensor: shape=(), dtype=float32, numpy=0.077528305>]

4. Mixed variables & v1.layers

   混合變量和v1.layers

Existing code often mixes lower-level TF 1.x variables and operations with higher-level v1.layers.

現有代碼通常將較低級別的tf1.x變量和操作與較高級別的v1.layers混合在一起。

轉換之前：

def model(x, training, scope='model'):
  with tf.variable_scope(scope, reuse=tf.AUTO_REUSE):
    W = tf.get_variable(
      "W", dtype=tf.float32,
      initializer=tf.ones(shape=x.shape),
      regularizer=tf.contrib.layers.l2_regularizer(0.04),
      trainable=True)
    if training:
      x = x + W
    else:
      x = x + W * 0.5
    x = tf.layers.conv2d(x, 32, 3, activation=tf.nn.relu)
    x = tf.layers.max_pooling2d(x, (2, 2), 1)
    x = tf.layers.flatten(x)
    return x

train_out = model(train_data, training=True)
test_out = model(test_data, training=False)

轉換之后：

To convert this code, follow the pattern of mapping layers to layers as in the previous example.

要轉換此代碼，請按照將層映射到上一個示例中的層的模式。

一般模式是：

• Collect layer parameters in __init__.
• Build the variables in build.
• Execute the calculations in call, and return the result.

The v1.variable_scope is essentially a layer of its own. So rewrite it as a tf.keras.layers.Layer. See the guide for details.

v1.variable_作用域本質上是它自己的一個層。作為重寫tf.keras.layer.Layer。有關詳細信息，請參閱指南。

# Create a custom layer for part of the model
class CustomLayer(tf.keras.layers.Layer):
  def __init__(self, *args, **kwargs):
    super(CustomLayer, self).__init__(*args, **kwargs)

  def build(self, input_shape):
    self.w = self.add_weight(
        shape=input_shape[1:],
        dtype=tf.float32,
        initializer=tf.keras.initializers.ones(),
        regularizer=tf.keras.regularizers.l2(0.02),
        trainable=True)

  # Call method will sometimes get used in graph mode,
  # training will get turned into a tensor
  @tf.function
  def call(self, inputs, training=None):
    if training:
      return inputs + self.w
    else:
      return inputs + self.w * 0.5

custom_layer = CustomLayer()
print(custom_layer([1]).numpy())
print(custom_layer([1], training=True).numpy())

[1.5]
[2.]

train_data = tf.ones(shape=(1, 28, 28, 1))
test_data = tf.ones(shape=(1, 28, 28, 1))

# Build the model including the custom layer
model = tf.keras.Sequential([
    CustomLayer(input_shape=(28, 28, 1)),
    tf.keras.layers.Conv2D(32, 3, activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
])

train_out = model(train_data, training=True)
test_out = model(test_data, training=False)

注意事項：

• Subclassed Keras models & layers need to run in both v1 graphs (no automatic control dependencies) and in eager mode

  • Wrap the call() in a tf.function() to get autograph and automatic control dependencies

• Don't forget to accept a training argument to call.

  • Sometimes it is a tf.Tensor
  • Sometimes it is a Python boolean.

• Create model variables in constructor or Model.build using self.add_weight().

  • In Model.build you have access to the input shape, so can create weights with matching shape.
  • Using tf.keras.layers.Layer.add_weight allows Keras to track variables and regularization losses.

• Don't keep tf.Tensors in your objects.

  • They might get created either in a tf.function or in the eager context, and these tensors behave differently.
  • Use tf.Variables for state, they are always usable from both contexts
  • tf.Tensors are only for intermediate values.

 5. A note on Slim & contrib.layers

    關於Slim&貢獻層

A large amount of older TensorFlow 1.x code uses the Slim library, which was packaged with TensorFlow 1.x as tf.contrib.layers. As a contrib module, this is no longer available in TensorFlow 2.0, even in tf.compat.v1. Converting code using Slim to TF 2.0 is more involved than converting repositories that use v1.layers. In fact, it may make sense to convert your Slim code to v1.layers first, then convert to Keras.

大量較舊的TensorFlow1.x代碼使用Slim庫，該庫與TensorFlow1.x打包為tf.contrib.layers公司. 作為一個contrib模塊，這在TensorFlow 2.0中不再可用，即使在tf.compat公司.1版。使用Slim將代碼轉換為tf2.0比轉換使用v1.layers的存儲庫要復雜得多。事實上，首先將Slim代碼轉換為v1.layers，然后再轉換為Keras可能是有意義的。

• Remove arg_scopes, all args need to be explicit
• If you use them, split normalizer_fn and activation_fn into their own layers
• Separable conv layers map to one or more different Keras layers (depthwise, pointwise, and separable Keras layers)
• Slim and v1.layers have different arg names & default values
• Some args have different scales
• If you use Slim pre-trained models, try out Keras's pre-traimed models from tf.keras.applications or TF Hub's TF2 SavedModels exported from the original Slim code.

Some tf.contrib layers might not have been moved to core TensorFlow but have instead been moved to the TF add-ons package.

一些出資人層可能沒有被移到核心TensorFlow，而是被移到了TF add-ons包中。

五、訓練

 There are many ways to feed data to a tf.keras model. They will accept Python generators and Numpy arrays as input.

有很多種方法將數據喂給tf.keras模型。它們將會接受Python生成器和Numpy數組作為輸入。

The recommended way to feed data to a model is to use the tf.data package, which contains a collection of high performance classes for manipulating data.

向模型喂數據的推薦方法是使用tf.data包。它包含用於操作數據的高性能類的集合。

If you are still using tf.queue, these are now only supported as data-structures, not as input pipelines.

如果你還在用tf.隊列，這些現在只支持作為數據結構，而不是作為輸入管道。

1.Using Datasets

   使用數據集

The TensorFlow Datasets package (tfds) contains utilities for loading predefined datasets as tf.data.Dataset objects.

TensorFlow Datasets包(tfds)包含加載預定義數據集作為tf.data.Dataset物體的公用程序。

For this example, load the MNISTdataset, using tfds:

對於本例，使用tfds加載MNISTdataset：

datasets, info = tfds.load(name='mnist', with_info=True, as_supervised=True)
mnist_train, mnist_test = datasets['train'], datasets['test']

Downloading and preparing dataset mnist/3.0.1 (download: 11.06 MiB, generated: 21.00 MiB, total: 32.06 MiB) to /home/kbuilder/tensorflow_datasets/mnist/3.0.1...

Warning:absl:Dataset mnist is hosted on GCS. It will automatically be downloaded to your
local data directory. If you'd instead prefer to read directly from our public
GCS bucket (recommended if you're running on GCP), you can instead pass
`try_gcs=True` to `tfds.load` or set `data_dir=gs://tfds-data/datasets`.


Dataset mnist downloaded and prepared to /home/kbuilder/tensorflow_datasets/mnist/3.0.1. Subsequent calls will reuse this data.

Then prepare the data for training:

然后准備訓練數據：

• Re-scale each image.
• Shuffle the order of the examples.
• Collect batches of images and labels.

BUFFER_SIZE = 10 # Use a much larger value for real code.
BATCH_SIZE = 64
NUM_EPOCHS = 5


def scale(image, label):
  image = tf.cast(image, tf.float32)
  image /= 255

  return image, label

To keep the example short, trim the dataset to only return 5 batches:

要使示例保持簡短，修剪數據集以僅返回5個批：

train_data = mnist_train.map(scale).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)
test_data = mnist_test.map(scale).batch(BATCH_SIZE)

STEPS_PER_EPOCH = 5

train_data = train_data.take(STEPS_PER_EPOCH)
test_data = test_data.take(STEPS_PER_EPOCH)

image_batch, label_batch = next(iter(train_data))

2. Use Keras training loops

    使用Keras訓練循環

 If you don't need low level control of your training process, using Keras's built-in fit, evaluate, and predict methods is recommended. These methods provide a uniform interface to train the model regardless of the implementation (sequential, functional, or sub-classed).

如果你不使用低水平的控制你的訓練過程，建議使用Keras的內置的擬合、評估和預測方法。這些方法提供了統一的接口來訓練模型，而不管實現是（序列的，函數的，或者子類的）。

The advantages of these methods include:

這些方法的優點包括：

• They accept Numpy arrays, Python generators and, tf.data.Datasets
• They apply regularization, and activation losses automatically.
• They support tf.distribute for multi-device training.
• They support arbitrary callables as losses and metrics.
• They support callbacks like tf.keras.callbacks.TensorBoard, and custom callbacks.
• They are performant, automatically using TensorFlow graphs.

Here is an example of training a model using a Dataset. (For details on how this works see tutorials.)

下面是一個使用數據集訓練模型的示例。（有關如何工作的詳細信息，請參見教程。）

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu',
                           kernel_regularizer=tf.keras.regularizers.l2(0.02),
                           input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10)
])

# Model is the full model w/o custom layers
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

model.fit(train_data, epochs=NUM_EPOCHS)
loss, acc = model.evaluate(test_data)

print("Loss {}, Accuracy {}".format(loss, acc))

Epoch 1/5
5/5 [==============================] - 0s 6ms/step - loss: 1.5323 - accuracy: 0.5063
Epoch 2/5
5/5 [==============================] - 0s 6ms/step - loss: 0.4105 - accuracy: 0.9219
Epoch 3/5
5/5 [==============================] - 0s 7ms/step - loss: 0.2495 - accuracy: 0.9531
Epoch 4/5
5/5 [==============================] - 0s 6ms/step - loss: 0.1806 - accuracy: 0.9875
Epoch 5/5
5/5 [==============================] - 0s 6ms/step - loss: 0.1416 - accuracy: 0.9937
5/5 [==============================] - 0s 4ms/step - loss: 1.5655 - accuracy: 0.6469
Loss 1.565544605255127, Accuracy 0.6468750238418579

3. Write your own loop

   書寫你自己的循環

If the Keras model's training step works for you, but you need more control outside that step, consider using the tf.keras.Model.train_on_batch method, in your own data-iteration loop.

如果Keras模型的訓練步驟適合你，但是你需要在步驟之外更多控制，考慮使用tf.keras.Model.train_on_batch方法，在你自己的數據迭代循環中。

Remember: Many things can be implemented as a tf.keras.callbacks.Callback.

記住：很多事情都可以作為tf.keras.callbacks.Callback實現。

This method has many of the advantages of the methods mentioned in the previous section, but gives the user control of the outer loop.

本方法具有之前所提到的所有的方法的優點，但是又給了用戶外部控制循環的能力。

You can also use tf.keras.Model.test_on_batch or tf.keras.Model.evaluate to check performance during training.

你也可以使用tf.keras.Model.test_on_batch或者tf.keras.Model.evaluate來檢查訓練過程中的性能。

Note: train_on_batch and test_on_batch, by default return the loss and metrics for the single batch. If you pass reset_metrics=False they return accumulated metrics and you must remember to appropriately reset the metric accumulators. Also remember that some metrics like AUC require reset_metrics=False to be calculated correctly.

注：在“批上訓練”和“批量測試”，默認情況下返回單個批次的損失和指標。如果傳遞reset_metrics=False，則返回累積的度量值，並且必須記住適當地重置度量值累加器。還請記住，某些指標（如AUC）需要reset_metrics=False才能正確計算。

To continue training the above model:

要繼續訓練上述模型：

# Model is the full model w/o custom layers
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

for epoch in range(NUM_EPOCHS):
  #Reset the metric accumulators
  model.reset_metrics()

  for image_batch, label_batch in train_data:
    result = model.train_on_batch(image_batch, label_batch)
    metrics_names = model.metrics_names
    print("train: ",
          "{}: {:.3f}".format(metrics_names[0], result[0]),
          "{}: {:.3f}".format(metrics_names[1], result[1]))
  for image_batch, label_batch in test_data:
    result = model.test_on_batch(image_batch, label_batch,
                                 # return accumulated metrics
                                 reset_metrics=False)
  metrics_names = model.metrics_names
  print("\neval: ",
        "{}: {:.3f}".format(metrics_names[0], result[0]),
        "{}: {:.3f}".format(metrics_names[1], result[1]))

train:  loss: 0.122 accuracy: 0.984
train:  loss: 0.133 accuracy: 0.984
train:  loss: 0.164 accuracy: 0.969
train:  loss: 0.167 accuracy: 0.969
train:  loss: 0.161 accuracy: 0.984

eval:  loss: 1.583 accuracy: 0.669
train:  loss: 0.074 accuracy: 1.000
train:  loss: 0.090 accuracy: 1.000
train:  loss: 0.089 accuracy: 1.000
train:  loss: 0.095 accuracy: 1.000
train:  loss: 0.090 accuracy: 1.000

eval:  loss: 1.567 accuracy: 0.747
train:  loss: 0.065 accuracy: 1.000
train:  loss: 0.068 accuracy: 1.000
train:  loss: 0.056 accuracy: 1.000
train:  loss: 0.069 accuracy: 1.000
train:  loss: 0.067 accuracy: 1.000

eval:  loss: 1.545 accuracy: 0.772
train:  loss: 0.053 accuracy: 1.000
train:  loss: 0.063 accuracy: 0.984
train:  loss: 0.050 accuracy: 1.000
train:  loss: 0.051 accuracy: 1.000
train:  loss: 0.049 accuracy: 1.000

eval:  loss: 1.520 accuracy: 0.778
train:  loss: 0.049 accuracy: 1.000
train:  loss: 0.046 accuracy: 1.000
train:  loss: 0.044 accuracy: 1.000
train:  loss: 0.045 accuracy: 1.000
train:  loss: 0.044 accuracy: 1.000

eval:  loss: 1.494 accuracy: 0.791

4. Customize the training step

   自定義培訓步驟

If you need more flexibility and control, you can have it by implementing your own training loop. There are three steps:

如果你需要更多的彈性和控制，你可以通過實現你自己的訓練循環擁有它。這里是三個步驟：

1. Iterate over a Python generator or tf.data.Dataset to get batches of examples.
2. Use tf.GradientTape to collect gradients.
3. Use one of the tf.keras.optimizers to apply weight updates to the model's variables.

記得：

• Always include a training argument on the call method of subclassed layers and models.
• Make sure to call the model with the training argument set correctly.
• Depending on usage, model variables may not exist until the model is run on a batch of data.
• You need to manually handle things like regularization losses for the model.

請注意相對於v1的簡化：

• There is no need to run variable initializers. Variables are initialized on creation.
• There is no need to add manual control dependencies. Even in tf.function operations act as in eager mode.

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu',
                           kernel_regularizer=tf.keras.regularizers.l2(0.02),
                           input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10)
])

optimizer = tf.keras.optimizers.Adam(0.001)
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

@tf.function
def train_step(inputs, labels):
  with tf.GradientTape() as tape:
    predictions = model(inputs, training=True)
    regularization_loss=tf.math.add_n(model.losses)
    pred_loss=loss_fn(labels, predictions)
    total_loss=pred_loss + regularization_loss

  gradients = tape.gradient(total_loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))

for epoch in range(NUM_EPOCHS):
  for inputs, labels in train_data:
    train_step(inputs, labels)
  print("Finished epoch", epoch)

Finished epoch 0
Finished epoch 1
Finished epoch 2
Finished epoch 3
Finished epoch 4

5. New-style metrics and losses

    新風格的指標和損失

In TensorFlow 2.0, metrics and losses are objects. These work both eagerly and in tf.functions.

在TensorFlow2.0中，度量和損失是對象。這兩種方法工作起來都很飢餓而且在tf.functions中.

A loss object is callable, and expects the (y_true, y_pred) as arguments:

loss對象是可調用的，並期望（y_true，y_pred）作為參數：

cce = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
cce([[1, 0]], [[-1.0,3.0]]).numpy()

4.01815

A metric object has the following methods:

度量對象具有如下方法：

• Metric.update_state() — add new observations
• Metric.result() —get the current result of the metric, given the observed values
• Metric.reset_states() — clear all observations.

The object itself is callable. Calling updates the state with new observations, as with update_state, and returns the new result of the metric.

對象本身是可調用的。調用用新的觀察值更新狀態，就像update_state一樣，並返回度量的新結果。

You don't have to manually initialize a metric's variables, and because TensorFlow 2.0 has automatic control dependencies, you don't need to worry about those either.

您不必手動初始化度量的變量，而且由於TensorFlow2.0具有自動控制依賴關系，因此您也不必擔心這些變量。

The code below uses a metric to keep track of the mean loss observed within a custom training loop.

下面的代碼使用一個度量來跟蹤在定制訓練循環中觀察到的平均損失。

# Create the metrics
loss_metric = tf.keras.metrics.Mean(name='train_loss')
accuracy_metric = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')

@tf.function
def train_step(inputs, labels):
  with tf.GradientTape() as tape:
    predictions = model(inputs, training=True)
    regularization_loss=tf.math.add_n(model.losses)
    pred_loss=loss_fn(labels, predictions)
    total_loss=pred_loss + regularization_loss

  gradients = tape.gradient(total_loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))
  # Update the metrics
  loss_metric.update_state(total_loss)
  accuracy_metric.update_state(labels, predictions)


for epoch in range(NUM_EPOCHS):
  # Reset the metrics
  loss_metric.reset_states()
  accuracy_metric.reset_states()

  for inputs, labels in train_data:
    train_step(inputs, labels)
  # Get the metric results
  mean_loss=loss_metric.result()
  mean_accuracy = accuracy_metric.result()

  print('Epoch: ', epoch)
  print('  loss:     {:.3f}'.format(mean_loss))
  print('  accuracy: {:.3f}'.format(mean_accuracy))

Epoch:  0
  loss:     0.175
  accuracy: 0.994
Epoch:  1
  loss:     0.149
  accuracy: 0.991
Epoch:  2
  loss:     0.133
  accuracy: 0.991
Epoch:  3
  loss:     0.113
  accuracy: 0.997
Epoch:  4
  loss:     0.101
  accuracy: 0.997

6. Keras metric names

   Keras度量名字

In TensorFlow 2.0 keras models are more consistent about handling metric names.

在TensorFlow 2.0中，keras模型在處理度量名稱方面更加一致。

Now when you pass a string in the list of metrics, that exact string is used as the metric's name. These names are visible in the history object returned by model.fit, and in the logs passed to keras.callbacks. is set to the string you passed in the metric list.

現在，當您在度量列表中傳遞一個字符串時，該字符串將用作度量的名稱。這些名稱在返回的歷史對象中可見模型.擬合，並在傳遞給keras.回調. 設置為在度量列表中傳遞的字符串。

model.compile(
    optimizer = tf.keras.optimizers.Adam(0.001),
    loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics = ['acc', 'accuracy', tf.keras.metrics.SparseCategoricalAccuracy(name="my_accuracy")])
history = model.fit(train_data)

 

5/5 [==============================] - 0s 6ms/step - loss: 0.1076 - acc: 0.9969 - accuracy: 0.9969 - my_accuracy: 0.9969

history.history.keys()

dict_keys(['loss', 'acc', 'accuracy', 'my_accuracy'])

This differs from previous versions where passing metrics=["accuracy"] would result in dict_keys(['loss', 'acc'])

這與以前的版本不同，在以前的版本中，傳遞metrics=[“accuracity”]將導致dict_鍵（['loss'，'acc']）

7. Keras optimizers

    Keras優化器

The optimizers in v1.train, like v1.train.AdamOptimizer and v1.train.GradientDescentOptimizer, have equivalents in tf.keras.optimizers.

v1.train中的優化器，比如v1.train.AdamOptimizer和v1.train.GradientDescentOptimizer，在tf.keras.optimizers擁有相同的功能.

Convert v1.train to keras.optimizers

將v1.train轉換為keras.optimizers

Here are things to keep in mind when converting your optimizers:

這里是你在轉換優化器時需要記住的幾點：

• Upgrading your optimizers may make old checkpoints incompatible.
• All epsilons now default to 1e-7 instead of 1e-8 (which is negligible in most use cases).
• v1.train.GradientDescentOptimizer can be directly replaced by tf.keras.optimizers.SGD.
• v1.train.MomentumOptimizer can be directly replaced by the SGD optimizer using the momentum argument: tf.keras.optimizers.SGD(..., momentum=...).
• v1.train.AdamOptimizer can be converted to use tf.keras.optimizers.Adam. The beta1 and beta2 arguments have been renamed to beta_1 and beta_2.
• v1.train.RMSPropOptimizer can be converted to tf.keras.optimizers.RMSprop. The decay argument has been renamed to rho.
• v1.train.AdadeltaOptimizer can be converted directly to tf.keras.optimizers.Adadelta.
• tf.train.AdagradOptimizer can be converted directly to tf.keras.optimizers.Adagrad.
• tf.train.FtrlOptimizer can be converted directly to tf.keras.optimizers.Ftrl. The accum_name and linear_name arguments have been removed.
• The tf.contrib.AdamaxOptimizer and tf.contrib.NadamOptimizer, can be converted directly to tf.keras.optimizers.Adamax and tf.keras.optimizers.Nadam. The beta1, and beta2 arguments have been renamed to beta_1 and beta_2.

New defaults for some tf.keras.optimizers

一些tf.keras.optimizers新的默認值

警告：如果您看到模型的收斂行為發生了變化，請檢查默認的學習速率。

There are no changes for optimizers.SGD, optimizers.Adam, or optimizers.RMSprop.

沒有更改優化器.SGD, 優化器.Adam，或優化器.RMSprop.

The following default learning rates have changed:

 以下默認學習率已更改：

• optimizers.Adagrad from 0.01 to 0.001
• optimizers.Adadelta from 1.0 to 0.001
• optimizers.Adamax from 0.002 to 0.001
• optimizers.Nadam from 0.002 to 0.001

8. TensorBoard

TensorFlow 2 includes significant changes to the tf.summary API used to write summary data for visualization in TensorBoard. For a general introduction to the new tf.summary, there are several tutorials available that use the TF 2 API. This includes a TensorBoard TF 2 Migration Guide

TensorFlow 2包括對tf.總結用於在TensorBoard中編寫可視化摘要數據的API。對於新的tf.總結，有幾個教程可以使用tf2api。這包括一個TensorBoard TF2遷移指南

 

 

使代碼2.0原生化：

1.替換tf.Session.run調用

2.使用Python對象來跟蹤變量和損失

3.升級您的訓練循環

4.升級數據輸入管道

https://www.mashangxue123.com/tensorflow/tf2-guide-migration_guide.html

https://tensorflow.google.cn/beta/guide/migration_guide

https://github.com/mashangxue/tensorflow2-zh/edit/master/r2/guide/migration_guide.md