使用 TensorBoard 記錄訓(xùn)練中的各項(xiàng)指標(biāo)

Use The TensorBoard to keep track of your training

在前面的學(xué)習(xí)中，我們學(xué)習(xí)到了如何使用 TensorBoard 來記錄 Loss 等基本的參數(shù)。那么我們可以定義更加復(fù)雜的參數(shù)指標(biāo)，以至于可以自定義指標(biāo)嗎？答案是可以的，那么我們這節(jié)課便來學(xué)習(xí)一下如何在 TensorBoard 之中輸出更加復(fù)雜的指標(biāo)甚至自定義指標(biāo)。

In the previous tutorial, we learned how to use the TensorBoard to record basic parameters such as Loss. So can we define more complex parameter metrics to the point where we can customize metrics? The answer is yes, so let’s take a look at how to output more complex metrics or even custom metrics in the TensorBoard.

自定義指標(biāo)大致可以分為兩種：

There are two broad categories of custom metrics:

• 使用回調(diào)進(jìn)行自定義的輸出； Custom output using callbacks;
• 在自定義循環(huán)中手動(dòng)添加輸出。 Manually add output in a custom loop

我們這節(jié)課來分別學(xué)習(xí)以下如何使用兩者來進(jìn)行自定義的輸出。

In this lesson, we will learn how to use each of the following to customize the output.

1. 使用 CallBack 自定義輸出

1.customizing output with CallBack

使用 CallBack 進(jìn)行自定義輸出的方法是采用其內(nèi)置的一些 CallBack 或者一些自定義的 CallBack，而這些 CallBack 之中包含有進(jìn)行指標(biāo)輸出的功能，故而我們可以使用他們進(jìn)行自定義輸出。

The way to use callbacks for custom output is to take some of the built-in callbacks or some of the custom callbacks that include the ability to do metric output, so we can use them for custom output.

使用 CallBack 進(jìn)行自定義輸出的大體步驟分為如下幾步：

The general steps for custom output using CallBack are as follows:

• 使用 tf.summary.create_file_writer() API 來編寫一個(gè)文件寫入器，我們會(huì)使用該寫入器來記錄我們的指標(biāo)； Use of TF. It’s not gonNA happen. The create () API to write a file writer that we will use to record our metrics;
• 自定義回調(diào)，并在回調(diào)的過程之中使用 tf.summary.scalar() 函數(shù)進(jìn)行指標(biāo)的寫入； Customize the callback, and use TF. During the callback. It’s not gonNA happen. The scalar () function writes the pointer;
• 在訓(xùn)練的過程之中調(diào)用該 CallBack。 This CallBack is called during the course of training

在這里，我們會(huì)采用一個(gè)學(xué)習(xí)率的例子來進(jìn)行演示：

Here, we’ll use a learning rate example to illustrate:

import tensorflow as tf

(x_train, y_train),(x_test, y_test) = tf.keras.datasets.mnist.load_data()

x_train, x_test = x_train / 255.0, x_test / 255.0

file_writer = tf.summary.create_file_writer("logs/2")
file_writer.set_as_default()

# 定義學(xué)習(xí)曲線，并且輸出指標(biāo)
def my_lr_schedule(epoch):
  learning_rate = 0.1
  learning_rate /= 2

  tf.summary.scalar('lr', data=learning_rate, step=epoch)
  return learning_rate

lr_callback = tf.keras.callbacks.LearningRateScheduler(my_lr_schedule)

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(256, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=[])

# 調(diào)用學(xué)習(xí)率回調(diào)
model.fit(
    x_train,
    y_train,
    epochs=3,
    validation_data=(x_test, y_test),
    callbacks=[lr_callback],
)

以上的模型比較簡單，因此我們可以通過運(yùn)行如下命令來查看結(jié)果：

The above model is relatively simple, so we can see the results by running the following command:

tensorboard --logdir logs/2

于是，我們便可以查看到我們自定義的學(xué)習(xí)率指標(biāo)的曲線的變化情況：

Then we can see how the curve of our custom learning rate indicator changes:

2. 在自定義循環(huán)之中自定義輸出指標(biāo)

2. Customize output metrics in a custom loop

在上面的例子之中，我們發(fā)現(xiàn)了在調(diào)用回調(diào)的過程之中，最核心的語句是如下的指示輸出語句：

In the example above, we found that the core statement during a callback call is the following directive output statement:

tf.summary.scalar()

既然如此，那么我們是否可以自己定義何時(shí)進(jìn)行輸出，以及輸出什么內(nèi)容呢？答案是肯定的，我們這一小結(jié)就會(huì)采用 tf.summary.scalar() 這一個(gè)核心 API 來進(jìn)行自定義的指標(biāo)的輸出。

In that case, can we define for ourselves when to output and what to output? The answer is yes, and we’ll use TF. For this summary. It’s not gonNA happen. SCALAR () is the core API for custom metric output.

首先，我們?nèi)匀徊捎们懊娴哪Ｐ团c數(shù)據(jù)：

First, we still use the previous models and data:

import tensorflow as tf
import numpy as np
(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.mnist.load_data()
# 數(shù)據(jù)歸一化
train_images = train_images / 255.0
test_images = test_images / 255.0

train_dataset = tf.data.Dataset.from_tensor_slices((train_images, train_labels))
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(64)

valid_dataset = tf.data.Dataset.from_tensor_slices((test_images, test_labels))
valid_dataset = valid_dataset.batch(64)

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(256, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

然后我們就可以定義我們的文件寫入器，并且指定其日志寫入目錄：

Then we can define our file writer and specify its log to write to the directory:

file_writer = tf.summary.create_file_writer("logs/3")
file_writer.set_as_default()

最后就到了我們的重頭戲，我們可以編寫自定義循環(huán)，并且在自定義循環(huán)之中定義我們要輸出的指標(biāo)，以及輸出的頻率等：

Finally, we get to the point where we can write a custom loop, and in the custom loop we define the metrics we want to output, the frequency of output, and so on:

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy()
optimizer = tf.keras.optimizers.Adam()
val_acc = tf.keras.metrics.SparseCategoricalAccuracy()
epochs = 3
for epoch in range(epochs):
    for batch_i, (x_batch_train, y_batch_train) in enumerate(train_dataset):
        with tf.GradientTape() as tape:
          outputs = model(x_batch_train, training=True)
          loss_value = loss_fn(y_batch_train, outputs)

        grads = tape.gradient(loss_value, model.trainable_weights)
        optimizer.apply_gradients(zip(grads, model.trainable_weights))

        if batch_i % 10 == 0:
            # 對訓(xùn)練數(shù)據(jù)進(jìn)行Log
            tf.summary.scalar('train_loss', data=float(loss_value), step=epoch*len(train_dataset)+batch_i)
            print("Loss at step %d: %.4f" % (epoch*len(train_dataset)+batch_i, float(loss_value)))
            

    for batch_i, (x_batch_train, y_batch_train) in enumerate(valid_dataset):
        # 對測試數(shù)據(jù)進(jìn)行Log
        outputs = model(x_batch_train, training=False)
        val_acc.update_state(y_batch_train, outputs)
    tf.summary.scalar('valid_acc', data=float(val_acc.result()), step=epoch)
    val_acc.reset_states()

在這里，我們在訓(xùn)練的過程之中使用 tf.summary.scalar 這個(gè) API 進(jìn)行指標(biāo)的輸出；而且我們讓其沒 10個(gè)批次進(jìn)行一次輸出。

Here, we use the TF. In the training process. It’s not gonNA happen. The scalar API does the metric’s output; and we let it do it all at once in less than 10 batches.

于是我們可以得到我們在訓(xùn)練集上的 Loss 指標(biāo)為：

So we get our Loss indicator on the training set as follows:

由此，我們完成了如何自定義進(jìn)行指標(biāo)輸出的工作。

This completes the work of customizing the output of the metric.

3. 小結(jié)

3. Summary

在這節(jié)課之中，我們學(xué)習(xí)了如何自定義指標(biāo)的輸出，其中包括使用通過CallBack來在訓(xùn)練的過程中輸出指標(biāo)以及如何在自定義訓(xùn)練的過程中輸出指標(biāo)。

In this lesson, we learned how to customize the output of metrics, including using CallBack to output metrics during training and how to output metrics during custom training.