Tensorflow相关知识（二）运用loss及gradients更新variables

古城微笑少年丶 2022-03-15 02:20 247阅读 0赞

背景：之前使用tensorflow的时候一直用下载好的模型的训练的代码，我们想要细化到模型内部，看看tensorflow到底如何执行模型训练的步骤。

参考：[http://wiki.jikexueyuan.com/project/tensorflow-zh/api\_docs/python/train.html][http_wiki.jikexueyuan.com_project_tensorflow-zh_api_docs_python_train.html]

[https://www.cnblogs.com/wuzhitj/p/6648641.html][https_www.cnblogs.com_wuzhitj_p_6648641.html]

**目录**

一、总览Optimizers

二、计算gradients并apply

2.1 计算graidents并apply的方法

2.2 运算并apply梯度相关函数

函数tf.train.Optimizer.\_\_init\_\_

函数tf.train.Optimizer.minimize

函数tf.train.Optimizer.compute\_gradients

函数tf.train.Optimizer.apply\_gradients

2.3 Gating Gradients并行的运用梯度

2.4 Slots额外训练变量

2.5 optimizers

三、运算梯度

3.1 梯度运算

tf.gradients计算偏导和

class tf.AggregationMethod

3.2 Gradient Clipping梯度截断

tf.clip\_by\_value通过梯度值截断梯度

其他函数

四、退化学习率

五、移动平均

六、协调器和队列运行器(Coordinator and QueueRunner)分布式执行(Summary Operations)

七、记录写入文件Adding Summaries to Event Files

八、Training utilities

tf.train.global\_step(sess, global\_step\_tensor)

tf.train.write\_graph(graph\_def, logdir, name, as\_text=True)

--------------------

## 一、总览Optimizers ##

用于运算相应的loss的梯度，然后将梯度应用于variable上。

class tf.train.Optimizers

这个class定义了用于训练模型的API和Ops，但是不会直接用到这个类，而是用到它的子类，比如`GradientDescentOptimizer`, `AdagradOptimizer`, `MomentumOptimizer这些`

例如：

# Create an optimizer with the desired parameters.
    opt = GradientDescentOptimizer(learning_rate=0.1)
    # Add Ops to the graph to minimize a cost by updating a list of variables.
    # "cost" is a Tensor, and the list of variables contains variables.Variable
    # objects.
    opt_op = opt.minimize(cost, <list of variables>)

训练过程之中使用返回的ops

# Execute opt_op to do one step of training:
    opt_op.run()

# 二、计算gradients并apply #

## 2.1 计算graidents并apply的方法 ##

`一次性的:`

`函数minimize()可以运算梯度并且将梯度应用于variable，因为此函数是简单的合并了分步骤的两个函数compute_gradients()`和`apply_gradients()`.

**分步的 ：**

1.  Compute the gradients with `compute_gradients()`.
2.  Process the gradients as you wish.
3.  Apply the processed gradients with `apply_gradients()`.

例如：

# Create an optimizer.
    opt = GradientDescentOptimizer(learning_rate=0.1)
    
    # Compute the gradients for a list of variables.
    grads_and_vars = opt.compute_gradients(loss, <list of variables>)
    
    # grads_and_vars is a list of tuples (gradient, variable).  Do whatever you
    # need to the 'gradient' part, for example cap them, etc.
    capped_grads_and_vars = [(MyCapper(gv[0]), gv[1])) for gv in grads_and_vars]
    
    # Ask the optimizer to apply the capped gradients.
    opt.apply_gradients(capped_grads_and_vars)

注意，得到的capped\_grads\_and\_vars是一个tuples，（元组不可更改，只可连接和删除）

## 2.2 运算并apply梯度相关函数 ##

### **`函数tf.train.Optimizer.__init__`** ###

**`tf.train.Optimizer.__init__(use_locking, name)`**

Create a new Optimizer.This must be called by the constructors of subclasses.

输入参数:

*  **`use_locking`**: Bool. If True apply use locks to prevent concurrent updates to variables.
 *  **`name`**: A non-empty string. The name to use for accumulators created for the optimizer.

### **`函数tf.train.Optimizer.minimize`** ###

**`tf.train.Optimizer.minimize(loss, global_step=None, var_list=None, gate_gradients=1, name=None)`**

Add operations to minimize 'loss' by updating 'var\_list'.也就是上文说的一次性的调用这个函数来运用loss更新相应的variable

函数简单的结合compute\_gradients() and apply\_gradients(). 如果想要在运用梯度处理数据之前进行一些操作，需要将两个函数分开运行compute\_gradients() and apply\_gradients() .

输入参数:

*  **`loss`**: A Tensor containing the value to minimize.
 *  **`global_step`**: Optional Variable to increment by one after the variables have been updated.
 *  **`var_list`**: Optional list of variables.Variable to update to minimize 'loss'. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE\_VARIABLES.
 *  **`gate_gradients`**: How to gate the computation of gradients. Can be GATE\_NONE, GATE\_OP, or GATE\_GRAPH.（仅仅是并行性的问题）
 *  **`name`**: Optional name for the returned operation.

Returns:

An Operation that updates the variables in 'var\_list'. If 'global\_step' was not None, that operation also increments global\_step.

### `函数tf.train.Optimizer.compute_gradients` ###

`tf.train.Optimizer.compute_gradients(loss, var_list=None, gate_gradients=1)`

Compute gradients of "loss" for the variables in "var\_list".

This is the first part of minimize(). It returns a list of (gradient, variable) pairs where "gradient" is the gradient for "variable". Note that "gradient" can be a Tensor, a IndexedSlices, or None if there is no gradient for the given variable.

Args:

*  **`loss`**: A Tensor containing the value to minimize.
 *  **`var_list`**: Optional list of variables.Variable to update to minimize "loss". Defaults to the list of variables collected in the graph under the key GraphKey.TRAINABLE\_VARIABLES.
 *  **`gate_gradients`**: How to gate the computation of gradients. Can be GATE\_NONE, GATE\_OP, or GATE\_GRAPH.

Returns:

A list of (gradient, variable) pairs.

Raises:

*  **`TypeError`**: If var\_list contains anything else than variables.Variable.
 *  **`ValueError`**: If some arguments are invalid.

### `函数tf.train.Optimizer.apply_gradients` ###

`tf.train.Optimizer.apply_gradients(grads_and_vars, global_step=None, name=None)`

Apply gradients to variables.

This is the second part of minimize(). It returns an Operation that applies gradients.

Args:

*  **`grads_and_vars`**: List of (gradient, variable) pairs as returned by compute\_gradients().
 *  **`global_step`**: Optional Variable to increment by one after the variables have been updated.
 *  **`name`**: Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

Returns:

An Operation that applies the specified gradients. If 'global\_step' was not None, that operation also increments global\_step.

## 2.3 Gating Gradients并行的运用梯度 ##

这部分推导可能涉及BP算法，暂时不深究。

其值可以取：GATE\_NONE, GATE\_OP 或 GATE\_GRAPH   
**GATE\_NONE :** 并行地计算和应用梯度。提供最大化的并行执行，但是会导致有的数据结果没有再现性。比如两个matmul操作的梯度依赖输入值，使用GATE\_NONE可能会出现有一个梯度在其他梯度之前便应用到某个输入中，导致出现不可再现的(non-reproducible)结果   
**GATE\_OP:** 对于每个操作Op，确保每一个梯度在使用之前都已经计算完成。这种做法防止了那些具有多个输入，并且梯度计算依赖输入情形中，多输入Ops之间的竞争情况出现。   
**GATE\_GRAPH:** 确保所有的变量对应的所有梯度在他们任何一个被使用前计算完成。该方式具有最低级别的并行化程度，但是对于想要在应用它们任何一个之前处理完所有的梯度计算时很有帮助的。

## 2.4 Slots额外训练变量 ##

一些optimizer的之类，比如 MomentumOptimizer 和 AdagradOptimizer 分配和管理着额外的用于训练的变量。这些变量称之为’Slots’，Slots有相应的名称，可以向optimizer访问的slots名称。有助于在log debug一个训练算法以及报告slots状态

`tf.train.Optimizer.get_slot_names()获得slots的名称`

Return a list of the names of slots created by the Optimizer.

See get\_slot().

Returns:

A list of strings

--------------------

`tf.train.Optimizer.get_slot(var, name)获得相应的slots值`

Return a slot named "name" created for "var" by the Optimizer.

Some Optimizer subclasses use additional variables. For example Momentum and Adagrad use variables to accumulate updates. This method gives access to these Variables if for some reason you need them.

Use get\_slot\_names() to get the list of slot names created by the Optimizer.

Args:

*  **`var`**: A variable passed to minimize() or apply\_gradients().
 *  **`name`**: A string.

Returns:

The Variable for the slot if it was created, None otherwise.

## 2.5 optimizers ##

具体参见 [http://wiki.jikexueyuan.com/project/tensorflow-zh/api\_docs/python/train.html][http_wiki.jikexueyuan.com_project_tensorflow-zh_api_docs_python_train.html]

<table> 
 <tbody> 
  <tr> 
   <td><strong>class tf.train.GradientDescentOptimizer</strong></td> 
   <td>使用梯度下降算法的Optimizer</td> 
  </tr> 
  <tr> 
   <td>tf.train.GradientDescentOptimizer.__init__(learning_rate,&nbsp;<br> use_locking=False, name=’GradientDescent’)</td> 
   <td>构建一个新的梯度下降优化器(Optimizer)</td> 
  </tr> 
  <tr> 
   <td><strong>class tf.train.AdadeltaOptimizer</strong></td> 
   <td>使用<a href="http://arxiv.org/abs/1212.5701" rel="nofollow">Adadelta算法</a>的Optimizer</td> 
  </tr> 
  <tr> 
   <td>tf.train.AdadeltaOptimizer.__init__(learning_rate=0.001,&nbsp;<br> rho=0.95, epsilon=1e-08,&nbsp;<br> use_locking=False, name=’Adadelta’)</td> 
   <td>创建Adadelta优化器</td> 
  </tr> 
  <tr> 
   <td><strong>class tf.train.AdagradOptimizer</strong></td> 
   <td>使用<a href="http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf" rel="nofollow">Adagrad算法</a>的Optimizer</td> 
  </tr> 
  <tr> 
   <td>tf.train.AdagradOptimizer.__init__(learning_rate,&nbsp;<br> initial_accumulator_value=0.1,&nbsp;<br> use_locking=False, name=’Adagrad’)</td> 
   <td>创建Adagrad优化器</td> 
  </tr> 
  <tr> 
   <td>class tf.train.MomentumOptimizer</td> 
   <td>使用<a href="http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf" rel="nofollow">Momentum</a>算法的Optimizer</td> 
  </tr> 
  <tr> 
   <td>tf.train.MomentumOptimizer.__init__(learning_rate,&nbsp;<br> momentum, use_locking=False,&nbsp;<br> name=’Momentum’, use_nesterov=False)</td> 
   <td>创建momentum优化器<br> momentum：动量，一个tensor或者浮点值</td> 
  </tr> 
  <tr> 
   <td><strong>class tf.train.AdamOptimizer</strong></td> 
   <td>使用<a href="http://arxiv.org/abs/1412.6980" rel="nofollow">Adam 算法</a>的Optimizer</td> 
  </tr> 
  <tr> 
   <td>tf.train.AdamOptimizer.__init__(learning_rate=0.001,<br> beta1=0.9, beta2=0.999, epsilon=1e-08,<br> use_locking=False, name=’Adam’)</td> 
   <td>创建Adam优化器</td> 
  </tr> 
  <tr> 
   <td>class tf.train.FtrlOptimizer</td> 
   <td>使用<a href="https://www.eecs.tufts.edu/~dsculley/papers/ad-click-prediction.pdf" rel="nofollow">FTRL 算法</a>的Optimizer</td> 
  </tr> 
  <tr> 
   <td>tf.train.FtrlOptimizer.__init__(learning_rate,&nbsp;<br> learning_rate_power=-0.5,&nbsp;<br> initial_accumulator_value=0.1,&nbsp;<br> l1_regularization_strength=0.0,&nbsp;<br> l2_regularization_strength=0.0,<br> use_locking=False, name=’Ftrl’)</td> 
   <td>创建FTRL算法优化器</td> 
  </tr> 
  <tr> 
   <td>class tf.train.RMSPropOptimizer</td> 
   <td>使用<a href="http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf" rel="nofollow">RMSProp算法</a>的Optimizer</td> 
  </tr> 
  <tr> 
   <td>tf.train.RMSPropOptimizer.__init__(learning_rate,&nbsp;<br> decay=0.9, momentum=0.0, epsilon=1e-10,&nbsp;<br> use_locking=False, name=’RMSProp’)</td> 
   <td>创建RMSProp算法优化器</td> 
  </tr> 
 </tbody> 
</table>

#  三、运算梯度 #

可以通过更底层的函数来对梯度进行更底层的操作。

## 3.1 梯度运算 ##

### `tf.gradients`计算偏导和 ###

注意区分，前面的`tf.train.Optimizer.compute_gradients和这里的tf.gradients是不一样的。`

`tf.gradients(ys, xs, grad_ys=None, name='gradients', colocate_gradients_with_ops=False, gate_gradients=False, aggregation_method=None)`

Constructs symbolic partial derivatives of `ys` w.r.t. x in `xs`.构建一个符号函数，计算ys关于xs中x的偏导的和，  
返回xs中每个x对应的sum(dy/dx)

`ys` and `xs` are each a `Tensor` or a list of tensors. `grad_ys` is a list of `Tensor`, holding the gradients received by the `ys`. The list must be the same length as `ys`.

`gradients()` adds ops to the graph to output the partial derivatives of `ys` with respect to `xs`. It returns a list of `Tensor` of length `len(xs)` where each tensor is the `sum(dy/dx)` for y in `ys`.

`grad_ys` is a list of tensors of the same length as `ys` that holds the initial gradients for each y in `ys`. When `grad_ys` is None, we fill in a tensor of '1's of the shape of y for each y in `ys`. A user can provide their own initial 'grad\_ys\` to compute the derivatives using a different initial gradient for each y (e.g., if one wanted to weight the gradient differently for each value in each y).

Args:

*  **`ys`**: A `Tensor` or list of tensors to be differentiated.
 *  **`xs`**: A `Tensor` or list of tensors to be used for differentiation.
 *  **`grad_ys`**: Optional. A `Tensor` or list of tensors the same size as `ys` and holding the gradients computed for each y in `ys`.
 *  **`name`**: Optional name to use for grouping all the gradient ops together. defaults to 'gradients'.
 *  **`colocate_gradients_with_ops`**: If True, try colocating gradients with the corresponding op.
 *  **`gate_gradients`**: If True, add a tuple around the gradients returned for an operations. This avoids some race conditions.
 *  **`aggregation_method`**: Specifies the method used to combine gradient terms. Accepted values are constants defined in the class `AggregationMethod`.

Returns:

A list of `sum(dy/dx)` for each x in `xs`.

Raises:

*  **`LookupError`**: if one of the operations between `x` and `y` does not have a registered gradient function.
 *  **`ValueError`**: if the arguments are invalid.

### `class tf.AggregationMethod`  ###

A class listing aggregation methods used to combine gradients.集合的方法用于聚集梯度。

Computing partial derivatives can require aggregating gradient contributions. This class lists the various methods that can be used to combine gradients in the graph:主要用于聚集梯度  计算偏导数需要聚集梯度贡献,这个类拥有在计算图中聚集梯度的很多方法。

*  `ADD_N`: All of the gradient terms are summed as part of one operation using the "AddN" op. It has the property that all gradients must be ready before any aggregation is performed.
 *  `DEFAULT`: The system-chosen default aggregation method.

<table> 
 <tbody> 
  <tr> 
   <td>tf.gradients(ys, xs, grad_ys=None, name=’gradients’,&nbsp;<br> colocate_gradients_with_ops=False, gate_gradients=False,&nbsp;<br> aggregation_method=None)</td> 
   <td>构建一个符号函数，计算ys关于xs中x的偏导的和，<br> 返回xs中每个x对应的sum(dy/dx)</td> 
  </tr> 
  <tr> 
   <td>tf.stop_gradient(input, name=None)</td> 
   <td>停止计算梯度，<br> 在EM算法、Boltzmann机等可能会使用到</td> 
  </tr> 
 </tbody> 
</table>

## 3.2 Gradient Clipping梯度截断 ##

可以在图中加截断函数，用这些函数实现基本的数据截断。对于梯度爆炸和梯度消失特别有用（particularly useful for handling exploding or vanishing gradients.）

### `tf.clip_by_value通过梯度值截断梯度` ###

`tf.clip_by_value(t, clip_value_min, clip_value_max, name=None)`

设置张量的最大值最小值.

`给定张量t`,操作返回一个相同类型和形状的张量 它的值被 `clip_value_min` and `clip_value_max`. 所有小于 `clip_value_min` are 设置为 `clip_value_min`. 所有大于 `clip_value_max` 设置为 `clip_value_max`.

Args:

*  **`t`**: A `Tensor`.
 *  **`clip_value_min`**: A 0-D (scalar) `Tensor`. The minimum value to clip by.
 *  **`clip_value_max`**: A 0-D (scalar) `Tensor`. The maximum value to clip by.
 *  **`name`**: A name for the operation (optional).

Returns:

A clipped `Tensor`.

### 其他函数 ###

<table> 
 <tbody> 
  <tr> 
   <td>tf.clip_by_value(t, clip_value_min, clip_value_max, name=None)</td> 
   <td>基于定义的min与max对tesor数据进行截断操作，<br> 目的是为了应对梯度爆发或者梯度消失的情况</td> 
  </tr> 
  <tr> 
   <td>tf.clip_by_norm(t, clip_norm, axes=None, name=None)</td> 
   <td>使用L2范式标准化tensor最大值为clip_norm<br> 返回 t * clip_norm / l2norm(t)</td> 
  </tr> 
  <tr> 
   <td>tf.clip_by_average_norm(t, clip_norm, name=None)</td> 
   <td>使用平均L2范式规范tensor数据t，<br> 并以clip_norm为最大值<br> 返回 t * clip_norm / l2norm_avg(t)</td> 
  </tr> 
  <tr> 
   <td>tf.clip_by_global_norm(t_list,&nbsp;<br> clip_norm, use_norm=None, name=None)</td> 
   <td>返回t_list[i] * clip_norm / max(global_norm, clip_norm)<br> 其中global_norm = sqrt(sum([l2norm(t)**2 for t in t_list]))</td> 
  </tr> 
  <tr> 
   <td>tf.global_norm(t_list, name=None)</td> 
   <td>返回global_norm = sqrt(sum([l2norm(t)**2 for t in t_list]))</td> 
  </tr> 
 </tbody> 
</table>

# 四、退化学习率 #

`tf.train.exponential_decay(learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None)`

对学习率进行指数衰退，公式如下

decayed_learning_rate = learning_rate *
                            decay_rate ^ (global_step / decay_steps)

If the argument `staircase` is `True`, then `global_step /decay_steps` is an integer division and the decayed learning rate follows a staircase function.

例如: 每100000 步进行0.96的decay:

...
    global_step = tf.Variable(0, trainable=False)
    starter_learning_rate = 0.1
    learning_rate = tf.exponential_decay(starter_learning_rate, global_step,
                                         100000, 0.96, staircase=True)
    optimizer = tf.GradientDescent(learning_rate)
    # Passing global_step to minimize() will increment it at each step.
    optimizer.minimize(...my loss..., global_step=global_step)

# 五、移动平均 #

一些训练优化算法，比如GradientDescent 和Momentum 在优化过程中便可以使用到移动平均方法。使用移动平均常常可以较明显地改善结果。

<table> 
 <thead> 
  <tr> 
   <th style="width:586px;">操作</th> 
   <th style="width:262px;">描述</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td style="width:586px;">class tf.train.ExponentialMovingAverage</td> 
   <td style="width:262px;">将指数衰退加入到移动平均中</td> 
  </tr> 
  <tr> 
   <td style="width:586px;">tf.train.ExponentialMovingAverage.apply(var_list=None)</td> 
   <td style="width:262px;">对var_list变量保持移动平均</td> 
  </tr> 
  <tr> 
   <td style="width:586px;">tf.train.ExponentialMovingAverage.average_name(var)</td> 
   <td style="width:262px;">返回var均值的变量名称</td> 
  </tr> 
  <tr> 
   <td style="width:586px;">tf.train.ExponentialMovingAverage.average(var)</td> 
   <td style="width:262px;">返回var均值变量</td> 
  </tr> 
  <tr> 
   <td style="width:586px;">tf.train.ExponentialMovingAverage.variables_to_restore(moving_avg_variables=None)</td> 
   <td style="width:262px;">返回用于保存的变量名称的映射</td> 
  </tr> 
 </tbody> 
</table>

# Create variables.
    var0 = tf.Variable(...)
    var1 = tf.Variable(...)
    # ... use the variables to build a training model...
    ...
    # Create an op that applies the optimizer.  This is what we usually
    # would use as a training op.
    opt_op = opt.minimize(my_loss, [var0, var1])
    
    # Create an ExponentialMovingAverage object
    ema = tf.train.ExponentialMovingAverage(decay=0.9999)
    
    # Create the shadow variables, and add ops to maintain moving averages
    # of var0 and var1.
    maintain_averages_op = ema.apply([var0, var1])
    
    # Create an op that will update the moving averages after each training
    # step.  This is what we will use in place of the usuall trainig op.
    with tf.control_dependencies([opt_op]):
        training_op = tf.group(maintain_averages_op)
    
    ...train the model by running training_op...

恢复shadow variable values

# Create a Saver that loads variables from their saved shadow values.
    shadow_var0_name = ema.average_name(var0)
    shadow_var1_name = ema.average_name(var1)
    saver = tf.train.Saver({shadow_var0_name: var0, shadow_var1_name: var1})
    saver.restore(...checkpoint filename...)
    # var0 and var1 now hold the moving average values

# **六、协调器和队列运行器(Coordinator and QueueRunner)分布式执行(Summary Operations)** #

参见[https://www.cnblogs.com/wuzhitj/p/6648641.html][https_www.cnblogs.com_wuzhitj_p_6648641.html]

# 七、记录写入文件***Adding Summaries to Event Files*** #

<table> 
 <thead> 
  <tr> 
   <th>操作</th> 
   <th>描述</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>class tf.train.SummaryWriter</td> 
   <td>将summary协议buffer写入事件文件中</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.__init__(logdir, graph=None, max_queue=10, flush_secs=120, graph_def=None)</td> 
   <td>创建一个SummaryWriter实例以及新建一个事件文件</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.add_summary(summary, global_step=None)</td> 
   <td>将一个summary添加到事件文件中</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.add_session_log(session_log, global_step=None)</td> 
   <td>添加SessionLog到一个事件文件中</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.add_event(event)</td> 
   <td>添加一个事件到事件文件中</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.add_graph(graph, global_step=None, graph_def=None)</td> 
   <td>添加一个Graph到时间文件中</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.add_run_metadata(run_metadata, tag, global_step=None)</td> 
   <td>为一个单一的session.run()调用添加一个元数据信息</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.flush()</td> 
   <td>刷新时间文件到硬盘中</td> 
  </tr> 
  <tr> 
   <td>tf.train.SummaryWriter.close()</td> 
   <td>将事件问价写入硬盘中并关闭该文件</td> 
  </tr> 
  <tr> 
   <td>tf.train.summary_iterator(path)</td> 
   <td>一个用于从时间文件中读取时间协议buffer的迭代器</td> 
  </tr> 
 </tbody> 
</table>

*tf.train.SummaryWriter*

从session之中去除summary op, and pass it to a [SummaryWriter][http_wiki.jikexueyuan.com_project_tensorflow-zh_api_docs_python_train.html] 写入event file. Event files 包括 [`Event`][Event] protos that can contain `Summary` protos along with the timestamp and step. You can then use TensorBoard to visualize the contents of the event files. See [TensorBoard and Summaries][] for more details.  
如果我们传递一个Graph进入该构建器中，它将被添加到Event files当中，这一点与使用add\_graph()具有相同功能。   
TensorBoard 将从事件文件中提取该graph，并将其显示。所以我们能直观地看到我们建立的graph。我们通常从我们启动的session中传递graph：

...create a graph...
    # Launch the graph in a session.
    sess = tf.Session()
    # Create a summary writer, add the 'graph' to the event file.
    writer = tf.train.SummaryWriter(<some-directory>, sess.graph)

# 八、Training utilities #

--------------------

### `tf.train.global_step(sess, global_step_tensor)`  ###

Small helper to get the global step.

# Creates a variable to hold the global_step.
    global_step_tensor = tf.Variable(10, trainable=False, name='global_step')
    # Creates a session.
    sess = tf.Session()
    # Initializes the variable.
    sess.run(global_step_tensor.initializer)
    print 'global_step:', tf.train.global_step(sess, global_step_tensor)
    
    global_step: 10

Args:

*  **`sess`**: A brain `Session` object.
 *  **`global_step_tensor`**: `Tensor` or the `name` of the operation that contains the global step.

Returns:

The global step value.

--------------------

### `tf.train.write_graph(graph_def, logdir, name, as_text=True)`  ###

Writes a graph proto on disk.

The graph is written as a binary proto unless as\_text is `True`.

v = tf.Variable(0, name='my_variable')
    sess = tf.Session()
    tf.train.write_graph(sess.graph_def, '/tmp/my-model', 'train.pbtxt')

Args:

*  **`graph_def`**: A `GraphDef` protocol buffer.
 *  **`logdir`**: Directory where to write the graph.
 *  **`name`**: Filename for the graph.
 *  **`as_text`**: If `True`, writes the graph as an ASCII proto.

[http_wiki.jikexueyuan.com_project_tensorflow-zh_api_docs_python_train.html]: http://wiki.jikexueyuan.com/project/tensorflow-zh/api_docs/python/train.html
[https_www.cnblogs.com_wuzhitj_p_6648641.html]: https://www.cnblogs.com/wuzhitj/p/6648641.html
[Event]: https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/util/event.proto
[TensorBoard and Summaries]: http://wiki.jikexueyuan.com/project/tensorflow-zh/how_tos/summaries_and_tensorboard/index.html