TensorFlow官方文档学习|TensorFlow运作方式入门

雨点打透心脏的1/2处 2022-09-28 14:24 149阅读 0赞

## **认识MINIST数据集**    ##

<table> 
 <thead> 
  <tr> 
   <th>&nbsp;</th> 
   <th>目的</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td><code>data_sets.train</code></td> 
   <td>55000个图像和标签（labels），作为主要训练集。</td> 
  </tr> 
  <tr> 
   <td><code>data_sets.validation</code></td> 
   <td>5000个图像和标签，用于迭代验证训练准确度。</td> 
  </tr> 
  <tr> 
   <td><code>data_sets.test</code></td> 
   <td>10000个图像和标签，用于最终测试训练准确度（trained accuracy）。</td> 
  </tr> 
 </tbody> 
</table>

from tensorflow.examples.tutorials.mnist import input_data
    mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
    
    print (mnist.train.images.shape)
    print (mnist.train.labels.shape)
    print (mnist.validation.images.shape)
    print (mnist.validation.labels.shape)
    print (mnist.test.images.shape)
    print (mnist.test.labels.shape)

输出结果：

Extracting MNIST_data/train-images-idx3-ubyte.gz  #训练集图片 - 55000 张 训练图片, 5000 张 验证图片
    Extracting MNIST_data/train-labels-idx1-ubyte.gz  #训练集图片对应的数字标签
    Extracting MNIST_data/t10k-images-idx3-ubyte.gz   #测试集图片 - 10000 张 图片
    Extracting MNIST_data/t10k-labels-idx1-ubyte.gz   #测试集图片对应的数字标签
    (55000, 784) #训练集图像的shape
    (55000, 10)  #训练集图像标签的shape
    (5000, 784)  #验证集图像的shape
    (5000, 10)   #验证集图像标签的shape
    (10000, 784) #测试集图像的shape
    (10000, 10)  #测试集图像标签的shape

将数据存为变量，查看数据：

val_data=mnist.validation.images
    val_label=mnist.validation.labels
    
    print('验证集图像：\n',val_data)
    print('验证集标签：\n',val_label)

输出结果：

验证集图像：
     [[ 0.  0.  0. ...,  0.  0.  0.]
     [ 0.  0.  0. ...,  0.  0.  0.]
     [ 0.  0.  0. ...,  0.  0.  0.]
     ..., 
     [ 0.  0.  0. ...,  0.  0.  0.]
     [ 0.  0.  0. ...,  0.  0.  0.]
     [ 0.  0.  0. ...,  0.  0.  0.]]
    验证集标签：
     [[ 0.  0.  0. ...,  0.  0.  0.]
     [ 1.  0.  0. ...,  0.  0.  0.]
     [ 0.  0.  0. ...,  0.  0.  0.]
     ..., 
     [ 0.  0.  1. ...,  0.  0.  0.]
     [ 0.  1.  0. ...,  0.  0.  0.]
     [ 0.  0.  1. ...,  0.  0.  0.]]

## 构建图表 （Build the Graph） ##

在为数据创建占位符之后，就可以运行`mnist.py`文件，经过三阶段的模式函数操作：`inference()`，`loss()`，和`training()`。图表就构建完成了。

1.`inference()` —— 尽可能地构建好图表，满足促使神经网络向前反馈并做出预测的要求。

2.`loss()` —— 往inference图表中添加生成损失（loss）所需要的操作（ops）。

3.`training()` —— 往损失图表中添加计算并应用梯度（gradients）所需的操作。

## 推理（Inference） ##

def inference(images, hidden1_units, hidden2_units):
      """构建图表.它接受图像占位符为输入，
         在此基础上借助ReLu(Rectified Linear Units)激活函数，
         构建一对完全连接层（layers），
         以及一个有着十个节点（node）、指明了输出logtis模型的线性层。
      Args:
        images: 接受图像占位符为输入
        hidden1_units: 第一个隐层的大小
        hidden2_units: 第二个隐层的大小
      Returns:
        softmax_linear: 包含预测结果的tensor
      """
      
     # Hidden 1
     with tf.name_scope('hidden1'):
     weights = tf.Variable(
            tf.truncated_normal([IMAGE_PIXELS, hidden1_units],
                                stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))),
            name='weights')
        biases = tf.Variable(tf.zeros([hidden1_units]),
                             name='biases')
        hidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)
      # Hidden 2
      with tf.name_scope('hidden2'):
        weights = tf.Variable(
            tf.truncated_normal([hidden1_units, hidden2_units],
                                stddev=1.0 / math.sqrt(float(hidden1_units))),
            name='weights')
        biases = tf.Variable(tf.zeros([hidden2_units]),
                             name='biases')
        hidden2 = tf.nn.relu(tf.matmul(hidden1, weights) + biases)
      # Linear
      with tf.name_scope('softmax_linear'):
        weights = tf.Variable(
            tf.truncated_normal([hidden2_units, NUM_CLASSES],
                                stddev=1.0 / math.sqrt(float(hidden2_units))),
            name='weights')
        biases = tf.Variable(tf.zeros([NUM_CLASSES]),
                             name='biases')
        logits = tf.matmul(hidden2, weights) + biases
      return logits

每一层都创建于一个唯一的[`tf.name_scope`][tf.name_scope]之下，创建于该作用域之下的所有元素都将带有其前缀。

with tf.name_scope('hidden1'):

例如，当这些层是在 `hidden1`作用域下生成时，赋予权重变量的独特名称将会是" `hidden1/weights`"。

在定义的作用域中，每一层所使用的权重和偏差都在[`tf.Variable`][tf.Variable]实例中生成，并且包含了各自期望的shape。

每个变量在构建时，都会获得初始化操作（initializer ops）。

*  对weights,通过[`tf.truncated_normal`][tf.truncated_normal]函数初始化权重变量，赋予的shape则是一个二维tensor，其中第一个维度代表该层中权重变量所连接（connect from）的单元数量：输入图像的大小，第二个维度代表该层中权重变量所连接到的（connect to）单元数量：第二个隐层的单元数量。````[`tf.truncated_normal`][tf.truncated_normal]初始函数将根据所得到的均值和标准差，生成一个随机分布。
 *  通过[`tf.zeros`][tf.zeros]函数初始化偏差变量（biases），确保所有偏差的起始值都是0，而它们的shape则是其在该层中所接到的（connect to）单元数量。

# Hidden 1
      with tf.name_scope('hidden1'):
        weights = tf.Variable(
            tf.truncated_normal([IMAGE_PIXELS, hidden1_units],
                                stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))),
            name='weights')
        biases = tf.Variable(tf.zeros([hidden1_units]),
                             name='biases')
        hidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)

定义了每层的权重和偏置后，使用激活函数（选取ReLU）连接各层。

hidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)
    
    hidden2 = tf.nn.relu(tf.matmul(hidden1, weights) + biases)
    
    logits = tf.matmul(hidden2, weights) + biases

## 损失（Loss） ##

def loss(logits, labels):
      """通过添加所需的损失(输出层与label间)操作，进一步构建图表。
      Args:
        logits: Logits tensor, float - [batch_size, NUM_CLASSES].
        labels: Labels tensor, int32 - [batch_size].
      Returns:
        loss: Loss tensor of type float.
      """
      labels = tf.to_int64(labels)
      cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
          labels=labels, logits=logits, name='xentropy')
      return tf.reduce_mean(cross_entropy, name='xentropy_mean')

tf.nn.sparse\_softmax\_cross\_entropy\_with\_logits()函数计算输出层logits的softmax，

然后将其与labels求交叉熵。

tf.reduce\_mean()求cross\_entropy的平均值，将其作为总损失。

tensorflow中有一类在tensor的某一维度上求值的函数。如：
    
    求最大值tf.reduce_max(input_tensor, reduction_indices=None, keep_dims=False, name=None)
    
    求平均值tf.reduce_mean(input_tensor, reduction_indices=None, keep_dims=False, name=None)
    
    参数（1）input_tensor:待求值的tensor。
    
    参数（2）reduction_indices:在哪一维上求解。
    
    参数（3）（4）可忽略

## 训练（Training） ##

def training(loss, learning_rate):
      ""设置用于训练的Op
      Creates a summarizer to track the loss over time in TensorBoard.
      创建优化程序并为所有变量使用梯度下降算法。
      函数返回的Op是在启动会话时使模型开始训练的Op。
      Args:
        loss: Loss tensor, from loss().
        learning_rate: The learning rate to use for gradient descent.
      Returns:
        train_op: The Op for training.
      """
      # Add a scalar summary for the snapshot loss.
      tf.summary.scalar('loss', loss)
      # 实例化一个GradientDescentOptimizer，按照一定的学习速率应用梯度下降算法
      optimizer = tf.train.GradientDescentOptimizer(learning_rate)
      # 生成一个变量用于保存全局训练步骤（global training step）的数值
      global_step = tf.Variable(0, name='global_step', trainable=False)
      # 使用minimize()函数更新系统中的权重以使得loss变小、并作为增加全局步骤的计数器
      train_op = optimizer.minimize(loss, global_step=global_step)
      return train_op

## 评估（Evaluation） ##

def evaluation(logits, labels):
      """Evaluate the quality of the logits at predicting the label.
      Args:
        logits: Logits tensor, float - [batch_size, NUM_CLASSES].
        labels: Labels tensor, int32 - [batch_size], with values in the
          range [0, NUM_CLASSES).
      Returns:
        A scalar int32 tensor with the number of examples (out of batch_size)
        that were predicted correctly.
      """
      # For a classifier model, we can use the in_top_k Op.
      # It returns a bool tensor with shape [batch_size] that is true for
      # the examples where the label is in the top k (here k=1)
      # of all logits for that example.
      correct = tf.nn.in_top_k(logits, labels, 1)
      # Return the number of true entries.
      return tf.reduce_sum(tf.cast(correct, tf.int32))

*  tf.nn.in\_top\_k(predictions, targets, k, name=None)：这个函数也是专门用于分类的，意思是说：“对样本所属的类别进行估计，得到结果predictions，但是样本真正所属的类别是targets，该函数就判断targets所代表的类别在predictions中是否排在前k个，如果是，那就返回true“。
 *  tf.cast(x, dtype, name=None)：将x或者x.values转换为dtype。  
    
    
        # tensor a is [1.8, 2.2], dtype=tf.float
        tf.cast(a, tf.int32) ==> [1, 2] # dtype=tf.int32
 *  tf.reduce\_sum(input\_tensor, reduction\_indices=None, keep\_dims=False, name=None)：计算输入tensor元素的和，或者按照reduction\_indices指定的轴进行求和。
    
        # ‘x’ is [[1, 1, 1]
        # [1, 1, 1]]
        tf.reduce_sum(x) ==> 6

## 训练模型    ##

一旦图表构建完毕，就通过`fully_connected_feed.py`文件中的用户代码进行循环地迭代式训练和评估。

def run_training():
      """Train MNIST for a number of steps."""
      # 获得数据到data_sets中.
      data_sets = input_data.read_data_sets(FLAGS.input_data_dir, FLAGS.fake_data)
    
      # 将所有已构建的操作与tf的默认图关联起来
      with tf.Graph().as_default():
        # Generate placeholders for the images and labels.
        images_placeholder, labels_placeholder = placeholder_inputs(
            FLAGS.batch_size)
    
        # Build a Graph that computes predictions from the inference model.
        logits = mnist.inference(images_placeholder,
                                 FLAGS.hidden1,
                                 FLAGS.hidden2)
    
        # Add to the Graph the Ops for loss calculation.
        loss = mnist.loss(logits, labels_placeholder)
    
        # Add to the Graph the Ops that calculate and apply gradients.
        train_op = mnist.training(loss, FLAGS.learning_rate)
    
        # Add the Op to compare the logits to the labels during evaluation.
        eval_correct = mnist.evaluation(logits, labels_placeholder)
    
        # Build the summary Tensor based on the TF collection of Summaries.
        summary = tf.summary.merge_all()
    
        # Add the variable initializer Op.
        init = tf.global_variables_initializer()
    
        # Create a saver for writing training checkpoints.
        saver = tf.train.Saver()
    
        # Create a session for running Ops on the Graph.
        sess = tf.Session()
    
        # Instantiate a SummaryWriter to output summaries and the Graph.
        summary_writer = tf.summary.FileWriter(FLAGS.log_dir, sess.graph)
    
        # And then after everything is built:
    
        # Run the Op to initialize the variables.
        sess.run(init)
    
        # 启动训练循环
        for step in xrange(FLAGS.max_steps):
          start_time = time.time()
    
          # Fill a feed dictionary with the actual set of images and labels
          # for this particular training step.
          feed_dict = fill_feed_dict(data_sets.train,
                                     images_placeholder,
                                     labels_placeholder)
    
          # Run one step of the model.  The return values are the activations
          # from the `train_op` (which is discarded) and the `loss` Op.  To
          # inspect the values of your Ops or variables, you may include them
          # in the list passed to sess.run() and the value tensors will be
          # returned in the tuple from the call.
          _, loss_value = sess.run([train_op, loss],
                                   feed_dict=feed_dict)
    
          duration = time.time() - start_time
    
          # Write the summaries and print an overview fairly often.
          if step % 100 == 0:
            # Print status to stdout.
            print('Step %d: loss = %.2f (%.3f sec)' % (step, loss_value, duration))
            # Update the events file.
            summary_str = sess.run(summary, feed_dict=feed_dict)
            summary_writer.add_summary(summary_str, step)
            summary_writer.flush()
    
          # Save a checkpoint and evaluate the model periodically.
          if (step + 1) % 1000 == 0 or (step + 1) == FLAGS.max_steps:
            checkpoint_file = os.path.join(FLAGS.log_dir, 'model.ckpt')
            saver.save(sess, checkpoint_file, global_step=step)
            # Evaluate against the training set.
            print('Training Data Eval:')
            do_eval(sess,
                    eval_correct,
                    images_placeholder,
                    labels_placeholder,
                    data_sets.train)
            # Evaluate against the validation set.
            print('Validation Data Eval:')
            do_eval(sess,
                    eval_correct,
                    images_placeholder,
                    labels_placeholder,
                    data_sets.validation)
            # Evaluate against the test set.
            print('Test Data Eval:')
            do_eval(sess,
                    eval_correct,
                    images_placeholder,
                    labels_placeholder,
                    data_sets.test)

## 像Graph中提供数据 ##

TensorFlow的[feed机制][feed]可以在应用运行时向Graph输入数据。

在每一步训练过程中，首先会根据训练数据生成一个`feed dictionary`，这里面会包含本次循环中使用到的训练数据集。

feed_dict = fill_feed_dict(data_sets.train,
                                     images_placeholder,
                                     labels_placeholder)

fill\_feed\_dict()函数：

def fill_feed_dict(data_set, images_pl, labels_pl):
      """Fills the feed_dict for training the given step.
      A feed_dict takes the form of:
      Args:
        data_set: The set of images and labels, from input_data.read_data_sets()
        images_pl: The images placeholder, from placeholder_inputs().
        labels_pl: The labels placeholder, from placeholder_inputs().
      Returns:
        feed_dict: The feed dictionary mapping from placeholders to values.
      """
      # fill_feed_dict函数会查询给定的DataSet，索要下一批次batch_size的图像和标签，
      # 与占位符相匹配的Tensor则会包含下一批次的图像和标签。
      images_feed, labels_feed = data_set.next_batch(FLAGS.batch_size,
                                                     FLAGS.fake_data)
      # 然后，以占位符为哈希键，创建一个Python字典对象，键值则是其代表的反馈Tensor。
     feed_dict = {
          images_pl: images_feed,
          labels_pl: labels_feed,
      }
      # 这个字典随后作为feed_dict参数，传入sess.run()函数中，为这一步的训练提供输入样例。
     return feed_dict

## 评估模型 ##

每隔一千个训练步骤，我们的代码会尝试使用训练数据集与测试数据集，对模型进行评估。``

`do_eval`函数会被调用三次，分别使用训练数据集、验证数据集合测试数据集。

# Evaluate against the training set.
            print('Training Data Eval:')
            do_eval(sess,
                    eval_correct,
                    images_placeholder,
                    labels_placeholder,
                    data_sets.train)
            # Evaluate against the validation set.
            print('Validation Data Eval:')
            do_eval(sess,
                    eval_correct,
                    images_placeholder,
                    labels_placeholder,
                    data_sets.validation)
            # Evaluate against the test set.
            print('Test Data Eval:')
            do_eval(sess,
                    eval_correct,
                    images_placeholder,
                    labels_placeholder,
                    data_sets.test)

do\_eval()函数：

def do_eval(sess,
                eval_correct,
                images_placeholder,
                labels_placeholder,
                data_set):
      """Runs one evaluation against the full epoch of data.
      Args:
        sess: The session in which the model has been trained.
        eval_correct: The Tensor that returns the number of correct predictions.
        images_placeholder: The images placeholder.
        labels_placeholder: The labels placeholder.
        data_set: The set of images and labels to evaluate, from
          input_data.read_data_sets().
      """
      # And run one epoch of eval.
      true_count = 0  
      steps_per_epoch = data_set.num_examples // FLAGS.batch_size
      num_examples = steps_per_epoch * FLAGS.batch_size # 数据数量
      for step in xrange(steps_per_epoch):
        feed_dict = fill_feed_dict(data_set,
                                   images_placeholder,
                                   labels_placeholder)
        true_count += sess.run(eval_correct, feed_dict=feed_dict) # 预测正确的数量
      precision = float(true_count) / num_examples #准确率
      print('  Num examples: %d  Num correct: %d  Precision @ 1: %0.04f' %
            (num_examples, true_count, precision))

使用tensorboard没有成功，暂时放弃。（2017/3/13）

/root/anaconda3/lib/python3.5

/root/anaconda3/lib/python3.5/site-packages/tensorflow`/tensorboard/tensorboard.py`

/root/anaconda3/lib/python3.5/python /root/anaconda3/lib/python3.5/site-packages/tensorflow/tensorboard/tensorboard.py --logdir=/tmp/tensorflow/mnist/logs/fully\_connected\_feed

## 参考文献 ##

1.  [tf.reduce\_mean()函数说明][tf.reduce_mean]
2.  [tf.nn.in\_top\_k()函数说明][tf.nn.in_top_k]
3.  [TensorFlow基本概念与常用函数][TensorFlow]

[tf.name_scope]: http://www.tensorfly.cn/tfdoc/api_docs/python/framework.html#name_scope
[tf.Variable]: http://www.tensorfly.cn/tfdoc/api_docs/python/state_ops.html#Variable
[tf.truncated_normal]: http://www.tensorfly.cn/tfdoc/api_docs/python/constant_op.html#truncated_normal
[tf.zeros]: http://www.tensorfly.cn/tfdoc/api_docs/python/constant_op.html#zeros
[feed]: https://www.tensorflow.org/versions/r0.11/how_tos/reading_data/#top_of_page
[tf.reduce_mean]: http://blog.csdn.net/qq_32166627/article/details/52734387
[tf.nn.in_top_k]: http://m.blog.csdn.net/article/details?id=52410569
[TensorFlow]: http://blog.csdn.net/lenbow/article/details/52152766