TensorFlow笔记

我不是女神ヾ 2021-09-21 11:01 489阅读 0赞

TensorFlow笔记

本文是基于Windows 10系统环境，进行TensorFlow的学习和使用

  • Windows 10
  • PyCharm

一、TensorFlow的简介

1.1 TensorFlow基本概念

1.1.1 张量

张量就是多维数组（列表），用“阶”表示张量的维度

  • 0阶张量
    0 阶张量称作标量，表示一个单独的数，例如 S=123
  • 1阶张量
    1 阶张量称作向量，表示一个一维数组，例如 V=[1,2,3]
  • 2阶张量
    2 阶张量称作矩阵，表示一个二维数组，它可以有 i 行 j 列个元素，每个元素可以用行号和列号共同索引到，例如 m=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
  • n阶张量
    判断张量是几阶的，就通过张量右边的方括号数，0 个是 0 阶，n 个是 n 阶，张量可以表示 0 阶到 n 阶数组（列表）；

1.2 常量

定义常量，并打印输出

  1. import tensorflow as tf #引入模块
  3. x = tf.constant([[1.0, 2.0]]) #定义一个 2 阶张量等于[[1.0,2.0]]
  4. w = tf.constant([[3.0], [4.0]]) #定义一个 2 阶张量等于[[3.0],[4.0]]
  5. y = tf.matmul(x, w) #实现 xw 矩阵乘法
  6. print(y) #打印出结果
  8. with tf.Session() as sess:
  9.     print(sess.run(y)) #执行会话并打印出执行后的结果

1.3 常用的生成随机数/数组的函数

  1. #生成正态分布随机数
  2. tf.random_normal()     
  4. #生成去掉过大偏离点的正态分布随机数 
  5. tf.truncated_normal()    
  7. #生成均匀分布随机数
  8. tf.random_uniform()     
  10. #表示生成全 0 数组
  11. tf.zeros  
  13. #表示生成全 1 数组 
  14. tf.ones 
  16. #表示生成全定值数组 
  17. tf.fill 
  19. #表示生成直接给定值的数组
  20. tf.constant

1.4 变量初始化/打印

  1. #引入模块
  2. import tensorflow as tf 
  4. #表示去掉偏离过大的正态分布
  5. w=tf.Variable(tf.truncated_normal([2,3],stddev=2, mean=0, seed=1))
  7. with tf.Session() as sess:
  8.     #初始化
  9.     init_op = tf.global_variables_initializer()
  10.     sess.run(init_op)
  11.     print(sess.run(y)) #执行会话并打印出执行后的结果

1.5 placeholder占位/喂入数据

  • 喂一组数据

    引入模块

    import tensorflow as tf

    喂一组数据：

    x = tf.placeholder(tf.float32, shape=(1, 2))
    w1=tf.Variable(tf.random_normal([2,3],stddev=1,seed=1))
    y=tf.matmul(x,w1)

    with tf.Session() as sess:

    1. #初始化
    2. init_op = tf.global_variables_initializer()
    3. sess.run(init_op)
    4. sess.run(y, feed_dict={ x: [[0.5,0.6]]})

  • 喂N组数据

    引入模块

    import tensorflow as tf

    喂N组数据：

    x = tf.placeholder(tf.float32, shape=(None, 2))
    w1=tf.Variable(tf.random_normal([2,3],stddev=1,seed=1))
    y=tf.matmul(x,w1)

    with tf.Session() as sess:

    1. #初始化
    2. init_op = tf.global_variables_initializer()
    3. sess.run(init_op)
    4. sess.run(y, feed_dict={ x: [[0.1,0.2],[0.2,0.3],[0.3,0.4],[0.4,0.5]]})

1.6 损失函数

  • 均方误差MSE

    M S E ( y _ , y ) = ∑ i = 1 n ( y − y _ ) 2 n MSE(y_{\_},y)= \dfrac{\sum_{i=1}^n (y-y_{\_})^2}{n} MSE(y_​,y)=n∑i=1n​(y−y_​)2​

    引入模块

    import tensorflow as tf

    loss = tf.reducemean(tf.square(y, y))

  • 交叉熵ce

    H ( y _ , y ) = ∑ y _ ∗ l o g ( y ) H(y_{\_},y)= \sum{y_{\_}*log(y)} H(y_​,y)=∑y_​∗log(y)

    引入模块

    import tensorflow as tf

    ce = -tf.reducemean(y*tf.log(tf.clip_by_value(y, 1e-12, 1.0)))

  • softmax函数
    当n分类的n个输出 ( y 1 , y 2 , . . . , y n ) (y_{1},y_{2},…,y_{n}) (y1​,y2​,…,yn​)通过 s o f t m a x ( ) softmax() softmax()函数，便满足了概率分布要求：
    ∀ x , P ( X = x ) ∈ [ 0 , 1 ] ， ∑ x P ( X = x ) = 1 \forall{x}, P(X=x) \isin{[0,1]} ，\sum_{x}{P(X=x)=1} ∀x,P(X=x)∈[0,1]，x∑​P(X=x)=1
    s o f t m a x ( y i ) = e y i ∑ j = 1 n e y i softmax(y_{i})=\dfrac{e^{y_{i}}}{\sum_{j=1}^{n}{e^{y_{i}}}} softmax(yi​)=∑j=1n​eyi​eyi​​

    引入模块

    import tensorflow as tf

    ce = tf.nn.sparsesoftmax_cross_entropy_with_logits(logits=y,lables=tf.argmax(y, 1))
    cem = tf.reduce_mean(ce)

1.7 学习率

训练模型每次参数更新的幅度
w n + 1 = w n − l e a r n i n g _ r a t e ∗ ∇ w_{n+1}=w_{n}-learning{\_}rate*\nabla wn+1​=wn​−learning_rate∗∇
学习率设置过大，模型振荡不收敛；
学习了设置过小，模型收敛速度慢；

  • 指数衰减学习率

    引入模块

    import tensorflow as tf

    global_step = tf.Variable(0, trainable=False)
    learning_rate = tf.train.exponential_decay(learning_rate_base,global_step,learning_rate_step, learning_rate_decay,staircase=True)

1.8 滑动平均（影子值）

记录了每一个参数一段时间内过往值的平均值，增加了模型的泛化性，针对于所有参数：w和b

  1. #引入模块
  2. import tensorflow as tf 
  4. ema = tf.train.ExponentialMovingAverage(MOVING_AVERAGE_DECAY, global_step)
  5. ema_op= ema.apply(tf.trainable_variables())
  6. with tf.control_dependencies([train_step, ema_op]):
  7.     train_op = tf.no_op(name='train')

1.9 正则化

记录了每一个参数一段时间内过往值的平均值，增加了模型的泛化性，针对于所有参数：w和b

  1. #引入模块
  2. import tensorflow as tf 
  3. tf.add_to_collection('losses', tf.contrib,layers.l2_regularizer(regularizer)(w))
  5. loss_total = cem + tf.add_n(tf.get_collection('losses'))

2.0 反向传播训练方法

以减小 loss 值为优化目标，有梯度下降、momentum 优化器、adam 优化器等优化方法。

  • 梯度下降法

    引入模块

    import tensorflow as tf

    train_step=tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)

  • momentum 优化器

    引入模块

    import tensorflow as tf

    train_step=tf.train.MomentumOptimizer(learning_rate, momentum).minimize(loss)

  • adam 优化器

    引入模块

    import tensorflow as tf

    train_step=tf.train.AdamOptimizer(learning_rate).minimize(loss)

二、TensorFlow的mnist手写识别实例

  • mnist_forward.py

    import tensorflow as tf

    INPUT_NODE = 784
    OUTPUT_NODE = 10
    LAYER1_NODE = 500

    def get_weight(shape, regularizer):

    1. w = tf.Variable(tf.truncated_normal(shape,stddev=0.1))
    2. if regularizer != None: tf.add_to_collection('losses', tf.contrib.layers.l2_regularizer(regularizer)(w))
    3. return w
  1. def get_bias(shape):  
  2.     b = tf.Variable(tf.zeros(shape))  
  3.     return b
  5. def forward(x, regularizer):
  6.     w1 = get_weight([INPUT_NODE, LAYER1_NODE], regularizer)
  7.     b1 = get_bias([LAYER1_NODE])
  8.     y1 = tf.nn.relu(tf.matmul(x, w1) + b1)
  10.     w2 = get_weight([LAYER1_NODE, OUTPUT_NODE], regularizer)
  11.     b2 = get_bias([OUTPUT_NODE])
  12.     y = tf.matmul(y1, w2) + b2
  13.     return y

  • mnist_backward.py

    import tensorflow as tf
    from tensorflow.examples.tutorials.mnist import input_data
    import mnist_forward
    import os

    BATCH_SIZE = 200
    LEARNING_RATE_BASE = 0.1
    LEARNING_RATE_DECAY = 0.99
    REGULARIZER = 0.0001
    STEPS = 50000
    MOVING_AVERAGE_DECAY = 0.99
    MODEL_SAVE_PATH=”./model/“
    MODEL_NAME=”mnist_model”

  1. def backward(mnist):
  3.     x = tf.placeholder(tf.float32, [None, mnist_forward.INPUT_NODE])
  4.     y_ = tf.placeholder(tf.float32, [None, mnist_forward.OUTPUT_NODE])
  5.     y = mnist_forward.forward(x, REGULARIZER)
  6.     global_step = tf.Variable(0, trainable=False)    
  8.     ce = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=y, labels=tf.argmax(y_, 1))
  9.     cem = tf.reduce_mean(ce)
  10.     loss = cem + tf.add_n(tf.get_collection('losses'))
  12.     learning_rate = tf.train.exponential_decay(
  13.         LEARNING_RATE_BASE,
  14.         global_step,
  15.         mnist.train.num_examples / BATCH_SIZE, 
  16.         LEARNING_RATE_DECAY,
  17.         staircase=True)
  19.     train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss, global_step=global_step)
  21.     ema = tf.train.ExponentialMovingAverage(MOVING_AVERAGE_DECAY, global_step)
  22.     ema_op = ema.apply(tf.trainable_variables())
  23.     with tf.control_dependencies([train_step, ema_op]):
  24.         train_op = tf.no_op(name='train')
  26.     saver = tf.train.Saver()
  28.     with tf.Session() as sess:
  29.         init_op = tf.global_variables_initializer()
  30.         sess.run(init_op)
  32.         ckpt = tf.train.get_checkpoint_state(MODEL_SAVE_PATH)
  33.         if ckpt and ckpt.model_checkpoint_path:
  34.             saver.restore(sess, ckpt.model_checkpoint_path)
  36.         for i in range(STEPS):
  37.             xs, ys = mnist.train.next_batch(BATCH_SIZE)
  38.             _, loss_value, step = sess.run([train_op, loss, global_step], feed_dict={ x: xs, y_: ys})
  39.             if i % 1000 == 0:
  40.                 print("After %d training step(s), loss on training batch is %g." % (step, loss_value))
  41.                 saver.save(sess, os.path.join(MODEL_SAVE_PATH, MODEL_NAME), global_step=global_step)
  44. def main():
  45.     mnist = input_data.read_data_sets("./data/", one_hot=True)
  46.     backward(mnist)
  48. if __name__ == '__main__':
  49.     main()

  • mnist_test.py

    coding:utf-8

    import time
    import tensorflow as tf
    from tensorflow.examples.tutorials.mnist import input_data
    import mnist_forward
    import mnist_backward
    TEST_INTERVAL_SECS = 5

    def test(mnist):

    1. with tf.Graph().as_default() as g:
    2.     x = tf.placeholder(tf.float32, [None, mnist_forward.INPUT_NODE])
    3.     y_ = tf.placeholder(tf.float32, [None, mnist_forward.OUTPUT_NODE])
    4.     y = mnist_forward.forward(x, None)
    6.     ema = tf.train.ExponentialMovingAverage(mnist_backward.MOVING_AVERAGE_DECAY)
    7.     ema_restore = ema.variables_to_restore()
    8.     saver = tf.train.Saver(ema_restore)
    10.     correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
    11.     accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))        
    13.     while True:
    14.         with tf.Session() as sess:
    15.             ckpt = tf.train.get_checkpoint_state(mnist_backward.MODEL_SAVE_PATH)
    16.             if ckpt and ckpt.model_checkpoint_path:
    17.                 saver.restore(sess, ckpt.model_checkpoint_path)
    18.                 global_step = ckpt.model_checkpoint_path.split('/')[-1].split('-')[-1]
    19.                 accuracy_score = sess.run(accuracy, feed_dict={ x: mnist.test.images, y_: mnist.test.labels})
    20.                 print("After %s training step(s), test accuracy = %g" % (global_step, accuracy_score))
    21.             else:
    22.                 print('No checkpoint file found')
    23.                 return
    24.         time.sleep(TEST_INTERVAL_SECS)

    def main():

    1. mnist = input_data.read_data_sets("./data/", one_hot=True)
    2. test(mnist)

    if name == ‘main‘:

    1. main()

  • mnist_app.py

    coding:utf-8

    import tensorflow as tf
    import numpy as np
    from PIL import Image
    import mnist_backward
    import mnist_forward

    def restore_model(testPicArr):

    1. with tf.Graph().as_default() as tg:
    2.     x = tf.placeholder(tf.float32, [None, mnist_forward.INPUT_NODE])
    3.     y = mnist_forward.forward(x, None)
    4.     preValue = tf.argmax(y, 1)
    6.     variable_averages = tf.train.ExponentialMovingAverage(mnist_backward.MOVING_AVERAGE_DECAY)
    7.      variables_to_restore = variable_averages.variables_to_restore()
    8.      saver = tf.train.Saver(variables_to_restore)
    10.     with tf.Session() as sess:
    11.         ckpt = tf.train.get_checkpoint_state(mnist_backward.MODEL_SAVE_PATH)
    12.         if ckpt and ckpt.model_checkpoint_path:
    13.             saver.restore(sess, ckpt.model_checkpoint_path)
    15.             preValue = sess.run(preValue, feed_dict={ x:testPicArr})
    16.             return preValue
    17.         else:
    18.             print("No checkpoint file found")
    19.             return -1

    def pre_pic(picName):

    1. img = Image.open(picName)
    2. reIm = img.resize((28,28), Image.ANTIALIAS)
    3. im_arr = np.array(reIm.convert('L'))
    4. threshold = 50
    5. for i in range(28):
    6.     for j in range(28):
    7.         im_arr[i][j] = 255 - im_arr[i][j]
    8.          if (im_arr[i][j] < threshold):
    9.              im_arr[i][j] = 0
    10.         else: im_arr[i][j] = 255
    12. nm_arr = im_arr.reshape([1, 784])
    13. nm_arr = nm_arr.astype(np.float32)
    14. img_ready = np.multiply(nm_arr, 1.0/255.0)
    16. return img_ready

    def application():

    1. testNum = input("input the number of test pictures:")
    2. for i in range(testNum):
    3.     testPic = raw_input("the path of test picture:")
    4.     testPicArr = pre_pic(testPic)
    5.     preValue = restore_model(testPicArr)
    6.     print "The prediction number is:", preValue

    def main():

    1. application()

    if name == ‘main‘:

    1. main()

发表评论

表情：
评论列表 （有 0 条评论，489人围观）

还没有评论，来说两句吧...

相关阅读