使用tensorflow框架在Colab上跑通猫狗识别代码

约定不等于承诺〃 2022-12-31 06:28 186阅读 0赞

更多编程教程请到：[菜鸟教程][Link 1] https://www.piaodoo.com/

友情链接：

[高州阳光论坛][Link 2]https://www.hnthzk.com/

[人人影视][Link 3]http://www.sfkyty.com/

**一、 前提：**

有Google账号（具体怎么注册账号这里不详述，大家都懂的，自行百度）在你的Google邮箱中关联好colab（怎样在Google邮箱中使用colab在此不详述，自行百度）

**二、 现在开始：**

因为我们使用的是colab，所以就不必为安装版本对应的anaconda、python以及tensorflow尔苦恼了，经过以下配置就可以直接开始使用了。

![Image 1][]

在colab中新建代码块，运行以下代码来下载需要的数据集

# In this exercise you will train a CNN on the FULL Cats-v-dogs dataset
    # This will require you doing a lot of data preprocessing because
    # the dataset isn't split into training and validation for you
    # This code block has all the required inputs
    import os
    import zipfile
    import random
    import tensorflow as tf
    from tensorflow.keras.optimizers import RMSprop
    from tensorflow.keras.preprocessing.image import ImageDataGenerator
    from shutil import copyfile
    # This code block downloads the full Cats-v-Dogs dataset and stores it as 
    # cats-and-dogs.zip. It then unzips it to /tmp
    # which will create a tmp/PetImages directory containing subdirectories
    # called 'Cat' and 'Dog' (that's how the original researchers structured it)
    # If the URL doesn't work, 
    # .  visit https://www.microsoft.com/en-us/download/confirmation.aspx?id=54765
    # And right click on the 'Download Manually' link to get a new URL

!wget --no-check-certificate  
“https://github.com/ADlead/Dogs-Cats/archive/master.zip”  
\-O “/tmp/cats-and-dogs.zip”

local\_zip = ‘/tmp/cats-and-dogs.zip’  
zip\_ref = zipfile.ZipFile(local\_zip, ‘r’)  
zip\_ref.extractall(’/tmp’)  
zip\_ref.close()

运行结果：

在colab中默认安装TensorFlow1.14，所以会提示让升级tensorflow，可以不用理会，需要升级为2.0的也可以自行百度去升级。  
接下来会提示我们需要的数据集以压缩包的形式已经下载好了

![Image 1][]

运行以下代码来解压下载好的数据集并把训练图像集划分成训练图像集和测试图像集，分别用于训练模型和测试模型。把25000张图像划分成20000张训练图像和5000张测试图像。深度学习的框架使用的是tensorflow，为了能让tensorflow分批输入数据进行训练，把所有的图像像素信息存储成batch文件。训练集100个batch文件，每个文件有200张图像。测试集1个batch文件，共5000张图像。

import cv2 as cv
    import os
    import numpy as np

import random  
import pickle

import time

start\_time = time.time()

data\_dir = ‘/tmp/Dogs-Cats-master/data’  
batch\_save\_path = ‘/tmp/Dogs-Cats-master/batch\_files’

# 创建batch文件存储的文件夹 #

os.makedirs(batch\_save\_path, exist\_ok=True)

# 图片统一大小：100 \* 100 #

# 训练集 20000：100个batch文件，每个文件200张图片 #

# 验证集 5000： 一个测试文件，测试时 50张 x 100 批次 #

# 进入图片数据的目录，读取图片信息 #

all\_data\_files = os.listdir(os.path.join(data\_dir, ‘train/’))

# print(all\_data\_files) #

# 打算数据的顺序 #

random.shuffle(all\_data\_files)

all\_train\_files = all\_data\_files\[:20000\]  
all\_test\_files = all\_data\_files\[20000:\]

train\_data = \[\]  
train\_label = \[\]  
train\_filenames = \[\]

test\_data = \[\]  
test\_label = \[\]  
test\_filenames = \[\]

# 训练集 #

for each in all\_train\_files:  
img = cv.imread(os.path.join(data\_dir,‘train/’,each),1)  
resized\_img = cv.resize(img, (100,100))

img\_data = np.array(resized\_img)  
train\_data.append(img\_data)  
if ‘cat’ in each:  
train\_label.append(0)  
elif ‘dog’ in each:  
train\_label.append(1)  
else:  
raise Exception(’%s is wrong train file’%(each))  
train\_filenames.append(each)

# 测试集 #

for each in all\_test\_files:  
img = cv.imread(os.path.join(data\_dir,‘train/’,each), 1)  
resized\_img = cv.resize(img, (100,100))

img\_data = np.array(resized\_img)  
test\_data.append(img\_data)  
if ‘cat’ in each:  
test\_label.append(0)  
elif ‘dog’ in each:  
test\_label.append(1)  
else:  
raise Exception(’%s is wrong test file’%(each))  
test\_filenames.append(each)

print(len(train\_data), len(test\_data))

# 制作100个batch文件 #

start = 0  
end = 200  
for num in range(1, 101):  
batch\_data = train\_data\[start: end\]  
batch\_label = train\_label\[start: end\]  
batch\_filenames = train\_filenames\[start: end\]  
batch\_name = ‘training batch \{\} of 15’.format(num)

all\_data = \{  
‘data’:batch\_data,  
‘label’:batch\_label,  
‘filenames’:batch\_filenames,  
‘name’:batch\_name  
\}

with open(os.path.join(batch\_save\_path, ‘train\_batch\_\{\}’.format(num)), ‘wb’) as f:  
pickle.dump(all\_data, f)

start += 200  
end += 200

# 制作测试文件 #

all\_test\_data = \{  
‘data’:test\_data,  
‘label’:test\_label,  
‘filenames’:test\_filenames,  
‘name’:‘test batch 1 of 1’  
\}

with open(os.path.join(batch\_save\_path, ‘test\_batch’), ‘wb’) as f:  
pickle.dump(all\_test\_data, f)

end\_time = time.time()  
print(‘制作结束, 用时\{\}秒’.format(end\_time - start\_time))

运行结果：

![Image 1][]

运行以下编写卷积层、池化层、全连接层、搭建tensorflow的计算图、定义占位符、计算损失函数、预测值、准确率以及训练部分的代码

import tensorflow as tf
    import numpy as np
    import cv2 as cv
    import os
    import pickle

‘’’ 全局参数 ‘’’  
IMAGE\_SIZE = 100  
LEARNING\_RATE = 1e-4  
TRAIN\_STEP = 10000  
TRAIN\_SIZE = 100  
TEST\_STEP = 100  
TEST\_SIZE = 50

IS\_TRAIN = True

SAVE\_PATH = ‘/tmp/Dogs-Cats-master/model/’

data\_dir = ‘/tmp/Dogs-Cats-master/batch\_files’  
pic\_path = ‘/tmp/Dogs-Cats-master/data/test1’

‘’’’’’

def load\_data(filename):  
‘’‘从batch文件中读取图片信息’’’  
with open(filename, ‘rb’) as f:  
data = pickle.load(f, encoding=‘iso-8859-1’)  
return data\[‘data’\],data\[‘label’\],data\[‘filenames’\]

# 读取数据的类 #

class InputData:  
def **init**(self, filenames, need\_shuffle):  
all\_data = \[\]  
all\_labels = \[\]  
all\_names = \[\]  
for file in filenames:  
data, labels, filename = load\_data(file)

all_data.append(data)
      all_labels.append(labels)
      all_names += filename
    
    self._data = np.vstack(all_data)
    self._labels = np.hstack(all_labels)
    print(self._data.shape)
    print(self._labels.shape)
    
    self._filenames = all_names
    
    self._num_examples = self._data.shape[0]
    self._need_shuffle = need_shuffle
    self._indicator = 0
    if self._indicator:
      self._shuffle_data()

def \_shuffle\_data(self):  
\# 把数据再混排  
p = np.random.permutation(self.\_num\_examples)  
self.\_data = self.\_data\[p\]  
self.\_labels = self.\_labels\[p\]

def next\_batch(self, batch\_size):  
‘’‘返回每一批次的数据’’’  
end\_indicator = self.\_indicator + batch\_size  
if end\_indicator > self.\_num\_examples:  
if self.\_need\_shuffle:  
self.\_shuffle\_data()  
self.\_indicator = 0  
end\_indicator = batch\_size  
else:  
raise Exception(‘have no more examples’)  
if end\_indicator > self.\_num\_examples:  
raise Exception(‘batch size is larger than all examples’)  
batch\_data = self.\_data\[self.\_indicator : end\_indicator\]  
batch\_labels = self.\_labels\[self.\_indicator : end\_indicator\]  
batch\_filenames = self.\_filenames\[self.\_indicator : end\_indicator\]  
self.\_indicator = end\_indicator  
return batch\_data, batch\_labels, batch\_filenames

# 定义一个类 #

class MyTensor:  
def **init**(self):

# 载入训练集和测试集
    train_filenames = [os.path.join(data_dir, 'train_batch_%d'%i) for i in range(1, 101)]
    test_filenames = [os.path.join(data_dir, 'test_batch')]
    self.batch_train_data = InputData(train_filenames, True)
    self.batch_test_data = InputData(test_filenames, True)
    
    pass

def flow(self):  
self.x = tf.placeholder(tf.float32, \[None, IMAGE\_SIZE, IMAGE\_SIZE, 3\], ‘input\_data’)  
self.y = tf.placeholder(tf.int64, \[None\], ‘output\_data’)  
self.keep\_prob = tf.placeholder(tf.float32)

# self.x = self.x / 255.0 需不需要这一步？
    
    # 图片输入网络中
    fc = self.conv_net(self.x, self.keep_prob)
    
    self.loss = tf.losses.sparse_softmax_cross_entropy(labels=self.y, logits=fc)
    self.y_ = tf.nn.softmax(fc) # 计算每一类的概率
    self.predict = tf.argmax(fc, 1)
    self.acc = tf.reduce_mean(tf.cast(tf.equal(self.predict, self.y), tf.float32))
    
    self.train_op = tf.train.AdamOptimizer(LEARNING_RATE).minimize(self.loss)
    self.saver = tf.train.Saver(max_to_keep=1)
    
    print('计算流图已经搭建.')

# 训练 #

def myTrain(self):  
acc\_list = \[\]  
with tf.Session() as sess:  
sess.run(tf.global\_variables\_initializer())

for i in range(TRAIN_STEP):
        train_data, train_label, _ = self.batch_train_data.next_batch(TRAIN_SIZE)
    
        eval_ops = [self.loss, self.acc, self.train_op]
        eval_ops_results = sess.run(eval_ops, feed_dict={
          self.x:train_data,
          self.y:train_label,
          self.keep_prob:0.7
        })
        loss_val, train_acc = eval_ops_results[0:2]
    
        acc_list.append(train_acc)
        if (i+1) % 100 == 0:
          acc_mean = np.mean(acc_list)
          print('step:{0},loss:{1:.5},acc:{2:.5},acc_mean:{3:.5}'.format(
            i+1,loss_val,train_acc,acc_mean
          ))
        if (i+1) % 1000 == 0:
          test_acc_list = []
          for j in range(TEST_STEP):
            test_data, test_label, _ = self.batch_test_data.next_batch(TRAIN_SIZE)
            acc_val = sess.run([self.acc],feed_dict={
              self.x:test_data,
              self.y:test_label,
              self.keep_prob:1.0
            })
            test_acc_list.append(acc_val)
          print('[Test ] step:{0}, mean_acc:{1:.5}'.format(
            i+1, np.mean(test_acc_list)
          ))
      # 保存训练后的模型
      os.makedirs(SAVE_PATH, exist_ok=True)
      self.saver.save(sess, SAVE_PATH + 'my_model.ckpt')

def myTest(self):  
with tf.Session() as sess:  
model\_file = tf.train.latest\_checkpoint(SAVE\_PATH)  
model = self.saver.restore(sess, save\_path=model\_file)  
test\_acc\_list = \[\]  
predict\_list = \[\]  
for j in range(TEST\_STEP):  
test\_data, test\_label, test\_name = self.batch\_test\_data.next\_batch(TEST\_SIZE)  
for each\_data, each\_label, each\_name in zip(test\_data, test\_label, test\_name):  
acc\_val, y\_\_, pre, test\_img\_data = sess.run(  
\[self.acc, self.y\_, self.predict, self.x\],  
feed\_dict=\{  
self.x:each\_data.reshape(1, IMAGE\_SIZE, IMAGE\_SIZE, 3),  
self.y:each\_label.reshape(1),  
self.keep\_prob:1.0  
\}  
)  
predict\_list.append(pre\[0\])  
test\_acc\_list.append(acc\_val)

# 把测试结果显示出来
          self.compare_test(test_img_data, each_label, pre[0], y__[0], each_name)
      print('[Test ] mean_acc:{0:.5}'.format(np.mean(test_acc_list)))

def compare\_test(self, input\_image\_arr, input\_label, output, probability, img\_name):  
classes = \[‘cat’, ‘dog’\]  
if input\_label == output:  
result = ‘正确’  
else:  
result = ‘错误’  
print(‘测试【\{0\}】,输入的label:\{1\}, 预测得是\{2\}:\{3\}的概率:\{4:.5\}, 输入的图片名称:\{5\}’.format(  
result,input\_label, output,classes\[output\], probability\[output\], img\_name  
))

def conv\_net(self, x, keep\_prob):  
conv1\_1 = tf.layers.conv2d(x, 16, (3, 3), padding=‘same’, activation=tf.nn.relu, name=‘conv1\_1’)  
conv1\_2 = tf.layers.conv2d(conv1\_1, 16, (3, 3), padding=‘same’, activation=tf.nn.relu, name=‘conv1\_2’)  
pool1 = tf.layers.max\_pooling2d(conv1\_2, (2, 2), (2, 2), name=‘pool1’)

conv2_1 = tf.layers.conv2d(pool1, 32, (3, 3), padding='same', activation=tf.nn.relu, name='conv2_1')
    conv2_2 = tf.layers.conv2d(conv2_1, 32, (3, 3), padding='same', activation=tf.nn.relu, name='conv2_2')
    pool2 = tf.layers.max_pooling2d(conv2_2, (2, 2), (2, 2), name='pool2')
    
    conv3_1 = tf.layers.conv2d(pool2, 64, (3, 3), padding='same', activation=tf.nn.relu, name='conv3_1')
    conv3_2 = tf.layers.conv2d(conv3_1, 64, (3, 3), padding='same', activation=tf.nn.relu, name='conv3_2')
    pool3 = tf.layers.max_pooling2d(conv3_2, (2, 2), (2, 2), name='pool3')
    
    conv4_1 = tf.layers.conv2d(pool3, 128, (3, 3), padding='same', activation=tf.nn.relu, name='conv4_1')
    conv4_2 = tf.layers.conv2d(conv4_1, 128, (3, 3), padding='same', activation=tf.nn.relu, name='conv4_2')
    pool4 = tf.layers.max_pooling2d(conv4_2, (2, 2), (2, 2), name='pool4')
    
    flatten = tf.layers.flatten(pool4) # 把网络展平，以输入到后面的全连接层
    
    fc1 = tf.layers.dense(flatten, 512, tf.nn.relu)
    fc1_dropout = tf.nn.dropout(fc1, keep_prob=keep_prob)
    fc2 = tf.layers.dense(fc1, 256, tf.nn.relu)
    fc2_dropout = tf.nn.dropout(fc2, keep_prob=keep_prob)
    fc3 = tf.layers.dense(fc2, 2, None) # 得到输出fc3
    
    return fc3

def main(self):  
self.flow()  
if IS\_TRAIN is True:  
self.myTrain()  
else:  
self.myTest()

def final\_classify(self):  
all\_test\_files\_dir = ‘./data/test1’  
all\_test\_filenames = os.listdir(all\_test\_files\_dir)  
if IS\_TRAIN is False:  
self.flow()  
\# self.classify()  
with tf.Session() as sess:  
model\_file = tf.train.latest\_checkpoint(SAVE\_PATH)  
mpdel = self.saver.restore(sess,save\_path=model\_file)

predict_list = []
        for each_filename in all_test_filenames:
          each_data = self.get_img_data(os.path.join(all_test_files_dir,each_filename))
          y__, pre, test_img_data = sess.run(
            [self.y_, self.predict, self.x],
            feed_dict={
              self.x:each_data.reshape(1, IMAGE_SIZE, IMAGE_SIZE, 3),
              self.keep_prob: 1.0
            }
          )
          predict_list.append(pre[0])
          self.classify(test_img_data, pre[0], y__[0], each_filename)
    
    else:
      print('now is training model...')

def classify(self, input\_image\_arr, output, probability, img\_name):  
classes = \[‘cat’,‘dog’\]  
single\_image = input\_image\_arr\[0\] \#\* 255  
if output == 0:  
output\_dir = ‘cat/’  
else:  
output\_dir = ‘dog/’  
os.makedirs(os.path.join(’./classiedResult’, output\_dir), exist\_ok=True)  
cv.imwrite(os.path.join(’./classiedResult’,output\_dir, img\_name),single\_image)  
print(‘输入的图片名称:\{0\}，预测得有\{1:5\}的概率是\{2\}:\{3\}’.format(  
img\_name,  
probability\[output\],  
output,  
classes\[output\]  
))

# 根据名称获取图片像素 #

def get\_img\_data(self,img\_name):  
img = cv.imread(img\_name)  
resized\_img = cv.resize(img, (100, 100))  
img\_data = np.array(resized\_img)

return img_data

if **name** == ‘**main**’:

mytensor = MyTensor()  
mytensor.main() \# 用于训练或测试

# mytensor.final\_classify() \# 用于最后的分类 #

print(‘hello world’)

运行结果：

![Image 1][]

参考：[https://www.jianshu.com/p/9ee2533c8adb][https_www.jianshu.com_p_9ee2533c8adb]

代码出处：[https://github.com/ADlead/Dogs-Cats.git][https_github.com_ADlead_Dogs-Cats.git]

到此这篇关于使用tensorflow框架在Colab上跑通猫狗识别代码的文章就介绍到这了,更多相关tensorflow框架在Colab上跑通猫狗识别内容请搜索菜鸟教程www.piaodoo.com以前的文章或继续浏览下面的相关文章希望大家以后多多支持菜鸟教程www.piaodoo.com！

[Link 1]: https://www.piaodoo.com/
[Link 2]: https://www.hnthzk.com/
[Link 3]: http://www.sfkyty.com/
[Image 1]: 
[https_www.jianshu.com_p_9ee2533c8adb]: https://www.jianshu.com/p/9ee2533c8adb
[https_github.com_ADlead_Dogs-Cats.git]: https://github.com/ADlead/Dogs-Cats.git