机器学习入门-第七天

雨点打透心脏的1/2处 2023-03-13 11:53 20阅读 0赞

# 1, Deep Learning简介 #

### 1,什么是深度学习 ###

*  深度学习的model是一个深度神经网络结构（neural structure）
 *  深度学习的“深度”是指神经网络的隐层（hidden layer）数量足够多
 *  深度学习是自动提取特征（Feature extractor），不需要像逻辑回归那样特征转换（Feature engineering）
 *  每一个neuron里面的sigmoid function，在Deep Learning中被称为**activation function**(激励函数)
 *  有很多层layers的neural network，被称为**DNN(Deep Neural Network)**

### 2,[深度学习的三个步骤][Link 1] ###

*  Step1：构建神经网络（Neural network）---神经网络的创建包括3部分：一是神经网络有多少隐层（layer）、二是每一层有多少神经元（neuron）、三是每个神经元之间如何连接
 *  Step2：模型评估（Goodness of function）
 *  Step3：选择最优函数（Pick best function）

![format_png][]

**神经网络的概念:**把多个Logistic Regression前后connect在一起，然后把一个Logistic Regression称之为neuron，整个称之为neural network. 我们可以用不同的方法连接这些neuron，就可以得到不同的structure，neural network里的每一个Logistic Regression都有自己的weight和bias，这些weight和bias集合起来，就是这个network的parameter. 比较常用的一个连接方式是**Fully Connect Feedforward Network(全连接前馈网络),其概念**：前馈（feedforward）也可以称为前向，从信号流向来理解就是输入信号进入网络后，信号流动是单向的，即信号从前一层流向后一层，一直到输出层，其中任意两层之间的连接并没有反馈（feedback），亦即信号没有从后一层又返回到前一层。因为layer和layer之间，所有的neuron都是两两连接，所以它叫Fully connected的network.

![format_png 1][]![format_png 2][]

### 3,神经网络是如何工作的? ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70][]

每一层的计算都是一样的,每一层都用矩阵运算, 矩阵运算的好处是，可以用GPU加速，GPU对matrix的运算是比CPU要来的快的，所以我们写neural network的时候，习惯把它写成matrix operation，然后call GPU来加速它.

![format_png 3][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 1][]

**注：input、output的dimension，加上network structure，就可以确定一个model的形状，前两个是容易知道的，而决定这个network的structure则是整个Deep Learning中最为关键的步骤. **手写数字识别例子我们已知的constraint只有input是256维，output是10维，而**中间要有几个hidden layer，每个layer要有几个neuron，也即network的structure, 都是需要我们自己去设计的**，它们近乎是决定了function set长什么样子.

![format_png 4][]

### 4,如何评价神经网络的好坏(Goodness of function),即确定损失函数 ###

![format_png 5][]

### 5,找到一个最好的函数（最佳参数） ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 2][]

![format_png 6][]

### 6, Back Propagation方法update DNN参数 ###

**BP（后向传播）是用来快速计算DNN梯度的, 从而更快的更新参数.**

如何有效地把这个近百万维的vector给计算出来，这就是Backpropagation要做的事情，所以**Backpropagation并不是一个和gradient descent不同的training的方法，它就是gradient descent，它只是一个比较有效率的算法**，让你在计算大量数据的gradient的vector的时候更有效率.

1.  损失函数(Loss function)是定义在单个训练样本上的，也就是就算一个样本的误差，比如我们想要分类，就是预测的类别和实际类别的区别，是一个样本的，用L表示。
2.  代价函数(Cost function)是定义在整个训练集上面的，也就是所有样本的误差的总和的平均，也就是损失函数的总和的平均，有没有这个平均其实不会影响最后的参数的求解结果。
3.  总体损失函数(Total loss function)是定义在整个训练集上面的，也就是所有样本的误差的总和。也就是平时我们反向传播需要最小化的值。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 3][]

![format_png 7][]![format_png 8][]

**1,Forward pass: 就是权值所对应的输入量**

![format_png 9][]![format_png 10][]

**2,Backward pass: **

![20200513213149115.png][]

![format_png 11][]

![format_png 12][]

**[一文弄懂神经网络中的反向传播法][Link 2]** [https://www.cnblogs.com/charlotte77/p/5629865.html][Link 2]

**3,为什么可以反向传播?**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 4][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 5][]

![format_png 13][]![format_png 14][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 6][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 7][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 8][]

**参考:**

[https://www.cnblogs.com/xxlad/p/11237862.html][https_www.cnblogs.com_xxlad_p_11237862.html]

[https://sakura-gh.github.io/ML-notes/ML-notes-html/8\_Deep-Learning.html][https_sakura-gh.github.io_ML-notes_ML-notes-html_8_Deep-Learning.html]

“深度學習”

•My Course: Machine learning and having it deep and structured

•http://speech.ee.ntu.edu.tw/~tlkagk/courses\_MLSD15\_2.html

•6 hour version: http://www.slideshare.net/tw\_dsconf/ss-62245351

•“Neural Networks and Deep Learning”•written by Michael Nielsen•http://neuralnetworksanddeeplearning.com/

•“Deep Learning” •written by Yoshua Bengio, Ian J. Goodfellow and Aaron Courville •http://www.deeplearningbook.org

[Link 1]: https://datawhalechina.github.io/leeml-notes/#/chapter13/chapter13?id=%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%9A%84%E4%B8%89%E4%B8%AA%E6%AD%A5%E9%AA%A4
[format_png]: /images/20230312/4cd22475db4e40fbb69c1c012c682c74.png
[format_png 1]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTUyODk5Mi8yMDE5MDcvMTUyODk5Mi0yMDE5MDcyNDE2MDE1MDAzOC0xNjg2OTA0NDE1LnBuZw?x-oss-process=image/format,png
[format_png 2]: /images/20230312/925b1b393b954f439817f49deb194a64.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200513150726794.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[format_png 3]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTUyODk5Mi8yMDE5MDcvMTUyODk5Mi0yMDE5MDcyNDE2NDIyODkwMy03MzA5OTM2MDYucG5n?x-oss-process=image/format,png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200513151913724.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[format_png 4]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9kYXRhd2hhbGVjaGluYS5naXRodWIuaW8vbGVlbWwtbm90ZXMvY2hhcHRlcjEzL3Jlcy9jaGFwdGVyMTMtMTYucG5n?x-oss-process=image/format,png
[format_png 5]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2xvc3MtZm9yLWV4YW1wbGUucG5n?x-oss-process=image/format,png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20200513153253670.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[format_png 6]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2RsLWdyYWRpZW50LnBuZw?x-oss-process=image/format,png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/2020051321224287.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[format_png 7]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2JwLWxvc3MucG5n?x-oss-process=image/format,png
[format_png 8]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2JwLWZvcndhcmQtYmFja3dhcmQucG5n?x-oss-process=image/format,png
[format_png 9]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9kYXRhd2hhbGVjaGluYS5naXRodWIuaW8vbGVlbWwtbm90ZXMvY2hhcHRlcjE0L3Jlcy9jaGFwdGVyMTQtNi5wbmc?x-oss-process=image/format,png
[format_png 10]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2ZvcndhcmQtcGFzcy5wbmc?x-oss-process=image/format,png
[20200513213149115.png]: https://img-blog.csdnimg.cn/20200513213149115.png
[format_png 11]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2JhY2t3YXJkLXBhc3MucG5n?x-oss-process=image/format,png
[format_png 12]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9kYXRhd2hhbGVjaGluYS5naXRodWIuaW8vbGVlbWwtbm90ZXMvY2hhcHRlcjE0L3Jlcy9jaGFwdGVyMTQtMTAucG5n?x-oss-process=image/format,png
[Link 2]: https://www.cnblogs.com/charlotte77/p/5629865.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20200513214630225.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20200513224333470.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[format_png 13]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2JwLW91dHB1dC1sYXllci5wbmc?x-oss-process=image/format,png
[format_png 14]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vU2FrdXJhLWdoL01MLW5vdGVzL3Jhdy9tYXN0ZXIvaW1nL2JwLW5vdC1vdXRwdXQtbGF5ZXIucG5n?x-oss-process=image/format,png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20200513220532621.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20200513220605337.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20200513220638892.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA==,size_16,color_FFFFFF,t_70
[https_www.cnblogs.com_xxlad_p_11237862.html]: https://www.cnblogs.com/xxlad/p/11237862.html
[https_sakura-gh.github.io_ML-notes_ML-notes-html_8_Deep-Learning.html]: https://sakura-gh.github.io/ML-notes/ML-notes-html/8_Deep-Learning.html