梯度下降、反向传播

迈不过友情╰ 2023-07-03 12:22 3阅读 0赞

# ![20191009191333910.png][] #

# [日萌社][Link 1] #

[人工智能AI：Keras PyTorch MXNet TensorFlow PaddlePaddle 深度学习实战（不定时更新）][AI_Keras PyTorch MXNet TensorFlow PaddlePaddle]

--------------------

[nltk 处理文本][nltk]

[注意力机制 SENet、CBAM][SENet_CBAM]

[卷积神经网络 处理文本：word2vec、TF-IDF、TextRank、字符卷积、词卷积、卷积神经网络文本分类模型的实现(Conv1D一维卷积、Conv2D二维卷积)][_word2vec_TF-IDF_TextRank_Conv1D_Conv2D]

[反向传播、链式求导][Link 2]

[梯度下降][Link 3]

[最小二乘法(LS算法)：实际为L2范数的一个具体应用(计算残差平方和)][LS_L2]

[线性回归 例子][Link 4]

--------------------

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70][]![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 1][]![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 2][]![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 3][]![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 4][]![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 5][]

【程序3-2】
    x = [(2, 0, 3), (1, 0, 3), (1, 1, 3), (1,4, 2), (1, 2, 4)]
    y = [5, 6, 8, 10, 11]
    epsilon = 0.002 #噪音值
    alpha = 0.02 #学习率
    diff = [0, 0]
    max_itor = 1000
    error0 = 0
    error1 = 0
    cnt = 0
    m = len(x)
    theta0 = 0 #θ权重参数
    theta1 = 0
    theta2 = 0
    
    while True:
        cnt += 1
        for i in range(m):
            diff[0] = (theta0 * x[i][0] + theta1 * x[i][1] + theta2 * x[i][2]) - y[i]
            theta0 -= alpha * diff[0] * x[i][0]
            theta1 -= alpha * diff[0] * x[i][1]
            theta2 -= alpha * diff[0] * x[i][2]
        error1 = 0
        for lp in range(len(x)):
            error1 += (y[lp] - (theta0 * x[lp][0] + theta1 * x[lp][1] + theta2 * x[lp][2])) ** 2 / 2
        if abs(error1 - error0) < epsilon: 
            break
        else:
            error0 = error1
    
    #theta0 : -0.510496, theta1 : 1.542565, theta2 : 2.140431, error0 : 0.246543, error1 : 0.244565
    print('theta0 : %f, theta1 : %f, theta2 : %f, error0 : %f, error1 : %f' % (theta0, theta1, theta2, error0, error1))
    #Done: theta0 : -0.510496, theta1 : 1.542565, theta2 : 2.140431
    print('Done: theta0 : %f, theta1 : %f, theta2 : %f' % (theta0, theta1, theta2)) 
    print('迭代次数: %d' % cnt) #迭代次数: 77
    
    #最终比较计算的值[5.400301,5.910797,7.453362,9.940628,11.136359] 和 y的值[5, 6, 8, 10, 11] 之间的相近性
    for i in range(m):
        result = theta0 * x[i][0] + theta1 * x[i][1] + theta2 * x[i][2]
        print('%f' % result)
    #5.400301
    #5.910797
    #7.453362
    #9.940628
    #11.136359

[20191009191333910.png]: https://img-blog.csdnimg.cn/20191009191333910.png
[Link 1]: http://www.rimengshe.com/
[AI_Keras PyTorch MXNet TensorFlow PaddlePaddle]: https://blog.csdn.net/zimiao552147572/article/details/88867161
[nltk]: https://azusa.blog.csdn.net/article/details/104203185
[SENet_CBAM]: https://azusa.blog.csdn.net/article/details/104201845
[_word2vec_TF-IDF_TextRank_Conv1D_Conv2D]: https://azusa.blog.csdn.net/article/details/104208463
[Link 2]: https://azusa.blog.csdn.net/article/details/104201797
[Link 3]: https://azusa.blog.csdn.net/article/details/104201755
[LS_L2]: https://azusa.blog.csdn.net/article/details/104201735
[Link 4]: https://azusa.blog.csdn.net/article/details/104201717
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200206202828401.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200206202834380.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20200206202835666.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20200206202840806.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20200206202844812.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20200206202848609.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9henVzYS5ibG9nLmNzZG4ubmV0,size_16,color_FFFFFF,t_70