caffe训练时，loss一般出现87.3365则保持不变

比眉伴天荒 2022-03-20 15:14 158阅读 0赞

caffe训练时，loss一般出现很奇怪的值：87.3365，则保持不变下去。直觉就有问题。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MTc3MDE2OQ_size_16_color_FFFFFF_t_70][]

搜索资料，找到多篇帖子：

[https://blog.csdn.net/u011215144/article/details/78479330][https_blog.csdn.net_u011215144_article_details_78479330]

[https://blog.csdn.net/cheese\_pop/article/details/79582077][https_blog.csdn.net_cheese_pop_article_details_79582077]

[https://blog.csdn.net/ying86615791/article/details/60757573][https_blog.csdn.net_ying86615791_article_details_60757573]

[https://blog.csdn.net/abcjennifer/article/details/46424949][https_blog.csdn.net_abcjennifer_article_details_46424949]

[https://blog.csdn.net/lyiiiiii61/article/details/79153014][https_blog.csdn.net_lyiiiiii61_article_details_79153014]

[https://blog.csdn.net/hyqsong/article/details/51933255][https_blog.csdn.net_hyqsong_article_details_51933255]

[https://blog.csdn.net/u010725283/article/details/78929684][https_blog.csdn.net_u010725283_article_details_78929684]  
[https://blog.csdn.net/u010417185/article/details/69664803][https_blog.csdn.net_u010417185_article_details_69664803]

对如上看到的帖子总结， 主要有以下几个方面需要关注：

**1、降低学习率，提高batchsize**

将学习率从0.01，降低到0.0000001，训练的batchsize现在是256

**2、降低weight**

跟学习率同步降低的

**3、输入归一化**

训练prototxt文件中，设置归一化参数：

transform\_param \{

scale: 0.00390625//像素归一化，1/255

**4、查看输入数据、标签是否有异常**

solver.prototxt中设置，debug\_info: true

标签仔细看了，从0到60，总分类个数也改成了61

数据都是严格按照预处理lmdb来的。

**5、数据shuffle**

已经在lmdb数据生成时进行了shuffle。

**6、batch normalization的设置**

[http://www.caffecn.cn/?/question/312][http_www.caffecn.cn_question_312]

[http://noahsnail.com/2018/07/05/2018-07-05-Caffe%E8%AE%AD%E7%BB%83%E6%97%B6Loss%E4%B8%8D%E4%B8%8B%E9%99%8D%E9%97%AE%E9%A2%98/][http_noahsnail.com_2018_07_05_2018-07-05-Caffe_E8_AE_AD_E7_BB_83_E6_97_B6Loss_E4_B8_8D_E4_B8_8B_E9_99_8D_E9_97_AE_E9_A2_98]

我这里碰到的问题，其实是通过第6点解决的。将batch normaliztion的use\_global\_stats由true改成false

batch\_norm\_param \{ use\_global\_stats: false \}

训练阶段是false，测试阶段是true。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MTc3MDE2OQ_size_16_color_FFFFFF_t_70]: /images/20220320/fdaada5da7e448edb38bfbf1c6655660.png
[https_blog.csdn.net_u011215144_article_details_78479330]: https://blog.csdn.net/u011215144/article/details/78479330
[https_blog.csdn.net_cheese_pop_article_details_79582077]: https://blog.csdn.net/cheese_pop/article/details/79582077
[https_blog.csdn.net_ying86615791_article_details_60757573]: https://blog.csdn.net/ying86615791/article/details/60757573
[https_blog.csdn.net_abcjennifer_article_details_46424949]: https://blog.csdn.net/abcjennifer/article/details/46424949
[https_blog.csdn.net_lyiiiiii61_article_details_79153014]: https://blog.csdn.net/lyiiiiii61/article/details/79153014
[https_blog.csdn.net_hyqsong_article_details_51933255]: https://blog.csdn.net/hyqsong/article/details/51933255
[https_blog.csdn.net_u010725283_article_details_78929684]: https://blog.csdn.net/u010725283/article/details/78929684
[https_blog.csdn.net_u010417185_article_details_69664803]: https://blog.csdn.net/u010417185/article/details/69664803
[http_www.caffecn.cn_question_312]: http://www.caffecn.cn/?/question/312
[http_noahsnail.com_2018_07_05_2018-07-05-Caffe_E8_AE_AD_E7_BB_83_E6_97_B6Loss_E4_B8_8D_E4_B8_8B_E9_99_8D_E9_97_AE_E9_A2_98]: http://noahsnail.com/2018/07/05/2018-07-05-Caffe%E8%AE%AD%E7%BB%83%E6%97%B6Loss%E4%B8%8D%E4%B8%8B%E9%99%8D%E9%97%AE%E9%A2%98/