各种激活函数

布满荆棘的人生 2023-01-10 14:58 200阅读 0赞

### 1、Sigmoid ###

![f(x)=\\tfrac\{1\}\{(1+e^\{-x\})\}][f_x_tfrac_1_1_e_-x]

### 2、Tanh ###

![f(x)=(1-exp(-2x))/(1+exp(-2x))][f_x_1-exp_-2x_1_exp_-2x]

### 3、Relu ###

![f(x)=max(0,x)][f_x_max_0_x]

### 4、LeakyRelu ###

![c7a7f587e29db58fa2c4b083b450ce9c.png][]

### 5、ReLu6 ###

![e01c1f70ffcd67b44de670ed013bdf96.png][]

### 6、ELU ###

![1b93bb27685553914f3b8ec605e9bc01.png][]

### 7、softplus ###

![c786f1075f8c75dbe29f3ea6d8ad80b4.png][]

### 8、swish ###

![f(x) = x\*sigmoid(x)][f_x_ _ x_sigmoid_x]

### 9、mish ###

![646984cc88ed3d77c525bac6e68675eb.png][]

详细内容参考：[https://www.cnblogs.com/missdx/p/12367486.html][https_www.cnblogs.com_missdx_p_12367486.html]

代码参考：[https://www.it610.com/article/1292023474222604288.htm][https_www.it610.com_article_1292023474222604288.htm]

[f_x_tfrac_1_1_e_-x]: https://private.codecogs.com/gif.latex?f%28x%29%3D%5Ctfrac%7B1%7D%7B%281&plus;e%5E%7B-x%7D%29%7D
[f_x_1-exp_-2x_1_exp_-2x]: https://private.codecogs.com/gif.latex?f%28x%29%3D%281-exp%28-2x%29%29/%281&plus;exp%28-2x%29%29
[f_x_max_0_x]: https://private.codecogs.com/gif.latex?f%28x%29%3Dmax%280%2Cx%29
[c7a7f587e29db58fa2c4b083b450ce9c.png]: https://img-blog.csdnimg.cn/img_convert/c7a7f587e29db58fa2c4b083b450ce9c.png
[e01c1f70ffcd67b44de670ed013bdf96.png]: https://img-blog.csdnimg.cn/img_convert/e01c1f70ffcd67b44de670ed013bdf96.png
[1b93bb27685553914f3b8ec605e9bc01.png]: https://img-blog.csdnimg.cn/img_convert/1b93bb27685553914f3b8ec605e9bc01.png
[c786f1075f8c75dbe29f3ea6d8ad80b4.png]: https://img-blog.csdnimg.cn/img_convert/c786f1075f8c75dbe29f3ea6d8ad80b4.png
[f_x_ _ x_sigmoid_x]: https://private.codecogs.com/gif.latex?f%28x%29%20%3D%20x*sigmoid%28x%29
[646984cc88ed3d77c525bac6e68675eb.png]: https://img-blog.csdnimg.cn/img_convert/646984cc88ed3d77c525bac6e68675eb.png
[https_www.cnblogs.com_missdx_p_12367486.html]: https://www.cnblogs.com/missdx/p/12367486.html
[https_www.it610.com_article_1292023474222604288.htm]: https://www.it610.com/article/1292023474222604288.htm