逻辑斯蒂回归的代价函数

叁歲伎倆 2021-12-16 09:25 263阅读 0赞

### Logistic回归的代价函数 ###

J ( θ ) = − 1 m \[ ∑ i = 1 m y ( i ) l o g   h θ ( x ( i ) ) + ( 1 − y ( i ) ) l o g ( 1 − h θ ( x ( i ) ) ) \] J(\\theta)=-\\frac\{1\}\{m\}\\left\[\\sum^m\_\{i=1\}y^\{(i)\}log\\ h\_\\theta(x^\{(i)\})+(1-y^\{(i)\})log(1-h\_\\theta(x^\{(i)\}))\\right\] J(θ)=−m1\[i=1∑my(i)log hθ(x(i))\+(1−y(i))log(1−hθ(x(i)))\]对代价函数求偏导：  
补充：  
 h θ ( x ( i ) ) = g ( x ( i ) θ ) g ( x ) = 1 1 + e − x g ′ ( x ) = g ( x ) ( 1 − g ( x ) ) ∂ h θ ( x ( i ) ) ∂ θ = h θ ′ ( x ( i ) ) ( x ( i ) ) T \\begin\{aligned\} &h\_\\theta(x^\{(i)\})=g(x^\{(i)\}\\theta)\\\\&g(x)=\\frac\{1\}\{1+e^\{-x\}\}\\\\ &g^\{&\#x27;\}(x)=g(x)(1-g(x))\\\\ &\\frac\{\\partial h\_\\theta(x^\{(i)\})\}\{\\partial \\theta\}=h^\{&\#x27;\}\_\\theta(x^\{(i)\})(x^\{(i)\})^T \\end\{aligned\} hθ(x(i))=g(x(i)θ)g(x)=1\+e−x1g′(x)=g(x)(1−g(x))∂θ∂hθ(x(i))=hθ′(x(i))(x(i))T  
 ∂ J ( θ ) ∂ θ = − 1 m \[ ∑ i = 1 m y ( i ) h θ ( x ( i ) ) − ( 1 − y ( i ) ) l o g ( 1 − h θ ( x ( i ) ) ) \] ∂ h θ ( x ( i ) ) ∂ θ = − 1 m \[ ∑ i = 1 m y ( i ) h θ ( x ( i ) ) − ( 1 − y ( i ) ) l o g ( 1 − h θ ( x ( i ) ) ) \] h θ ′ ( x ( i ) ) ( x ( i ) ) T = 1 m ∑ i = 1 m h θ ′ ( x ( i ) ) ( x ( i ) ) T h θ ( x ( i ) ) ( 1 − h θ ( x ( i ) ) ) ( h θ ( x ( i ) ) − y ( i ) ) = 1 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) ( x ( i ) ) T = 1 m X T ( h θ ( X ) − y ) \\begin\{aligned\} \\frac\{\\partial J(\\theta)\}\{\\partial \\theta\} &=-\\frac\{1\}\{m\}\\left\[\\sum^m\_\{i=1\}\\frac\{y^\{(i)\}\}\{h\_\\theta(x^\{(i)\})\}-\\frac\{(1-y^\{(i)\})\}\{log(1-h\_\\theta(x^\{(i)\}))\}\\right\]\\frac\{\\partial h\_\\theta(x^\{(i)\})\}\{\\partial \\theta\}\\\\ &=-\\frac\{1\}\{m\}\\left\[\\sum^m\_\{i=1\}\\frac\{y^\{(i)\}\}\{h\_\\theta(x^\{(i)\})\}-\\frac\{(1-y^\{(i)\})\}\{log(1-h\_\\theta(x^\{(i)\}))\}\\right\]h^\{&\#x27;\}\_\\theta(x^\{(i)\})(x^\{(i)\})^T \\\\&=\\frac\{1\}\{m\}\\sum^m\_\{i=1\}\\frac\{h^\{&\#x27;\}\_\\theta(x^\{(i)\})(x^\{(i)\})^T\}\{h\_\\theta(x^\{(i)\})(1-h\_\\theta(x^\{(i)\}))\}(h\_\\theta(x^\{(i)\})-y^\{(i)\})\\\\ &=\\frac\{1\}\{m\}\\sum^m\_\{i=1\}(h\_\\theta(x^\{(i)\})-y^\{(i)\})(x^\{(i)\})^T\\\\ &=\\frac\{1\}\{m\}X^T(h\_\\theta(X)-y) \\end\{aligned\} ∂θ∂J(θ)=−m1\[i=1∑mhθ(x(i))y(i)−log(1−hθ(x(i)))(1−y(i))\]∂θ∂hθ(x(i))=−m1\[i=1∑mhθ(x(i))y(i)−log(1−hθ(x(i)))(1−y(i))\]hθ′(x(i))(x(i))T=m1i=1∑mhθ(x(i))(1−hθ(x(i)))hθ′(x(i))(x(i))T(hθ(x(i))−y(i))=m1i=1∑m(hθ(x(i))−y(i))(x(i))T=m1XT(hθ(X)−y)请注意， x ( i ) x^\{(i)\} x(i)是数据条，是行向量； h θ ( X ) − y h\_\\theta(X)-y hθ(X)−y是列向量； θ \\theta θ是关于各属性权值的列向量。

#### 利用**Gradient Decent**求解最优值 θ \\theta θ ####

R e p e a t \{ θ j : = θ j − α ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) x j ( i ) \} s i m u l t a n e o u s l y   u p d a t e   a l l   θ j \\begin\{aligned\} Repeat\\\{&\\\\ &\\theta\_j:=\\theta\_j-\\alpha\\sum\_\{i=1\}^m(h\_\\theta(x^\{(i)\})-y^\{(i)\})x^\{(i)\}\_j\\\\ &\\\}simultaneously\\ update\\ all\\ \\theta\_j \\end\{aligned\} Repeat\{ θj:=θj−αi=1∑m(hθ(x(i))−y(i))xj(i)\}simultaneously update all θj  
你妈，被这些鬼东西搞了一下午，fuck！