【LR线性回归算法+梯度下降算法】机器学习公式推导计算+详细过程（入门必备）

- 日理万妓 2023-02-23 05:22 2阅读 0赞

**线性回归算法**

寻找一条直线，最大程度的”拟合“样本特征和样本标签的关系。

**公式**

h θ ( x ) = θ 0 x 0 + θ 1 x 1 + θ 2 x 2 + . . . + θ n x n h\_\\theta(x) = \\theta\_0x\_0 + \\theta\_1x\_1 + \\theta\_2x\_2 + ... + \\theta\_nx\_n hθ(x)=θ0x0\+θ1x1\+θ2x2\+...\+θnxn

上式为基本形式。

h θ ( x ) = θ T x h\_\\theta(x) = \\theta^Tx hθ(x)=θTx  
上式为向量形式。

**损失函数**  
 J ( θ ) = 1 2 m ∑ i = 0 m ( h θ ( x i ) − y i ) 2 J(\\theta) = \\frac\{1\}\{2m\} \\sum\_\{i=0\} ^ m (h\_\{\\theta\} (x ^ \{i\}) - y ^ \{i\}) ^ 2 J(θ)=2m1i=0∑m(hθ(xi)−yi)2  
其中：  
 y ( i ) = θ 0 x 0 i + θ 1 x 1 i + θ 2 x 2 i + . . . + θ n x n i y^\{(i)\} = \\theta\_0x^\{i\}\_0 + \\theta\_1x^\{i\}\_1 + \\theta\_2x^\{i\}\_2 + ... + \\theta\_nx^\{i\}\_n y(i)=θ0x0i\+θ1x1i\+θ2x2i\+...\+θnxni

**对损失函数求导**

*对 θ \\theta θ求偏导*  
 ∂ J ( θ ) ∂ θ = 2 ∑ i = 1 m ( y i − θ x i − θ 0 ) ∗ ( − x i ) \\frac\{\\partial J (\\theta)\}\{\\partial \\theta\} = 2\\sum\_\{i=1\} ^ m (y^\{i\} - \\theta x^\{i\} - \\theta\_0) \* (-x^\{i\}) ∂θ∂J(θ)=2i=1∑m(yi−θxi−θ0)∗(−xi)  
*对 θ 0 \\theta\_0 θ0求偏导*  
 ∂ J ( θ ) ∂ θ 0 = 2 ∑ i = 1 m ( y i − θ x i − θ 0 ) ∗ ( − 1 ) \\frac\{\\partial J (\\theta)\}\{\\partial \\theta\_0\} = 2\\sum\_\{i=1\} ^ m (y^\{i\} - \\theta x^\{i\} - \\theta\_0) \* (-1) ∂θ0∂J(θ)=2i=1∑m(yi−θxi−θ0)∗(−1)

**梯度下降公式**  
 θ j : = θ j − α ∂ ∂ θ j J ( θ ) \\theta\_j := \\theta\_j - \\alpha \\frac\{\\partial\}\{\\partial \\theta\_j\} J (\\theta) θj:=θj−α∂θj∂J(θ)

∂ ∂ θ j J ( θ ) = α ∂ ∂ θ j 1 2 m ∑ i = 0 m ( h θ ( x i ) − y i ) 2 \\frac\{\\partial \}\{\\partial \\theta\_j\} J (\\theta) = \\alpha \\frac\{\\partial \}\{\\partial \\theta\_j\}\\frac\{1\}\{2m\} \\sum\_\{i=0\} ^ m \{(h\_\\theta(x^\{i\}) - y^\{i\})\}^2 ∂θj∂J(θ)=α∂θj∂2m1i=0∑m(hθ(xi)−yi)2

= α 1 m ∑ i = 0 m ( h θ ( x i ) − y i ) ∗ x i = \\alpha \\frac\{1\}\{m\} \\sum\_\{i=0\} ^ m \{(h\_\\theta(x^\{i\}) - y^\{i\})\} \* x^\{i\} =αm1i=0∑m(hθ(xi)−yi)∗xi

*  式中 α \\alpha α表示学习率

**梯度下降公式推导**  
 θ j : = θ j − α ∂ ∂ θ j J ( θ ) \\theta\_j := \\theta\_j - \\alpha \\frac\{\\partial\}\{\\partial \\theta\_j\} J (\\theta) θj:=θj−α∂θj∂J(θ)

∂ ∂ θ j J ( θ ) = ∂ ∂ θ j 1 2 m ∑ i = 0 m ( h θ ( x i ) − y i ) 2 \\frac\{\\partial \}\{\\partial \\theta\_j\} J (\\theta) = \\frac\{\\partial \}\{\\partial \\theta\_j\}\\frac\{1\}\{2m\} \\sum\_\{i=0\} ^ m \{(h\_\\theta(x^\{i\}) - y^\{i\})\}^2 ∂θj∂J(θ)=∂θj∂2m1i=0∑m(hθ(xi)−yi)2

= 1 2 m ∑ i = 0 m ( ∂ ∂ θ j ( h θ ( x i ) − y i ) 2 ) = \\frac\{1\}\{2m\} \\sum\_\{i=0\} ^ m (\\frac\{\\partial \}\{\\partial \\theta\_j\} (h\_\\theta(x^\{i\}) - y^\{i\})^2) =2m1i=0∑m(∂θj∂(hθ(xi)−yi)2)

--------------------

*  连式法则：

z = f ( y ) z = f(y) z=f(y)

y = g ( x ) y = g(x) y=g(x)

z = f ( g ( x ) ) z = f(g(x)) z=f(g(x))

*  对 $ z $求导  
     ( z ) ′ = ( f ( g ( x ) ) ) ′ ∗ ( g ( x ) ) ′ (z)' = (f(g(x)))' \* (g(x))' (z)′=(f(g(x)))′∗(g(x))′

--------------------

= 1 2 m ∑ i = 0 m ( ∂ ∂ θ j ( h θ ( x i ) − y i ) 2 ) ∗ ( ∂ ∂ θ j ( h θ ( x i ) − y i ) ) = \\frac\{1\}\{2m\} \\sum\_\{i=0\} ^ m (\\frac\{\\partial \}\{\\partial \\theta\_j\} (h\_\\theta(x^\{i\}) - y^\{i\})^2)\* (\\frac\{\\partial \}\{\\partial \\theta\_j\}(h\_\\theta(x^\{i\}) - y^\{i\})) =2m1i=0∑m(∂θj∂(hθ(xi)−yi)2)∗(∂θj∂(hθ(xi)−yi))

*  求幂导

= 1 2 m ∑ i = 0 m 2 ∗ ( ( h θ ( x i ) − y i ) ) ∗ ( ∂ ∂ θ i ( ∑ i = 0 n θ i x i − y i ) ) = \\frac\{1\}\{2m\} \\sum\_\{i=0\} ^ m 2 \* ((h\_\\theta(x^\{i\}) - y^\{i\})) \* (\\frac\{\\partial \}\{\\partial \\theta\_i\}(\\sum\_\{i=0\} ^ n \\theta\_i x\_i - y^\{i\})) =2m1i=0∑m2∗((hθ(xi)−yi))∗(∂θi∂(i=0∑nθixi−yi))

= 1 m ∑ i = 0 m ( ( h θ ( x i ) − y i ) ) ∗ ( ∑ i = 0 n ( ∂ ∂ θ i θ i x i − ∂ ∂ θ i y i ) = \\frac\{1\}\{m\} \\sum\_\{i=0\} ^ m ((h\_\\theta(x^\{i\}) - y^\{i\})) \* (\\sum\_\{i=0\} ^ n (\\frac\{\\partial \}\{\\partial \\theta\_i\} \\theta\_i x^\{i\} - \\frac\{\\partial \}\{\\partial \\theta\_i\}y\_i) =m1i=0∑m((hθ(xi)−yi))∗(i=0∑n(∂θi∂θixi−∂θi∂yi)

= 1 m ∑ i = 0 m ( h θ ( x i ) − y i ) ∗ x i = \\frac\{1\}\{m\} \\sum\_\{i=0\} ^ m \{(h\_\\theta(x^\{i\}) - y^\{i\})\} \* x^\{i\} =m1i=0∑m(hθ(xi)−yi)∗xi

**算法原理**

*  将需要预测的数据 ”喂给“模型，计算得到预测值  y i y^\{i\} yi 。
 *  预测值和真实值相减，使得两者之间的误差尽可能的小。
 *  定义损失函数。对损失函数求偏导，求极值 。
 *  梯度下降，迭代去更新参数 θ j \\theta\_j θj 。
 *  直到  θ \\theta θ值达到我们预设的阈值，那么迭代停止 。

> > > #### 欢迎大家交流学习，任何问题都可以留言 ####