02 Multivariate Linear Regression

痛定思痛。 2021-11-26 11:16 161阅读 0赞

1.  h(x)  
    \\\[  
    \\begin\{align\*\}h\_\\theta(x) =\\begin\{bmatrix\}\\theta\_0 \\hspace\{2em\} \\theta\_1 \\hspace\{2em\} ... \\hspace\{2em\} \\theta\_n\\end\{bmatrix\}\\begin\{bmatrix\}x\_0 \\newline x\_1 \\newline \\vdots \\newline x\_n\\end\{bmatrix\}= \\theta^T x\\end\{align\*\}, x\_0^\{(i)\} = 1  
    \\\]
2.  Gradient descent equation  
    \\\[  
    \\begin\{align\*\}& \\text\{repeat until convergence:\} \\; \\lbrace \\newline \\; & \\theta\_j := \\theta\_j - \\alpha \\frac\{1\}\{m\} \\sum\\limits\_\{i=1\}^\{m\} (h\_\\theta(x^\{(i)\}) - y^\{(i)\}) \\cdot x\_j^\{(i)\} \\; & \\text\{for j := 0...n\}\\newline \\rbrace\\end\{align\*\}  
    \\\]
3.  当不同特征的值差距过大\\((>10^5)\\)时，需要特征缩放(Feature Scaling)
    
    \\\[  
    x\_i := \\frac\{x\_i - \\mu\_i\}\{s\_i\}  
    \\\]  
    Where \\(\\mu\_i\\) is the average of all the values for feature(i) and \\(s\_i\\) is the range of values(max - min), or \\(s\_i\\) is the standard deviation.
4.  Learning Rate  
    In automatic convergence test, declare convergence if \\(J(\\theta)\\) decreases by less than \\(1-^\{-3\}\\) in one iteration.
5.  Features and Polynomial Regression  
    可以将不同的特征值组合来更好的拟合数据，同时因为数据的组合，更加需要特征缩放来加快几何提高精度
6.  Normal Equation 正规方程 不需要特征缩放  
    \\\[  
    \\theta = (X^TX)^\{-1\}X^Ty  
    \\\]
7.  Comparation
    
    <table> 
     <thead> 
      <tr> 
       <th style="text-align:center;">Gradient Descent</th> 
       <th style="text-align:center;">Normal Equation</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td style="text-align:center;">need to choose \(\alpha\)</td> 
       <td style="text-align:center;">No need to choose \(\alpha\)</td> 
      </tr> 
      <tr> 
       <td style="text-align:center;">Needs many iterations</td> 
       <td style="text-align:center;">Don’t need to iterate</td> 
      </tr> 
      <tr> 
       <td style="text-align:center;">Works well even when n is large (\(&gt;10^4\))</td> 
       <td style="text-align:center;">Need to compute \((X^TX)^{-1}\)</td> 
      </tr> 
      <tr> 
       <td style="text-align:center;">\(O(kn^2)\)</td> 
       <td style="text-align:center;">Slow if n is very large \(O(n^3)\)</td> 
      </tr> 
     </tbody> 
    </table>
8.  If \\(X^TX\\) is noninvertible, the common causes might be having :
    
     *  Redundant features, where two features are very closely related (i.e. they are linearly dependent)
     *  Too many features (e.g. m ≤ n). In this case, delete some features or use "regularization" (to be explained in a later lesson).

转载于:https://www.cnblogs.com/QQ-1615160629/p/02-Multivariate-Linear-Regression.html