訓練集：\[\left\{ {\left( {{x^{\left( 1 \right)}},{y^{\left( 1 \right)}}} \right),\left( {{x^{\left( 2 \right)}},{y^{\left( 2 \right)}}} \right),...,\left( {{x^{\left( m \right)}},{y^{\left( m \right)}}} \right)} \right\}\]
\[x \in \left[ {\begin{array}{*{20}{c}}
{{x_0}}\\
{{x_2}}\\
\begin{array}{l}
.\\
.
\end{array}\\
{{x_n}}
\end{array}} \right]{x_0} = 1,y \in \left\{ {0,1} \right\}\]
\[{h_\theta }(x) = \frac{1}{{1 + {e^{ - {\theta ^T}x}}}}\]
如何選取參數θ？
損失函數
對於線性回歸：\[J\left( \theta \right) = \frac{1}{m}\sum\limits_{i = 1}^m {\frac{1}{2}{{\left( {{h_\theta }\left( {{x^{\left( i \right)}}} \right) - {y^{\left( i \right)}}} \right)}^2}} \]
令 \[{\mathop{\rm Cos}\nolimits} t\left( {{h_\theta }\left( x \right),y} \right) = \frac{1}{2}{\left( {{h_\theta }\left( x \right) - y} \right)^2}\]
對於邏輯回歸而言，這樣的損失函數是非凸面函數，應該對損失函數做處理
\[{\mathop{\rm Cos}\nolimits} t\left( {{h_\theta }\left( x \right),y} \right) = \left\{ {\begin{array}{*{20}{c}}
{\begin{array}{*{20}{c}}
{ - \log \left( {{h_\theta }\left( x \right)} \right)}&{\begin{array}{*{20}{c}}
{if}&{y = 1}
\end{array}}
\end{array}}\\
{\begin{array}{*{20}{c}}
{\begin{array}{*{20}{c}}
{ - \log \left( {1 - {h_\theta }\left( x \right)} \right)}&{if}
\end{array}}&{y = 0}
\end{array}}
\end{array}} \right.\]
當y=1時，函數的圖形如下圖
當y=0時，函數的圖形如下圖
代價函數可以寫成 \[{\mathop{\rm Cos}\nolimits} t\left( {{h_\theta }\left( x \right),y} \right) = - y\log \left( {{h_\theta }\left( x \right)} \right) - \left( {1 - y} \right)\log \left( {1 - {h_\theta }\left( x \right)} \right)\]
這時 \[J\left( \theta \right) = - \frac{1}{m}[\sum\limits_{i = 1}^m {{y^{\left( i \right)}}\log \left( {{h_\theta }\left( {{x^{\left( i \right)}}} \right)} \right) + \left( {1 - {y^{\left( i \right)}}} \right)\log \left( {1 - {h_\theta }\left( {{x^{\left( i \right)}}} \right)} \right)]} \]
我們需要最小化代價函數J(θ)來找到θ
找到θ后運用h(x)預測新的輸入x