# 机器学习之逻辑回归

## logistic函数

\$\$
{/theta}^{T}x=/theta_0x_0+/theta_1x_1+/theta_2x_2+…+/theta_nx_n
\$\$

\$\$
g(z)=/frac{1}{1+e^{-z}}
\$\$
logistic函数的图形如下：

\$\$
h_/theta(x)=g({/theta^T}{x})=/frac{1}{1+e^{-{/theta^T}{x}}}
\$\$

## 目标函数

\$\$
P(y=1{/mid}{x};{/theta})=h_/theta(x)
\$\$
\$\$
P(y=0{/mid}{x};{/theta})=1-h_/theta(x)
\$\$

\$\$
P(y{/mid}{x};{/theta})={(h_/theta(x))}^{y}{(1-h_/theta(x))}^{1-y}
\$\$

\$\$
L(/theta)=p(Y/mid{X};/theta)=/prod_{i=1}^{m}p(y^{(i)}{/mid}x^{(i)};/theta)=/prod_{i=1}^{m}{(h_/theta(x^{(i)}))^{y(i)}}{(1-h_/theta(x^{(i)}))^{1-y^{(i)}}}
\$\$

\$\$
l(/theta)=log(L(/theta))=/sum_{i=1}^{m}y^{(i)}logh_/theta(x^{(i)})+(1-y^{(i)})log(1-h_/theta(x^{(i)}))
\$\$

## 最大化参数似然性

\$\$
/frac{/partial}{/partial{/theta_j}}l(/theta)=/sum_{i=1}^{m}(y^{(i)}-h_/theta(x^{(i)}))x^{(i)}_j
\$\$

\$\$
/theta_j=/theta_j+/alpha/sum_{i=1}^{m}(y^{(i)}-h_/theta(x^{(i)}))x^{(i)}_j=/theta_j+/alpha/sum_{i=1}^{m}(y^{(i)}-/frac{1}{1+e^{-{/theta^T}{x^{(i)}}}})x^{(i)}_j
\$\$

## 逻辑回归模型的实现

6个训练样本的特征分别为(3,3), (3,4), (2,4), (-1,1), (-1,2), (1,1)对应的类别分别是1,1,1,0,0,0。现在要学习一个逻辑回归模型,并预测3个测试样本(2,1),(2,2),(2,3)的类别。

`load data;X = [ones(size(train_feature,1),1) train_feature];n = size(X,2); %特征数+1m = size(X,1); %样本数Y = train_label; %真实类别theta = zeros(n,1); %参数theta_new = zeros(n,1);epsilon = 0.0001; %收敛阈值alpha = 0.01; %学习率% 训练while 1    for(j = 1:n)        theta_new(j) = theta(j);        for(i = 1:m)            h = 1/(1+exp(-theta'*X(i, :)'));            theta_new(j) = theta_new(j) + alpha * (Y(i)-h) * X(i, j);         end;    end;    if norm(theta_new-theta) < epsilon        theta = theta_new;        break;    else        theta = theta_new;    end;end;% 测试test_feature = [ones(size(test_feature,1),1) test_feature];predict=[];predictLabel=[];for i=1:size(test_feature,1)    predict=[predict;1/(1+exp(-theta'*test_feature(i,:)'))];    if 1/(1+exp(-theta'*test_feature(i,:)'))>=0.5        predictLabel=[predictLabel;1];    else        predictLabel=[predictLabel;0];    end;end;`

`% 计算参数的似然性(仅用作分析)L(k) = 0;for(i = 1:m)    h = 1/(1+exp(-theta'*X(i, :)'));    L(k) = L(k) + (Y(i)*log(h) + (1-Y(i))*log(1-h));end;L(k) = exp(L(k));fprintf('The %dth iteration, L = %f /n', k, exp(L(k)));`

## 参考资料

1 斯坦福机器学习公开课及其课程笔记 by Andrew Ng