Week 4. 3rd Programming Assignment

이 주에는 Neural Network의 탄생 Intuition과 그 개념, 간단한 적용 예제들에 대해 배운다.

과제는 전 주의 Multiclass 분류와 이번 주의 Neural Network의 계산 방법 등에 대한 문제를 포함한다.

Neural Network문제는 주어진 weight에 대한 계산과 Predict만을 하기 때문에 아직 "학습"이라는 주제와는 거리가 조금 있어 보인다. 이번 과제도 역시 PDF를 잘 읽어보고 하자.

1. IrCostFunction.m

우선, 구체적으로 프로그램을 구현하기에 앞서, 자료들이 어떻게 표현되어 있는지 알아야한다. 이 과제에서는 어떤 20x20 pixel의 그림 5000장에 대해 학습을 시키는데, 이것은 X라는 5000x400짜리 배열로 주어진다. 따라서 어떤 행에 픽셀 하나가 들어있는 것으로 생각할 수 있다.

function [J, grad] = lrCostFunction(theta, X, y, lambda)
%LRCOSTFUNCTION Compute cost and gradient for logistic regression with 
%regularization
%   J = LRCOSTFUNCTION(theta, X, y, lambda) computes the cost of using
%   theta as the parameter for regularized logistic regression and the
%   gradient of the cost w.r.t. to the parameters. 

% Initialize some useful values
m = length(y); % number of training examples

% You need to return the following variables correctly 
J = 0;
grad = zeros(size(theta));
% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
%               You should set J to the cost.
%               Compute the partial derivatives and set grad to the partial
%               derivatives of the cost w.r.t. each parameter in theta
%
n=size(grad);
h=sigmoid(X*theta);
J=-sum(y.*log(h)+(1.-y).*log(1.-h))/m;
grad=X'*(h-y)/m;
J=J+lambda/(2*m)*(theta(2:n)'*theta(2:n));
temp=zeros(n);
temp=lambda/m*theta;
temp(1)=0;
grad=grad+temp;

% Hint: The computation of the cost function and gradients can be
%       efficiently vectorized. For example, consider the computation
%
%           sigmoid(X * theta)
%
%       Each row of the resulting matrix will contain the value of the
%       prediction for that example. You can make use of this to vectorize
%       the cost function and gradient computations. 
%
% Hint: When computing the gradient of the regularized cost function, 
%       there're many possible vectorized solutions, but one solution
%       looks like:
%           grad = (unregularized gradient for logistic regression)
%           temp = theta; 
%           temp(1) = 0;   % because we don't add anything for j = 0  
%           grad = grad + YOUR_CODE_HERE (using the temp variable)
%



% =============================================================

grad = grad(:);

end

IrCostFunction은 Regularization이 적용된 J와 grad를 그냥 구하면 된다. 이전 주차의 cost function과 크게 다르지 않다.

 

2. OnevsAll

function [all_theta] = oneVsAll(X, y, num_labels, lambda)
%ONEVSALL trains multiple logistic regression classifiers and returns all
%the classifiers in a matrix all_theta, where the i-th row of all_theta 
%corresponds to the classifier for label i
%   [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels
%   logistic regression classifiers and returns each of these classifiers
%   in a matrix all_theta, where the i-th row of all_theta corresponds 
%   to the classifier for label i

% Some useful variables
m = size(X, 1);
n = size(X, 2);

% You need to return the following variables correctly 
all_theta = zeros(num_labels, n + 1);

% Add ones to the X data matrix
X = [ones(m, 1) X];

% ====================== YOUR CODE HERE ======================
% Instructions: You should complete the following code to train num_labels
%               logistic regression classifiers with regularization
%               parameter lambda. 
%
% Hint: theta(:) will return a column vector.
%
% Hint: You can use y == c to obtain a vector of 1's and 0's that tell you
%       whether the ground truth is true/false for this class.
%
% Note: For this assignment, we recommend using fmincg to optimize the cost
%       function. It is okay to use a for-loop (for c = 1:num_labels) to
%       loop over the different classes.
%
%       fmincg works similarly to fminunc, but is more efficient when we
%       are dealing with large number of parameters.
%
% Example Code for fmincg:
%
%     % Set Initial theta
%     initial_theta = zeros(n + 1, 1);
%     
%     % Set options for fminunc
%     options = optimset('GradObj', 'on', 'MaxIter', 50);
% 
%     % Run fmincg to obtain the optimal theta
%     % This function will return theta and the cost 
%     [theta] = ...
%         fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ...
%                 initial_theta, options);
%
 options = optimset('GradObj', 'on', 'MaxIter', 50);
for i=1:num_labels
    initial_theta=zeros(n+1,1);
    [all_theta(i,:)]=fmincg (@(t)(lrCostFunction(t, X, (y == i), lambda)), ...
                 initial_theta, options)';
end

% =========================================================================


end

이건 뭐 어떻게 하라고 이미 주석에서 줬으니까 그대로만 하면 된다. y==i가 논리형 matrix가 되는 것만 좀 기억해두면 된다. 

 

3. Predict OnevsAll

OnevsAll의 결과를 가지고 그 그림이 어떤 수를 나타내는 건지 확인하는 작업이다. 

function p = predictOneVsAll(all_theta, X)
%PREDICT Predict the label for a trained one-vs-all classifier. The labels 
%are in the range 1..K, where K = size(all_theta, 1). 
%  p = PREDICTONEVSALL(all_theta, X) will return a vector of predictions
%  for each example in the matrix X. Note that X contains the examples in
%  rows. all_theta is a matrix where the i-th row is a trained logistic
%  regression theta vector for the i-th class. You should set p to a vector
%  of values from 1..K (e.g., p = [1; 3; 1; 2] predicts classes 1, 3, 1, 2
%  for 4 examples) 

m = size(X, 1);
num_labels = size(all_theta, 1);

% You need to return the following variables correctly 
p = zeros(size(X, 1), 1);

% Add ones to the X data matrix
X = [ones(m, 1) X];

% ====================== YOUR CODE HERE ======================
% Instructions: Complete the following code to make predictions using
%               your learned logistic regression parameters (one-vs-all).
%               You should set p to a vector of predictions (from 1 to
%               num_labels).
%
h=sigmoid(X*all_theta');
[~ ,p]=max(h,[],2);

% Hint: This code can be done all vectorized using the max function.
%       In particular, the max function can also return the index of the 
%       max element, for more information see 'help max'. If your examples 
%       are in rows, then, you can use max(A, [], 2) to obtain the max 
%       for each row.
%       

% =========================================================================


end

h=sigmoid(X*all_theta')에서 h=sigmoid(z)꼴로 만들었고, [~,p]=max(h,[],2);는 max함수에서 반환되는 각 row의 최대값이 있는 위치를 p 배열에 담는 코드이다.

 

4. Predict.m

function p = predict(Theta1, Theta2, X)
%PREDICT Predict the label of an input given a trained neural network
%   p = PREDICT(Theta1, Theta2, X) outputs the predicted label of X given the
%   trained weights of a neural network (Theta1, Theta2)

% Useful values
m = size(X, 1);
num_labels = size(Theta2, 1);
X=[ones(m,1) X];
% You need to return the following variables correctly 
p = zeros(size(X, 1), 1);
a2=sigmoid(Theta1*X');
a2=[ones(1,size(a2,2));a2];
a3=sigmoid(Theta2*a2);
[~,k]=max(a3,[],1);
p=k';
% ====================== YOUR CODE HERE ======================
% Instructions: Complete the following code to make predictions using
%               your learned neural network. You should set p to a 
%               vector containing labels between 1 to num_labels.
%
% Hint: The max function might come in useful. In particular, the max
%       function can also return the index of the max element, for more
%       information see 'help max'. If your examples are in rows, then, you
%       can use max(A, [], 2) to obtain the max for each row.
%

% =========================================================================


end

위랑 비슷하게 하면 됨