one vs all regression

Question

I have been reviewing an example from the course of Andrew Ng in Machine Learning which I found in https://github.com/jcgillespie/Coursera-Machine-Learning/tree/master/ex3. The example deals with logistic regression and one-vs-all classification. I have a doubt about this function:

function [all_theta] = oneVsAll(X, y, num_labels, lambda)
%ONEVSALL trains multiple logistic regression classifiers and returns all
%the classifiers in a matrix all_theta, where the i-th row of all_theta 
%corresponds to the classifier for label i
%   [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels
%   logisitc regression classifiers and returns each of these classifiers
%   in a matrix all_theta, where the i-th row of all_theta corresponds 
%   to the classifier for label i

% Some useful variables
m = size(X, 1);
n = size(X, 2);

% You need to return the following variables correctly 
all_theta = zeros(num_labels, n + 1);

% Add ones to the X data matrix
X = [ones(m, 1) X];

% ====================== YOUR CODE HERE ======================
% Instructions: You should complete the following code to train num_labels
%               logistic regression classifiers with regularization
%               parameter lambda. 
%
% Hint: theta(:) will return a column vector.
%
% Hint: You can use y == c to obtain a vector of 1's and 0's that tell use 
%       whether the ground truth is true/false for this class.
%
% Note: For this assignment, we recommend using fmincg to optimize the cost
%       function. It is okay to use a for-loop (for c = 1:num_labels) to
%       loop over the different classes.
%
%       fmincg works similarly to fminunc, but is more efficient when we
%       are dealing with large number of parameters.
%
% Example Code for fmincg:
%
%     % Set Initial theta
%     initial_theta = zeros(n + 1, 1);
%     
%     % Set options for fminunc
%     options = optimset('GradObj', 'on', 'MaxIter', 50);
% 
%     % Run fmincg to obtain the optimal theta
%     % This function will return theta and the cost 
%     [theta] = ...
%         fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ...
%                 initial_theta, options);
%

initial_theta = zeros(n + 1, 1);

options = optimset('GradObj', 'on', 'MaxIter', 50);

for i = 1:num_labels

    c = i * ones(size(y));
    fprintf('valores')
    [theta] = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options);
    all_theta(i,:) = theta;

end


% =========================================================================


end

I know that the lrCostFunction takes as parameters: theta, X, y and lambda, but I cannot figure it out from where the value of t comes from in the code that I posted above; specifically in this part:

[theta] = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options);

any help?

score 9 · Accepted Answer · answered May 16 '15 at 21:52

fmincg takes the handle of the objective function as the first argument, which in this case is a handle to lrCostFunction.

If you go inside fmincg.m, you will find the following lines:

argstr = ['feval(f, X'];                      % compose string used to call function

%---Code will not enter the following loop---%
for i = 1:(nargin - 3) %this will go from 1 to 0, thus the loop is skipped
   argstr = [argstr, ',P', int2str(i)];
end
% following will be executed
argstr = [argstr, ')'];

At the end of above code snippet, result will be,

argstr=feval(f,X');

If you a little ahead, you will see,

[f1 df1] = eval(argstr);                      % get function value and gradient

Therefore, the function handle f will run with an argument X'. Therefore, t=X', which makes sense too. The initial theta will converge to give you the final parameter vector for logistic regression.

score 5 · Answer 2 · edited Oct 16 '17 at 02:20

5

You can actually substitute.

for i=1 : num_labels

    [theta]= fmincg (@(t)(lrCostFunction(t, X, (y == i), lambda)),initial_theta, options);

all_theta(i,:)=theta;

edited Oct 16 '17 at 02:20

The Guy with The Hat

10,836
8
57
75

answered Jun 11 '16 at 07:20

tanay

51
1
1

3

what does [theta] do ? why not theta(i) ? or if its just for assignment, why the brackets at all? – nights Nov 26 '16 at 07:17
I guess there will be no problem if we write **theta** at place of **[theta]** – sonu_chauhan Mar 01 '18 at 12:03

nitya · Answer 3 · 2017-08-05T06:37:53.797

4

try this

for i = 1:num_labels,
    [all_theta(i,:)] = fmincg (@(t)(lrCostFunction(t, X, (y == i), lambda)), initial_theta, options);
end;

you also don't need to initialize all_theta in the beginning

edited Aug 05 '17 at 06:37

answered Aug 05 '17 at 06:05

nitya

41
2

one vs all regression

3 Answers3