linear_regression_matlab

Simple Linear Regression

1. View the dataset

2. Gradient descend

for iter = 1:num_iters
  error = X'*(X*theta - y);
  theta = theta - alpha/m*error;
  J_history(iter) = computeCost(X, y, theta);
end

3. Compute cost function

error = X*theta - y;
J = sum(error.*error)/(2*m);

4. Linear fit

5. Visualize cost function

Multiple Linear Regression

1. Feature normalization

mu = mean(X);
sigma = std(X);
n = length(mu);
for i = 1:n
    X_norm(:,i) = (X(:,i) - mu(i))/sigma(i);
end

2. Gradient descent

for iter = 1:num_iters
    error = X'*(X*theta - y);
    theta = theta - alpha/m*error;
    % Save the cost J in every iteration    
    J_history(iter) = computeCostMulti(X, y, theta);
end

3. Convergence vs learning rate

4. Normal equation (closed form solution)

theta = pinv(X'*X)*X'*y;