run Gradient Descent Algorithm
The Gradient Descent Algorithm is a popular optimization technique used in machine learning and deep learning to minimize a given objective function or to find the best possible solution for a problem. The algorithm works by iteratively updating the model parameters to minimize a cost function, which measures the difference between the predicted and actual output. In other words, gradient descent helps to find the minimum value of a function by taking steps proportional to the negative of the gradient, or the slope, of the function at the current point.
The primary concept behind the gradient descent algorithm is to find the direction of the steepest descent and take steps towards that direction to reach the minimum point. It starts with an initial random guess for the model parameters and computes the gradient of the cost function with respect to each parameter. Then, it updates the parameters in the direction of the negative gradient by a certain step size or learning rate. This process is repeated until the convergence criteria are met, or a predefined number of iterations are completed. The learning rate is a crucial hyperparameter in the gradient descent algorithm, as it determines the size of the steps taken towards the minimum point. A smaller learning rate may take more iterations to converge, while a larger learning rate may overshoot the minimum point and fail to converge.
% This script will pass sample parameters to gradient descent function and get Theta.
% I have used data consisting of populations as variables and profits as target values.
% Navigate to directory where you keep these three files on octave cli and then execute.
data = load('data_.txt');
Y = data(:, 2); % Y is a column vector with all entries in second column of data matrix.
n = length(Y);
x = [ones(n, 1), data(:,1)]; % x is the matrix with first column as ones and second data.
Theta = zeros(2, 1); % Initial Theta vector with both theta values set to zero (column).
Alpha = 0.01;
noi = 1500;
Theta = gradientdescent(x, Y, Theta, Alpha, noi);
fprintf('New theta vector is below:\n');
fprintf('%f\n', Theta);
fprintf('Expected Theta vector (approx)\n');
fprintf(' -3.6303\n 1.1664\n\n');
