Logistic Regression...

Logistic Regression is basically a supervised classification algorithm, it is a predictive analysis algorithm and based on the concept of probability. In a classification problem, the target variable(or output), y, can take only discrete values for given set of features(or inputs), X.


Logistic regression models the data using the sigmoid function.






We can call a Logistic Regression a Linear Regression model but the Logistic Regression uses a more complex cost function , this cost function can be defined as the ‘Sigmoid function’ or also known as the ‘logistic function’ instead of a linear function.


The hypothesis of logistic regression tends it to limit the cost function between 0 and 1. Therefore linear functions fail to represent it as it can have a value greater than 1 or less than 0 which is not possible as per the hypothesis of logistic regression.


Sigmoid Function :




The Cost Function : The cost function represents optimization objective i.e. we create a cost function and minimize it so that we can develop an accurate model with minimum error.


For logistic regression, the Cost function is defined as:
−log(hθ(x)) if y = 1
−log(1−hθ(x)) if y = 0




The above two functions can be compressed into a single function i.e.




Gradient Descent :
The main goal of Gradient descent is to minimize the cost value. i.e. min J(θ).

Now to minimize our cost function we need to run the gradient descent function on each parameter i.e.

Learn More