Logistic Regression
- Supervised Learning, Classification model
- Probabilistic classification model
- Like other algorithms, logistic regression can be used for
a. Binary Classification
b. Multinomial Classification
c. Ordinal Classification
​
- Logreg uses sigmoid function for learning and prediction
Sigmoid Function
- This can convert any value between -∞ to +∞ into the range 0 to 1.
- Formula is
s(y)=1/(1+e^y)​
Where,
-
s(y) is the predicted probability that the input belongs to the positive class (typically labeled as 1).
-
e is the base of the natural logarithm. = 2.718
-
y is the output of the linear equation from step 1.​
y = w1 x1 + w2 x2 + w3 x3 + ... + wn xn ​
​
where,
-
y is the linear combination of the input features.
-
w1, w2, …, wn are the coefficients (weights) of the input features.
-
x1, x2, …, xn​ are the input features.
​
​
​
y = [ (val < 0.5) = 0
(val > 0.5) = 1 ]
This is how calssification happens in LogReg or binary classes
- The below depicts a linear regression that has shows two boundaries : 1 and 0
- For the data point in X and Y We will fit a line equation. This is the best fit line.
- The output what we are getting here is a linear output
- If we exceed the data, the line will go beyond 1 or beyond 0
- The below depicts a logistic regression.
- unlike linear regression, there is going to be a sigmoid curve and the transistion is very smooth.
- No matter what the data value is, the curve will be between 0 and 1
​
ODDS Ratio
-The weight factors (w1 x1 + w2 x2 + ... + wn xn) , the value of wn's will be calculated by the odds ratio in logistic regression.
- The odds ratio is defined as the ratio of odds of A in the presence of B and odds of A in the absence of B or vice versa.
- This quantifies the strength of association between two event/ classes (A [Spam] & B [Not Spam])
ODD = Probability / (1 - Probability)
