What Is Logistic Regression? Why Use It? (Guide with Examples)

Will this customer buy? Is this email spam? Will this loan be repaid? Questions with a yes or no answer are everywhere in business — and logistic regression is the classic first model for them. This guide explains what logistic regression is, how it works, why it is used, and how to build one in Python, with a worked example.

What Is Logistic Regression?

Logistic regression is a supervised machine learning algorithm used for classification. It predicts the probability that an input belongs to a class — for example, a 0.83 (83%) probability that a customer will churn — and then turns that probability into a decision such as “yes” or “no”.

Despite the word “regression” in its name, it is a classification method. The name comes from how it works: it uses a regression-style equation, then passes the result through the logistic (sigmoid) function to get a probability between 0 and 1.

↑ Back to contents

Why Not Just Use Linear Regression?

Linear regression predicts any number, from minus infinity to plus infinity. For a yes/no problem that causes trouble:

• It can predict impossible probabilities such as −0.3 or 1.4
• A straight line fits 0/1 data badly and is easily pulled by outliers
• It does not give a natural decision rule

Logistic regression fixes this by squeezing the output into the 0–1 range with an S-shaped curve.

↑ Back to contents

How Logistic Regression Works

1. The sigmoid function

First it computes a linear score, exactly like linear regression:

z = b₀ + b₁x₁ + b₂x₂ + … + bₙxₙ

Then it converts that score into a probability with the sigmoid (logistic) function:

p = 1 ÷ (1 + e^−z)

• When z is large and positive, p approaches 1
• When z = 0, p = 0.5
• When z is large and negative, p approaches 0

2. Odds and log-odds

Odds = p ÷ (1 − p). A probability of 0.8 means odds of 4 to 1. The linear score z is actually the log of the odds (the “logit”):

log( p ÷ (1 − p) ) = b₀ + b₁x₁ + … + bₙxₙ

That is why logistic regression is called a linear model: it is linear in the log-odds, not in the probability.

3. Threshold and decision boundary

To make a decision, choose a threshold: by default, predict “yes” if p ≥ 0.5. The set of points where p = 0.5 forms the decision boundary — a straight line (or flat plane) separating the two classes.

The threshold does not have to be 0.5. For fraud or disease screening, where missing a case is costly, a lower threshold such as 0.3 catches more positives at the cost of more false alarms.

4. How it is trained

The model finds the coefficients that make the observed labels most likely — maximum likelihood estimation. In practice it minimises log loss (binary cross-entropy), which heavily penalises confident wrong predictions. There is no closed-form answer, so solvers use iterative methods such as gradient descent or Newton-type methods (e.g. lbfgs).

↑ Back to contents

Worked Example by Hand

Suppose a model predicting whether a student passes an exam has learned:

log-odds = −4 + 0.9 × study hours

Interpreting the coefficient: e^0.9 ≈ 2.46, so each extra hour of study multiplies the odds of passing by about 2.5×. (Illustrative numbers.)

↑ Back to contents

Types of Logistic Regression

↑ Back to contents

Why Use Logistic Regression?

1. Gives probabilities, not just labels — you can rank customers by risk or set your own threshold.
2. Easy to interpret — coefficients become odds ratios that explain each factor’s effect, which matters in banking, insurance and healthcare where decisions must be explained.
3. Fast and lightweight — trains quickly on large datasets and predicts in microseconds.
4. Strong baseline — often surprisingly accurate; any complex model should be compared against it.
5. Works well with limited data and is less prone to overfitting than complex models, especially with regularisation.
6. Foundation for deep learning — a neural network’s output layer for classification is essentially logistic regression (sigmoid or softmax).

↑ Back to contents

Real-World Applications

↑ Back to contents

How to Use Logistic Regression: Step by Step

1. Define a categorical target — e.g. converted (1) or not (0).
2. Collect labelled data and check the class balance.
3. Prepare features — handle missing values, one-hot encode categories, scale numeric features.
4. Split into training and test sets, using stratification to keep the class ratio.
5. Fit the model (with regularisation, which scikit-learn applies by default).
6. Predict probabilities with predict_proba.
7. Choose a threshold that matches the business cost of false positives vs false negatives.
8. Evaluate with a confusion matrix, precision, recall, F1 and ROC-AUC.
9. Interpret coefficients as odds ratios and check they make sense.
10. Deploy and monitor — recheck performance as data changes.

↑ Back to contents

Python Example: Predicting Exam Results

This example uses scikit-learn with a generated sample dataset of 500 students, so it runs as-is. Replace step 1 with your own CSV to use real data.

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix, roc_auc_score

# 1. Sample data: 500 students (replace with your own CSV)
rng = np.random.default_rng(7)
df = pd.DataFrame({
    "study_hours": rng.uniform(0, 10, 500).round(1),
    "attendance": rng.uniform(40, 100, 500).round(0),
})
z = -9 + 0.8 * df["study_hours"] + 0.08 * df["attendance"]
df["passed"] = (rng.random(500) < 1 / (1 + np.exp(-z))).astype(int)

X = df[["study_hours", "attendance"]]
y = df["passed"]

# 2. Train/test split (stratify keeps the pass/fail ratio the same)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

# 3. Fit logistic regression (scaling helps the solver)
model = make_pipeline(StandardScaler(), LogisticRegression())
model.fit(X_train, y_train)

# 4. Evaluate
proba = model.predict_proba(X_test)[:, 1]   # probability of passing
pred = (proba >= 0.5).astype(int)            # default 0.5 threshold
print("Accuracy:", round(accuracy_score(y_test, pred), 3))
print("ROC-AUC :", round(roc_auc_score(y_test, proba), 3))
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred))

# 5. Predict for a new student: 6 study hours, 85% attendance
new = pd.DataFrame({"study_hours": [6], "attendance": [85]})
print("Probability of passing:", round(model.predict_proba(new)[0, 1], 2))

# 6. Odds ratios on the original scale (refit without scaling for interpretation)
raw = LogisticRegression().fit(X_train, y_train)
for name, coef in zip(X.columns, raw.coef_[0]):
    print(f"{name}: coef = {coef:.3f}, odds ratio = {np.exp(coef):.2f}")

Output (abridged): accuracy 0.84, ROC-AUC 0.92; recall for “passed” 0.93, for “failed” 0.73; probability of passing for the new student 0.92; odds ratios: study_hours 2.05, attendance 1.09.

↑ Back to contents

Interpreting Coefficients (Odds Ratios)

Take e^coefficient to get the odds ratio:

• study_hours: 2.05 — each extra hour of study roughly doubles the odds of passing, holding attendance constant.
• attendance: 1.09 — each extra percentage point of attendance raises the odds of passing by about 9%.
• An odds ratio above 1 increases the odds, below 1 decreases them, and exactly 1 means no effect.

Remember: odds ratios describe association, not proof of cause and effect.

↑ Back to contents

Evaluation Metrics

↑ Back to contents

Assumptions & Limitations

Assumptions

• The log-odds are a linear function of the features
• Observations are independent
• Little multicollinearity between features
• A reasonably large sample, with enough examples of each class

Limitations and fixes

• Linear decision boundary — add interaction or polynomial features, or try tree-based models for complex patterns.
• Imbalanced classes — use class_weight="balanced", resampling, and judge with recall/F1, not accuracy.
• Sensitive to outliers and correlated features — clean data and use regularisation (L1/L2, controlled by C in scikit-learn).
• Perfect separation — when one feature splits classes perfectly, coefficients blow up; regularisation fixes it.

↑ Back to contents

Linear vs Logistic Regression

↑ Back to contents

How to Learn Logistic Regression

1. Learn Python and pandas basics
2. Understand linear regression first
3. Learn probability, odds and the sigmoid function
4. Build a binary classifier with scikit-learn and read the confusion matrix
5. Practise threshold tuning, class imbalance and regularisation
6. Do a project: churn, loan default, lead conversion or spam detection

Reference: the scikit-learn logistic regression guide.

Learn it in Jaipur: classification, evaluation metrics and tuning with scikit-learn are part of the machine learning module in our AI & Machine Learning course, and our Data Science and Predictive AI courses apply them to business problems. 6 months, 2-hour practical classes, IBM certificate on completion — in Jhotwara or live online.

↑ Back to contents

Frequently Asked Questions

Is logistic regression classification or regression?

It is a classification algorithm. It uses a regression equation internally to estimate probabilities, which is where the name comes from.

Why is it called “logistic”?

Because it uses the logistic (sigmoid) function to convert a linear score into a probability.

Can logistic regression handle more than two classes?

Yes. Multinomial logistic regression uses the softmax function to give a probability for each class; scikit-learn handles this automatically.

What threshold should I use?

0.5 is only a default. Pick the threshold based on the cost of each error type — use a precision–recall curve to compare options.

Is logistic regression still used today?

Yes. It is widely used in credit scoring, marketing and healthcare because it is fast, reliable and explainable, and it is the standard baseline for any classification project.