What Is Linear Regression? Why and How We Use It (Guide with Examples)

If you want to predict a number — a flat’s price, next month’s sales, a delivery time — linear regression is usually the first model to try. It is one of the oldest and most widely used tools in statistics and machine learning, and understanding it makes almost every other ML algorithm easier to learn. This guide explains what linear regression is, why we use it and how to use it, with a worked example and Python code.

What Is Linear Regression?

Linear regression is a supervised machine learning and statistical method that models the relationship between one or more input variables (features) and a continuous numeric output by fitting a straight line (or, with several inputs, a flat plane).

In plain words: it draws the line that best follows your data, then uses that line to predict new values. For example, it can learn that larger flats tend to cost more, and estimate the price of a flat of any size.

↑ Back to contents

The Equation Explained

For one input, the line is:

ŷ = b₀ + b₁x

• ŷ (y-hat) — the predicted value, e.g. price
• x — the input, e.g. area in sq ft
• b₁ (slope / coefficient) — how much ŷ changes when x increases by 1
• b₀ (intercept) — the predicted value when x is 0

With several inputs it becomes ŷ = b₀ + b₁x₁ + b₂x₂ + … + bₙxₙ. Each coefficient tells you the effect of that feature while the other features are held constant.

The difference between an actual value and the predicted value (y − ŷ) is called the residual or error.

↑ Back to contents

Types of Linear Regression

↑ Back to contents

How the Best-Fit Line Is Found

Many lines could pass through a cloud of points. Linear regression chooses the one with the smallest total error, using Ordinary Least Squares (OLS): it minimises the sum of squared residuals. Squaring makes all errors positive and penalises big misses more.

• Closed-form solution — for simple regression: b₁ = Σ(x − x̄)(y − ȳ) ÷ Σ(x − x̄)², and b₀ = ȳ − b₁x̄
• Gradient descent — an iterative method that nudges the coefficients step by step to reduce the error; used for very large datasets and the same idea that trains neural networks

↑ Back to contents

Worked Example by Hand

Suppose we have five flats:

1. Means: x̄ = 1,260 and ȳ = 46.8
2. Numerator: Σ(x − x̄)(y − ȳ) = 20,460
3. Denominator: Σ(x − x̄)² = 632,000
4. Slope: b₁ = 20,460 ÷ 632,000 ≈ 0.0324 → each extra sq ft adds about ₹3,240
5. Intercept: b₀ = 46.8 − 0.0324 × 1,260 ≈ 6.0
6. Model: Price ≈ 6.0 + 0.0324 × Area

Prediction: a 1,400 sq ft flat → 6.0 + 0.0324 × 1,400 ≈ ₹51.3 lakh. For this small, tidy dataset, R² is about 0.999 — real data is rarely this clean. (These numbers are illustrative, not actual Jaipur prices.)

↑ Back to contents

Why We Use Linear Regression

1. Prediction — estimate a number for new cases: prices, sales, demand, time.
2. Explanation — coefficients show how much each factor matters and in which direction, which managers can understand.
3. Simplicity and speed — trains in milliseconds, even on large datasets, and needs little tuning.
4. A strong baseline — any complex model should beat linear regression; if it does not, the extra complexity is not worth it.
5. Foundation for ML — logistic regression, regularisation, gradient descent and neural networks all build on the same ideas.
6. Trend analysis — measure whether something is rising or falling and how fast.

↑ Back to contents

Where Linear Regression Is Used

↑ Back to contents

How to Use Linear Regression: Step by Step

1. Define the target — a continuous number you want to predict.
2. Collect data — past records with the target and candidate features.
3. Explore — scatter plots and correlations to see which features relate to the target and whether the relationship looks roughly linear.
4. Prepare — handle missing values and outliers, encode categories (one-hot encoding), and create useful features (e.g. log of income).
5. Split — training and test sets (e.g. 80/20).
6. Fit — train the model on the training set.
7. Interpret — read the intercept and coefficients; do the signs make sense?
8. Evaluate — MAE, RMSE and R² on the test set.
9. Check assumptions — residual plots (see below).
10. Improve and deploy — add features, transform variables, try Ridge/Lasso, then use the model for predictions.

↑ Back to contents

Python Example: Predicting Flat Prices

This example uses scikit-learn with a generated sample dataset, so it runs as-is. Replace step 1 with pd.read_csv("your_file.csv") to use real data.

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

# 1. Create a sample dataset of 200 flats (replace with your own CSV)
rng = np.random.default_rng(42)
df = pd.DataFrame({
    "area_sqft": rng.integers(600, 2200, 200),
    "bedrooms": rng.integers(1, 5, 200),
    "age_years": rng.integers(0, 25, 200),
})
df["price_lakh"] = (5 + 0.035 * df["area_sqft"] + 3 * df["bedrooms"]
                    - 0.4 * df["age_years"] + rng.normal(0, 4, 200)).round(1)

# 2. Features (X) and target (y)
X = df[["area_sqft", "bedrooms", "age_years"]]
y = df["price_lakh"]

# 3. Train/test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# 4. Fit the model
model = LinearRegression().fit(X_train, y_train)
print("Intercept:", round(model.intercept_, 2))
for name, coef in zip(X.columns, model.coef_):
    print(f"{name}: {coef:.3f}")

# 5. Evaluate on unseen data
pred = model.predict(X_test)
print("MAE :", round(mean_absolute_error(y_test, pred), 2))
print("RMSE:", round(mean_squared_error(y_test, pred) ** 0.5, 2))
print("R²  :", round(r2_score(y_test, pred), 3))

# 6. Predict a new flat: 1,400 sq ft, 3 bedrooms, 5 years old
new_flat = pd.DataFrame({"area_sqft": [1400], "bedrooms": [3], "age_years": [5]})
print("Predicted price (lakh):", round(model.predict(new_flat)[0], 1))

Output (abridged): area_sqft 0.035, bedrooms 3.283, age_years −0.393; MAE 3.07, RMSE 3.78, R² 0.945; predicted price ₹61.2 lakh.

Reading it: each extra sq ft adds about ₹3,500, each extra bedroom about ₹3.3 lakh, and each year of age reduces the price by about ₹39,000 — holding the other features constant. The model recovered the patterns built into the sample data, and its typical error on unseen flats is about ₹3 lakh.

↑ Back to contents

Evaluating the Model

Always evaluate on test data the model has not seen. A high R² on training data alone can hide overfitting.

↑ Back to contents

Assumptions & How to Check Them

For reliable coefficients and confidence intervals, linear regression assumes (often remembered as LINE):

If prediction is your only goal, mild violations are often acceptable; if you want to interpret coefficients, check the assumptions carefully.

↑ Back to contents

Limitations and Common Mistakes

• Only captures linear patterns — use polynomial features, transformations or tree models for complex relationships.
• Sensitive to outliers — a few extreme values can tilt the line.
• Correlation is not causation — a coefficient shows association, not proof that one thing causes another.
• Extrapolation is risky — predictions far outside the training range are unreliable.
• Multicollinearity — correlated features make coefficients unstable and hard to interpret.
• Using it for categories — to predict yes/no outcomes use logistic regression instead.

Ridge, Lasso & Beyond

• Ridge (L2) — shrinks coefficients to reduce overfitting and handle multicollinearity.
• Lasso (L1) — can shrink some coefficients to exactly zero, selecting features automatically.
• Elastic Net — combines both.
• When relationships are strongly non-linear, try random forests or gradient boosting — and compare them against your linear baseline.

↑ Back to contents

Linear vs Logistic Regression

↑ Back to contents

How to Learn Linear Regression

1. Learn Python, pandas and basic plotting
2. Revise mean, variance, correlation and the equation of a line
3. Fit a simple regression by hand (like the example above), then with scikit-learn
4. Move to multiple regression, feature encoding and evaluation metrics
5. Practise checking residuals and assumptions; then learn Ridge and Lasso
6. Do a project: house prices, sales forecasting or delivery-time prediction

Reference: the scikit-learn linear models guide. Next, read What is supervised machine learning? and how to learn machine learning from scratch.

Learn it in Jaipur: regression, model evaluation and tuning with scikit-learn are part of the machine learning module in our AI & Machine Learning course; our Data Science and Predictive AI courses apply them to business forecasting. 6 months, 2-hour practical classes, IBM certificate on completion — in Jhotwara or live online.

↑ Back to contents

Frequently Asked Questions

Is linear regression machine learning or statistics?

Both. It began in statistics and is also one of the most basic supervised machine learning algorithms.

What is the difference between simple and multiple linear regression?

Simple linear regression uses one input feature; multiple linear regression uses two or more.

What is a good R² value?

It depends on the field. Physical or engineering data may reach 0.9+, while human behaviour data such as sales may be useful at 0.5. Compare against a baseline and look at MAE/RMSE in real units too.

Can linear regression handle categorical features?

Yes, after encoding them as numbers — usually one-hot encoding (e.g. separate 0/1 columns for each city area).

Do I need to scale features for linear regression?

Plain linear regression does not need scaling for accurate predictions, but scaling helps when comparing coefficients and is important for Ridge, Lasso and gradient descent.