What Is KNN (K-Nearest Neighbours)? Why and How to Use It (Guide with Examples)

“Tell me who your neighbours are, and I’ll tell you who you are.” That old saying is exactly how the K-Nearest Neighbours (KNN) algorithm works. It is one of the simplest machine learning algorithms to understand — and still useful in recommendations, search and anomaly detection. This guide explains what KNN is, how it works, why and when to use it, with a worked example and Python code.

What Is KNN?

K-Nearest Neighbours (KNN) is a supervised machine learning algorithm used for both classification and regression. To make a prediction for a new data point, it finds the k most similar points (the nearest neighbours) in the training data and:

• Classification — takes a majority vote of the neighbours’ labels
• Regression — takes the average of the neighbours’ values

KNN is called a lazy learner and a non-parametric method: it does not learn an equation during training. It simply stores the training data and does all the work at prediction time.

↑ Back to contents

A Simple Analogy

You move to a new area of Jaipur and want to know a good price for a flat. You look at the 5 flats most similar to yours — same size, nearby, similar age — and take their average price. You have just done KNN regression with k = 5. If instead you asked those 5 neighbours which school they chose and picked the most common answer, that is KNN classification.

↑ Back to contents

How KNN Works (Step by Step)

1. Choose k — the number of neighbours to consult (e.g. 5).
2. Scale the features so every feature contributes fairly to distance.
3. Measure distance from the new point to every training point.
4. Pick the k closest training points.
5. Predict — majority vote (classification) or average (regression). Optionally, weight closer neighbours more heavily.

↑ Back to contents

Distance Metrics

↑ Back to contents

Worked Example by Hand

A store wants to predict whether a new customer will buy a premium membership. Training data (illustrative):

New customer: age 33, spend ₹7 thousand. Euclidean distances, nearest first:

• k = 1: nearest is C → Yes
• k = 3: C, E, D → 2 Yes vs 1 No → Yes
• k = 5: C, E, D, B, A → 2 Yes vs 3 No → No

Two lessons: the choice of k can change the answer, and because age has a larger range than spend here, age dominates the distance — which is why scaling matters (see below).

↑ Back to contents

How to Choose k

• Small k (e.g. 1) — very flexible, follows noise → overfitting, unstable predictions
• Large k — smoother, more stable, but may blur real boundaries → underfitting
• Use an odd k for two-class problems to avoid ties
• Pick k with cross-validation — try several values and keep the one with the best validation score
• A rough starting point some practitioners use is √n (n = number of training samples), then tune from there

↑ Back to contents

Why Feature Scaling Is Essential

KNN is entirely distance-based. If one feature is measured in lakhs (income) and another in single digits (number of children), the large-scale feature swamps the distance and the other is effectively ignored. Always scale features first:

• Standardisation (StandardScaler) — mean 0, standard deviation 1
• Min–max scaling (MinMaxScaler) — each feature squeezed to 0–1
• Encode categorical features (one-hot) before computing distances

In the Python example below, scaling alone raises test accuracy from 0.78 to 1.0.

↑ Back to contents

KNN for Regression

For a numeric target, KNN averages the values of the k nearest neighbours — e.g. estimating a flat’s price from the 5 most similar flats. With weights="distance", closer neighbours count more. Compare it against linear regression: KNN can capture non-linear patterns, but cannot predict beyond the range of values it has seen.

↑ Back to contents

Why Use KNN?

1. Easy to understand and explain — “these similar cases had this outcome”.
2. No training phase — adding new data is instant; just store it.
3. Handles non-linear boundaries naturally, unlike logistic regression.
4. Works for classification and regression, and multi-class problems out of the box.
5. Few assumptions about how the data is distributed.
6. The idea behind similarity search — nearest-neighbour search over embeddings powers recommendations and RAG systems in modern AI.

↑ Back to contents

Real-World Applications

↑ Back to contents

Python Example: Classifying Wines

This example uses the wine dataset built into scikit-learn (178 wines, 13 chemical features, 3 classes), so it runs as-is. It shows the effect of scaling and chooses k by cross-validation.

from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report

# 1. Load a built-in dataset: 178 wines, 13 chemical features, 3 classes
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

# 2. Why scaling matters: KNN without and with feature scaling
unscaled = KNeighborsClassifier(n_neighbors=5).fit(X_train, y_train)
print("Without scaling:", round(accuracy_score(y_test, unscaled.predict(X_test)), 3))

# 3. Choose k with 5-fold cross-validation (on scaled data)
scores = {}
for k in range(1, 22, 2):
    model = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=k))
    scores[k] = cross_val_score(model, X_train, y_train, cv=5).mean()
best_k = max(scores, key=scores.get)
print("Best k:", best_k, "| CV accuracy:", round(scores[best_k], 3))

# 4. Train the final model and evaluate once on the test set
knn = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=best_k))
knn.fit(X_train, y_train)
pred = knn.predict(X_test)
print("With scaling   :", round(accuracy_score(y_test, pred), 3))
print(classification_report(y_test, pred))

Output (abridged): without scaling 0.778; best k = 15 with cross-validation accuracy 0.977; with scaling 1.0 on the test set.

A perfect test score is unusual — this is a small, clean teaching dataset with only 45 test samples. On real business data, expect lower scores and always rely on cross-validation.

↑ Back to contents

Advantages and Limitations

Curse of dimensionality

As the number of features grows, points become almost equally far from each other, so “nearest” loses meaning. Fixes: feature selection, dimensionality reduction such as PCA (see unsupervised learning), or a different algorithm.

Making KNN faster

• Tree-based search structures — KD-tree and Ball tree (scikit-learn chooses automatically with algorithm="auto")
• Approximate nearest-neighbour libraries such as FAISS for millions of vectors
• Reduce dimensions or sample the training data

↑ Back to contents

KNN vs Other Algorithms

↑ Back to contents

How to Learn KNN

1. Learn Python, pandas and basic plotting
2. Understand distance (Euclidean, Manhattan) with a small example by hand
3. Build a KNN classifier in scikit-learn and compare with and without scaling
4. Tune k with cross-validation; try weights="distance"
5. Try KNN regression and KNN imputation
6. Explore nearest-neighbour search over embeddings — the link to recommendations and RAG

Reference: the scikit-learn nearest neighbours guide. Related reads: linear regression, logistic regression.

Learn it in Jaipur: classification, regression, model evaluation and tuning with scikit-learn are part of the machine learning module in our AI & Machine Learning course, and our Data Science course applies them to real datasets. 6 months, 2-hour practical classes, IBM certificate on completion — in Jhotwara or live online.

↑ Back to contents

Frequently Asked Questions

Is KNN supervised or unsupervised?

KNN is supervised — it needs labelled training data. It is often confused with K-means, which is an unsupervised clustering algorithm.

What is the difference between KNN and K-means?

KNN predicts a label for a new point from its labelled neighbours (supervised). K-means groups unlabelled data into k clusters (unsupervised). The “k” means different things in each.

What is the best value of k?

There is no universal best k. Choose it with cross-validation; odd values avoid ties in two-class problems.

Why is KNN called a lazy learner?

Because it does not build a model during training — it just stores the data and does the computation when a prediction is requested.

Is KNN good for large datasets?

Plain KNN gets slow and memory-hungry on very large datasets. Use tree-based or approximate nearest-neighbour search, or choose a different algorithm.