Subscribe to get access
??Subscribe to read the rest of the comics, the fun you can’t miss ??
K-Nearest Neighbors (KNN) is a popular algorithm used for both classification and regression tasks. In KNN, the output is a class membership, which is assigned based on the majority of the k nearest data points. One of the advantages of KNN is its ability to adapt to new training data. Additionally, it doesn’t make any assumptions about the underlying data distribution, which can be beneficial in certain scenarios.
Example
Consider an example of using the K-Nearest Neighbors (KNN) algorithm with a small dataset with 5 samples and 2 input features. Let’s assume we are classifying the samples into two classes.
| Sample | Feature 1 | Feature 2 | Class |
|---|---|---|---|
| A | 1.0 | 1.0 | 0 |
| B | 2.0 | 2.0 | 0 |
| C | 3.0 | 3.0 | 1 |
| D | 4.0 | 4.0 | 1 |
| E | 5.0 | 5.0 | 1 |
Steps to Implement KNN:
- Choose the value of K: Let’s choose
.
- Calculate the distance between the point to be classified and all other points in the dataset. We’ll use Euclidean distance.
- Sort the distances and determine the nearest neighbors based on the chosen K value.
- Vote for the classes of the nearest neighbors.
- Assign the class with the highest vote to the point to be classified.
Example: Let’s classify a new point 
Step 1: Calculate Distances
Step 2: Sort Distances:
C : 0.71
D : 0.71
B : 2.12
E : 2.12
A : 3.54
Step 3: Determine Nearest Neighbors: The 3 nearest neighbors to (3.5, 3.5) are:
- C (0.71)
- D (0.71)
- B (2.12)
Step 4: Vote for Classes: The classes of the 3 nearest neighbors are:
- C: Class 1
- D: Class 1
- B: Class 0
The votes are:
- Class 0: 1 vote
- Class 1: 2 votes
Step 5: Assign Class: The new point 
Discover more from Science Comics
Subscribe to get the latest posts sent to your email.
