About KNN and SVM

This is an article about KNN and SVM.

Hello!

Today, we will learn about KNN and SVM.

K-Nearest Neighbors (K-Nearest Neighbors, KNN)

K-nearest neighbor (KNN) is one of the supervised learning algorithms used for classification and regression problems. We will discuss KNNs below.

principle of operation

Neighbor’s Choice: Select the K nearest neighbors to the new data point.
Majority Vote: For classification problems, a majority vote among K neighbors determines the class of new data points. For regression problems, use the average of K neighbors as the predicted value.

pros and cons

Advantages: Simple to implement, fast and intuitive to train.
The downside: computational costs can be high as the distance to all training data needs to be calculated during the prediction stage.

Utilization

Outlier detection: Can be used for outlier detection.
Recommendation system: can be used to find similar users or products.

Support Vector Machine (SVM)

Support Vector Machine (SVM) is a supervised learning algorithm used for classification and regression problems that maps data into high-dimensional spaces to find the optimal decision boundaries.

principle of operation

Find Decision Boundary: Find the best hyperplane to separate the data.
Find Support Vector: Find the support vector that is the closest data point to the hyperplane.
Kernel Trick: To solve nonlinear problems, use kernel functions to map data into higher-dimensional spaces.

pros and cons

Advantage: It performs well on high-dimensional data, and can solve nonlinear problems through kernel techniques.
Disadvantages: the interpretation of the model is difficult, sensitive to data preprocessing and parameter settings.

Utilization

Binary and multi-class classification: widely used for classification problems.
Outlier detection: Can be used for outlier detection.

at the end of the day

KNN and SVM are supervised learning algorithms that are useful for various problems based on their respective features.

Thank you!