10

Unlike other algorithms like linear regressions ,KNN doesn't seems to perform any calculation in the training phase. Like in case of linear regressions it finds the coefficients in the training phase.But what about KNN?

Shaurya Sheth
  • 185
  • 2
  • 11
user10418143
  • 220
  • 3
  • 11

4 Answers4

14

During training phase, KNN arranges the data (sort of indexing process) in order to find the closest neighbors efficiently during the inference phase. Otherwise, it would have to compare each new case during inference with the whole dataset making it quite inefficient.

You can read more about it at: https://scikit-learn.org/stable/modules/neighbors.html#nearest-neighbor-algorithms

J. Ferrarons
  • 560
  • 2
  • 7
6

KNN belongs to the group of lazy learners. As opposed to eager learners such as logistic regression, svms, neural nets, lazy learners just store the training data in memory. Then, during inference, it find the K nearest neighbours from the training data in order to classify the new instance.

gorjan
  • 5,405
  • 2
  • 20
  • 40
1

KNN is an instance based method, which completely relies on training examples, in other words, it memorizes all the training examples So in case of classification, whenever any examples appears, it compute euclidean distance between the input example and all the training examples, and returns the label of the closest training example based on the distance.

Ramesh Kumar
  • 1,508
  • 15
  • 16
1

Knn is lazy learner . It means that , like other algorithms learn in their training phase (Linear regression etc) , Knn learn in training phase . It actually just store data points in RAM at time of training .

Like in case of linear regressions it finds the coefficients in the training phase.But what about KNN?--> In case of KNN it tunes its parameter in testing phase . In testing phase it finds its optimal solution of parameters (K value , Distance calculating technique etc). Unlike other algorithms which learn in training phase and get tested in testing phase , KNN learn and get tested(K fold CV) for parameters in testing phase .

Distance calculation->https://scikit-learn.org/stable/modules/neighbors.html#nearest-neighbor-algorithms

KNN python docs->https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html

hemendra shinde
  • 11
  • 3