Why is KNN much faster than decision tree?

Question

Once in an interview, I encountered a question from the employer. He asked me why KNN classifier is much faster than decision tree for example in letter recognition or in face recognition?

I had completely no idea at that time. So I want to know in which terms should I compare the two classification methods in speed performance? Thanks.

there are quite a few comparisons online – Dreamwalker Mar 15 '13 at 09:03 — Dreamwalker, Mar 15 '13 at 09:03

score 6 · Accepted Answer · answered Mar 15 '13 at 09:35

6

Consider the following dataset: N samples, each has k attributes. In general :
1. naive KNN: O(1) [training Time] + O(NK) [query Time] = O (NK)
2. naive decision tree: O(N^2 * K * log(N)) [training Time] + O(log(N)) [query Time] = O(N^2 * K) -- Also for query time, we assume that the tree is balanced.
To calculate the complexities, I considered very simple implementation of each classifier. Already there are few improvements for implementing KNN and Decision Tree.

answered Mar 15 '13 at 09:35

Majid Darabi

731
6
15

Very appreciate your answering. Could you explain a little bit more about the training time for each of the classifier? Thanks. – zfz Mar 16 '13 at 08:18
@Majid, can you please provide a reference for further reading? – Samuel Oct 24 '17 at 18:01

Why is KNN much faster than decision tree?

1 Answers1