Can an SVM learn incrementally?

Question

I am using a multi-dimensional SVM classifier (SVM.NET, a wrapper for libSVM) to classify a set of features.

Given an SVM model, is it possible to incorporate new training data without having to recalculate on all previous data? I guess another way of putting it would be: is an SVM mutable?

I started going through Bishop's book to help answer this but I believe you might get a more insightful answer over at http://mathoverflow.net/ — wheaties, Oct 19 '10 at 11:50

score 14 · Accepted Answer · edited May 23 '17 at 10:30

14

Actually, it's usually called incremental learning. The question has come up before and is pretty well answered here : A few implementation details for a Support-Vector Machine (SVM).

In brief, it's possible but not easy, you would have to change the library you are using or implement the training algorithm yourself.

I found two possible solutions, SVMHeavy and LaSVM, that supports incremental training. But I haven't used either and don't know anything about them.

edited May 23 '17 at 10:30

Community

1
1

answered Oct 19 '10 at 11:55

Stompchicken

15,833
1
33
38

Thanks, I changed my question title to be more accurate. The question you refer to is a good answer to mine. – Petrus Theron Oct 19 '10 at 13:07
@larsmans Good point. It's also sometimes confused/conflated with active learning. – Stompchicken Oct 20 '10 at 09:59

score 2 · Answer 2 · edited Apr 13 '17 at 12:44

Online and incremental although similar but differ slightly. In online, its generally a single pass(epoch=1) or number of epochs could be configured. Where as, incremental would mean that you already have a model; no matter how it is built, but then model can be mutable by new examples. Also, a combination of online and incremental is often what is required.

Here is a list of tools with some remarks on the online and/or incremental SVM : https://stats.stackexchange.com/questions/30834/is-it-possible-to-append-training-data-to-existing-svm-models/51989#51989

Can an SVM learn incrementally?

2 Answers2

Linked

Related