0

Self-supervised learning has been on the rise over the past few years. Compared to other learning methods such as supervised and semi-supervised, it does have an edge since it does not require labeled data.

I would like to know if self-supervised learning has any disadvantages and in what ways semi-supervised learning is better than it.

Tony Jesuthasan
  • 17
  • 5
  • I’m voting to close this question because it is not about programming as defined in the [help] but about ML theory and/or methodology - please see the intro and NOTE in the `machine-learning` [tag info](https://stackoverflow.com/tags/machine-learning/info). – desertnaut Sep 08 '21 at 07:31

1 Answers1

0

I think that the best way to illustrate this problem is to cite the great Yann LeCun:

If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning (RL).

The different types of ML can be very good or not depending on the case. For example, for robotics or autonomous driving problems, RL would be the ideal solution given the nature of these algorithms. However, for a recommender system or a stock price predictor, you could probably find better (and simpler) solutions in supervised and unsupervised learning.

Reinforcement learning is very different from supervised and unsupervised learning in that it needs to be defined in terms of agent, states, and environment, rather than simply data (and labels in the case of supervised learning). Therefore, you will need those elements and define the interactions between them very carefully to train a good and reliable system that, as I mentioned above, might not be the most optimal (or even feasible) solution for the problem you are trying to solve.

cy789
  • 55
  • 7
marsolmos
  • 748
  • 9
  • 24
  • What about self-supervised learning in image classification? Are there any disadvantages you can think of? – Tony Jesuthasan Sep 08 '21 at 06:52
  • Could you provide a source for that quote? Thanks :) – stan0 Sep 08 '21 at 07:04
  • It seems that he said that at NIPS 2016 @stan0 (video here: https://www.youtube.com/watch?v=Ount2Y4qxQo&t=1072s) – marsolmos Sep 08 '21 at 07:07
  • @TonyJesuthasan again, the problem it's in the complexity of the system. Self supervised learning just adds a previous step to a "normal" image classifier, where you have an algorithm labeling images automatically instead of a human. My opinion here is that this system isn't better intrinsically than a human-based one. If you can have access to human-labeled data easily, just go for the simplest system (supervised learning). If you can't have access to human-labeled data, then you will need more sophisticated solutions like self-supervised learning. As I said in my answer, depends on the case. – marsolmos Sep 08 '21 at 07:15