1

Is Gradient Descent algorithm ever used during training of any unsupervised training like clustering, collaborative filtering, etc..?

Aman
  • 475
  • 2
  • 6
  • 10
  • if you can define a function you're trying to optimize, then you could use gradient descent to do so. Nothing jumps to mind those cases, but I wouldn't be surprised if there were applications – Rorschach Feb 21 '20 at 15:46

2 Answers2

1

Gradient descent can be used for a whole bunch of unsupervised learning tasks. In fact, neural networks, which use the Gradient Descent algorithm are used widely for unsupervised learning tasks, like representations of text or natural language in vector space (word2vec).

You can also think of dimensionality reduction techniques like autoencoders, which use Gradient Descent as well.

I am not aware of GD being directly used in clustering, but this link discusses an approach that utilizes Autoencoders with Kmeans, which use GD.

Read this link as well, which discusses a similar question.

Shaunak Sen
  • 548
  • 3
  • 8
0

In unsupervised algorithms, you don't need to do this. For example, in k-Means, where you are trying to minimize the mean squared error (MSE), you can minimize the error directly at each step given the assignments; no gradients needed.

Bhavuk Chawla
  • 212
  • 1
  • 10
  • k-means is just one simple example, though. How can you generalize to all unsupervised learning – Rorschach Feb 21 '20 at 15:49
  • @picklerick please do provide other examples – Bhavuk Chawla Feb 21 '20 at 15:51
  • In other clustering models, such as a mixture of Gaussians, the expectation-maximization (EM) algorithm is much more powerful and accurate than any gradient-descent based method. – Bhavuk Chawla Feb 21 '20 at 15:52
  • IIRC I've used gradient descent for various neighborhood analyses in the past -- any time you are doing simulated annealing for example – Rorschach Feb 21 '20 at 15:52