30

I have been looking into deep learning frameworks lately and have been wondering about the origin of the name of PyTorch.

With Keras, their home page nicely explains the name's origin, and with something like TensorFlow, the reasoning behind the name seems rather clear. For PyTorch, however, I cannot seem to come across why it is so named.

Of course, I understand the "Py-" prefix and also know that PyTorch is a successor in some sense of Torch. But I am still wondering: what is the original idea behind the "-Torch" part? Is it known what the origin of the name is?

iacob
  • 20,084
  • 6
  • 92
  • 119
Grayscale
  • 1,462
  • 1
  • 13
  • 20
  • I think this might help you. https://www.infoworld.com/article/3159120/artificial-intelligence/facebook-brings-gpu-powered-machine-learning-to-python.html – jits_on_moon Jul 26 '18 at 04:17
  • look for reference at the bottom of page, https://en.wikipedia.org/wiki/PyTorch – jits_on_moon Jul 26 '18 at 04:18
  • 2
    @Mr.J The article did not contain what I am looking for. – Grayscale Jul 26 '18 at 04:20
  • 1
    https://twitter.com/msalihkaragoz/status/1034920476148215808 – Salih Karagoz Aug 30 '18 at 21:55
  • Lest you think I'm applying some unknown standard, we [had a similar debate some time back about nearly the same thing](https://meta.stackoverflow.com/questions/384376/are-questions-about-the-motives-of-programming-library-developers-on-topic). Particularly [this comment](https://meta.stackoverflow.com/questions/384376/are-questions-about-the-motives-of-programming-library-developers-on-topic#comment692028_384376) – Machavity Sep 06 '20 at 14:47
  • 1
    @Machavity I think there is at least one important distinction from your linked question: this one evidently gives insight (via the origins of the software) which plausibly yields insight or understanding for someone programming with the software. The question about curl was very interesting, but probably does not satisfy the above. What do you think? – Grayscale Sep 14 '20 at 04:11
  • @Grayscale You have to torture the definition of "insight" to get there. I'm not inclined to delete the question or lock it, but it was garnering poor answers and the one answer not merely mentioning the one connection you did doesn't provide a hard answer, but is merely an educated guess. – Machavity Sep 14 '20 at 12:28

1 Answers1

33

Here a short answer, formed as another question:

Torch, SMORCH ???

PyTorch developed from Torch7. A precursor to the original Torch was a library called SVM-Torch, which was developed around 2001. The SVM stands for Support Vector Machines.

SVM-Torch is a decomposition algorithm similar to SVM-Light, but adapted to regression problems, according to this paper.

Also around this time, G.W.Flake described the sequential minimal optimization algorithm (SMO), which could be used to train SVMs on sparse data sets, and this was incorporated into NODElib.

Interestingly, this was called the SMORCH algorithm.

You can find out more about SMORCH in the NODElib docs

Optimization of the SVMs is:

  • performed by a variation of John Platt's sequential minimal
  • optimization (SMO) algorithm. This version of SMO is generalized
  • for regression, uses kernel caching, and incorporates several
  • heuristics; for these reasons, we refer to the optimization
  • algorithm as SMORCH.

So SMORCH =

Sequential
Minimal
Optimization
Regression
Caching
Heuristics

I can't answer definitively, but my thinking is "Torch" is a riff or evolution of "Light" from SVM-Light combined with a large helping of SMORCHiness. You'd need to check in with the authors of SVMTorch and SVM-Light to confirm that this is indeed what "sparked" the name. It is reasonable to assume that the "TO" of Torch stands for some other optimization, rather than SMO, such as Tensor Optimization, but I haven't found any direct reference... yet.

E_net4
  • 27,810
  • 13
  • 101
  • 139
Good Lux
  • 896
  • 9
  • 19
  • 1
    Wow, the history of the name is indeed quite interesting! I wonder if the move from SVM-Light to SVM-Torch has to do with a combination of the use of SMORCH and also a bit of a throwback to the old name in that torches produce light... – Grayscale Oct 17 '18 at 05:29
  • Also are you sure that SVM-Torch uses SMORCH? – Grayscale Oct 17 '18 at 05:31
  • Downvoted, because this question has nothing to do with programming and should be closed. What if I ask why Java is called Java or Android is called Android? Won't it be closed? – user1209216 Dec 03 '18 at 19:12
  • 9
    @user1209216 - Strongly disagree. That's like saying PHP has nothing to do with programming Personal Home Pages. It's nonsense to say that discussion about programming can't include the name of the programming language or algorithm itself...names sum up the entire 'gist' of the tool and are a great way to understand what a thing is, and its context. – Good Lux Dec 03 '18 at 19:19
  • And Android is a reference to writer Philip K Dick...cool, but agreed, not nearly as programming related on the surface, and Java just sounds cooler than Oak. If you dig in to who actually gave them those names, you will learn a lot about programming though! – Good Lux Dec 03 '18 at 19:24
  • @grayscale svm-torch does not use SMORCH. The SMORCH algorithm was part of NODElib...they were contemporaries, two approaches in the same problem space. For instance, this paper compares SVM-Torch with NODElib. jmlr.org/papers/volume1/collobert01a/collobert01a.pdf - Whether one was named because of, or in reference to the other, is not known. From all accounts I've gathered, the naming seems to have been coincidental, although uncanny from a historical viewpoint. SVM-Torch is said to have been named in a nod to SVMLight – Good Lux Dec 03 '18 at 19:49
  • @lux afaik so is about programming problems to solve.Definition/terminology about libs/frameworks/tools names has nothing to do with actual programming, even if you ask for tools used by programmers. Btw as for name, I think better is to ask devs, isn't it? – user1209216 Dec 03 '18 at 21:00