5

We can reload any function and/or variable in Clojure at runtime almost instantly. We can even change method signatures. The most we can do with Scala or Java is to use JRebel which is slow, commercial, and restricted. What is the difference that allows Clojure to be so interactive? Reading about this in Slack, I have found the following comments, but I wish to know more about it. Links to papers/articles clarifying the issue further is also appreciated (though not required).

It’s mostly because the language is set up to be reloadable. Clojure has a var indirection for every function or top level variable definition which you can mutate, so you can redefine just one function while keeping the rest of your environment the same and carry on

.

following up on that - there's indirection when the function name is in the code, but for a long running function that took another function as an argument (eg. you passed a handler function to an http server process startup) you can get the benefits of var indirection by hand - by passing #'handler instead of handler but otherwise you don't get the reloading (without restarting the process that took that arg)

.

kind of

direct linking replaces var calls being compiled with direct calls (edited) the var path however still exists and NEW code can still invoke via the vars

HappyFace
  • 3,439
  • 2
  • 24
  • 43
  • It looks like you have the answer to your question in the quotes you posted. Most Lisps allow this kind of editing AFAIK. You may want to have a look at smalltalk as well. – nha Apr 20 '18 at 13:56
  • @nha I feel like I don't understand the comments very well. What is a var indirection? Why don't other languages just copy this idea? What are the trade-offs? Why is #'handler different from handler? Can compiled methods (like from the core library) be redefined in user code? Are there security concerns? What are var paths? ... – HappyFace Apr 20 '18 at 14:03
  • 1
    Please read https://clojure.org/reference/vars http://swannodette.github.io/2014/12/17/whats-in-a-var http://blog.cognitect.com/blog/2016/9/15/works-on-my-machine-understanding-var-bindings-and-roots there are probably others but these should give you a good understanding of what a var is. – nha Apr 20 '18 at 14:08
  • The JVM isn't any more hotswapp-able at the byte code level for different languages. However if the language is higher level, it could make changes at the language level which don't look like unsupported changes at the byte code level. – Peter Lawrey Apr 20 '18 at 17:21
  • 1
    Also because given the semantics of clojure, what you're reloading are *generally pure functions*. This is generally not true of e.g. Java because in Java you'd be reloading class files, which would probably play havok with the structure of your current instances of the reloaded class. – Jared Smith Apr 20 '18 at 17:41

1 Answers1

13

The key to what you're asking lies on how Clojure identifies functions and runs them at runtime. First, Clojure functions are defined as vars, which is the Clojure name for their JVM root class, Var.

Clojure's runtime maintains a single ConcurrentHashMap called Namespaces. This map has Symbol keys (the namespace name) and Namespace values. Each Namespace in turn has an AtomicReference'd Clojure map (called "mappings") that is dynamically typed but which essentially has Clojure Symbol keys (the local variable name) and Var values.

When you invoke a Clojure function, it first looks up which namespace you're referencing in Namespaces and then looks up the specific variable in that namespace's mappings. This makes hot-loading code trivial - all you need to do is set a new <Symbol, Var> pair on a given namespace's mappings.

To go one level deeper, Clojure also maintains an awareness of "frames" (i.e. threads or additional bindings that might temporarily re-define variables within a local scope). These have their own ThreadLocal storage and a variable that is found in one of these will be used instead of the variable currently being stored in the namespace's mappings.

Clojure's approach here is possible because it doesn't try to store functions as JVM functions, but rather as Java Objects themselves that are kept in a map that can be rapidly accessed.

Clojure knows these Objects are in fact callable by checking to see if they satisfy a function interface (IFn). An object satisfies IFn by having an Invoke method. This is used for a wide number of pretty clever purposes, and explains why many of Clojure's core data structures (maps, vectors, keywords, etc.) are all also callable as functions.

Venantius
  • 2,471
  • 2
  • 28
  • 36
  • 1
    Doesn’t this make calling functions slow? – HappyFace Apr 23 '18 at 13:20
  • 1
    @HappyFace Clojure functions _are_ slow when compared to a "pure JVM" equivalent. But at the end of the day, "slow" is relative - the things that make a program perform badly under strain tend to be defined by algorithmic properties that don't scale well, not the fact that a function invocation took slightly longer (which is only a linear performance hit). – Venantius Apr 23 '18 at 13:44
  • 1
    @HappyFace The traditional advice when it comes to performance is: make sure you have a problem first before you optimize it. When it comes to Clojure, optimization can mean converting certain critical paths to pure Java, but most of the time there are higher value optimizations to be made just in refactoring one's code. – Venantius Apr 23 '18 at 13:45
  • 1
    just wondering: how much of the `IFn#invoke` slowness is eliminated by the JIT compiler? – Erik Kaplun Jul 08 '19 at 12:50