7

I'm trying to automatically compile / convert code written with Pythonic semantics into native and fast Javascript code.

What tools can do this, with nice debugging support possible like with Java etc?

Has anyone done this?

Why?

I'm trying to write some visualisation code with a complex main loop, a timeline, some physics simulation, and some complex interaction. I.E: it IS an actual CPU bound problem.

Writing with Javascript and testing in it's browser environment is harder to debug than say, Java, .NET or Python running in a decent IDE. But for doing actual large scale web development with complex client side code, it's necessary to at least compile to Javascript, if not directly write in it.

Background: Recent advances

Emscripten allows compiling C/C++ to Javascript, that can run with increasing efficiency in the browser due to ArrayBuffer's typed array support and new browser JS engines, as ASM.js and LLJS take advantage of Mozilla's recent speed improvements (that other venders will likely soon follow).

Altjs.org has a laundry list of Javascript alternaltives, but doesn't yet focus on the recent speed improvements or nice semantics specifically but it is becoming common place for people to code for browsers with better tools. Emscripten in particular has loads of amazing demos.

Possible options already considered:

  • Shedskin - Currently I have tried getting Shedskin working but I have limited C++/C skills (Emscripten only exposes a C API for the Boehm inspired garbage collector it uses, and Shedskin needs a C++ garbage collection class for it's objects, which doesn't exist yet).
  • Unladen Swallow / RPython, to LLVM - have not been able to setup correctly on Ubuntu yet
  • Boo to Java then to LLVM (not been able to setup on my Ubuntu system yet)

Additional constraints:

  • I need to use this on my Ubuntu system.
  • The compiled Javascript should probably be less than 1 MB
  • Debugging in the native language which is also cross compiled, should still be possible, allowing taking advantage of existing debug tools.

"This process of constructing instruction tables should be very fascinating. There need be no real danger of it ever becoming a drudge, for any processes that are quite mechanical may be turned over to the machine itself." -- Alan M. Turing, 1946

Luke Stanley
  • 1,274
  • 1
  • 16
  • 32
  • Your motivation ("why") seems to be covered very well by [Google Web Toolkit](https://developers.google.com/web-toolkit/overview) (albeit with Java, not Python), have you given it a thought? – Oak Mar 23 '13 at 15:03
  • Yes, I've given tools like GWT and PlayN consideration but they don't yet support ASM.js optimisation. @Oak – Luke Stanley Mar 23 '13 at 15:16

2 Answers2

2

You want a high level dynamic language that compiles down to efficient low level JavaScript? There is no such thing. If dynamic languages were fast we would not need asm.js in the first place.

If you want to write code that compiles to efficient JavaScript your will have to learn a lower level language. The reason why Emscripten is fast is because it compiles from a low level language (C/C++) that allows for greater compiler optimization than regular JavaScript. That is also the reason why asm.js and LLVM can be faster. They get their speed from not have dynamic types, garbage collection (this specifically is what makes it possible to use ArrayBuffer for memory) and other high-level features.

Bottom line is. There exists no tools for compiling a language with Pythonic semantics into native and fast Javascript code. And depending on what you mean by semantics it is unlikely that such a thing will appear since Python is a slow language in itself.

The best option right now for generating fast JavaScript is Emscripten. You could also consider LLJS or writing fast JavaScript by hand (Chrome has debugging tools for this).

Also, considering the title of your question you are very concerned about syntax. You should not be. When choosing the right language for the job the syntax is one of the least important factors.

paldepind
  • 4,680
  • 3
  • 31
  • 36
  • No offence but you must mean there is no such thing to your knowledge. Which is quite different... Looks like Shedskin which can be 200x faster than regular Python may be do-able, so your point is not correct and echoes what I've already pointed out. Shedskin is not the only way either, it's only one of several attempts. – Luke Stanley Mar 25 '13 at 16:01
  • I was talking about high level dynamic languages which Shedskin is not (no dynamic typing, etc.). It's definitely true that a restricted subset of Python could compile to fast code. And yes, obviously I'm only talking to the extend of my current knowledge and the fact that asm.js is still a very young project. – paldepind Mar 25 '13 at 20:46
  • You seem sure it's not possible to implement a dynamic language that could act as if things were static and be efficient. Your mental model for where the delays lay is just a model. The truth is programs and compilers especially can do all kinds of things to make things fast. Do you know about PyPy for example? There isn't one way to make a language turn into raw assembly language, there are many ways. Also, Shedskin can actually do at least some dynamic typing. – Luke Stanley Mar 26 '13 at 00:14
  • 1
    "You seem sure it's not possible to implement a dynamic language that could act as if things were static and be efficient." - Then you're not programming in a dynamic language anymore. Anyway, your question puzzles me. You claim to have a task at hand that requires fast JS, yet you refuse to use any of of the solutions available today for creating fast JS. That isn't going to get you far. – paldepind Mar 26 '13 at 08:58
  • Shedskin is sufficiently dynamic, for example, call it what you like. Emscripten (like LLJS) IS available today, and it's able to run faster than anything else that I know of... I'm not afraid to push the envelope a little to get a better overall solution that's nicer to develop with even if it takes a bit of time to figure out the build process. What makes you think you can know it's not going to get me far? – Luke Stanley Mar 26 '13 at 22:51
1

Since you mentioned shedskin yourself, I would imaging that you can share some of your experience (and explain what exactly in your opinion shedskin is missing, except that its input is a restricted python grammar). I could also assume that Cython/Pyrex are not acceptable (due too grammar restrictions).

If shedskin is too much in alpha stage for you, then you might be looking for something like Numba project, which includes a compiler of dynamic python into LLVM as well as llvm-py which allows to link LLVM exposed bytecode similar as ctypes allows to link shared-libraries and build LLVM IR compilers.

Here is a cut from the blog where it is shown how one can use Numba as JIT for numpy (incl. performance comparison with equivalent Cython code):

import numpy as np
from numba import double
from numba.decorators import jit

@jit(arg_types=[double[:,:], double[:,:]])
def pairwise_numba(X, D):
    M = X.shape[0]
    N = X.shape[1]
    for i in range(M):
        for j in range(M):
            d = 0.0
            for k in range(N):
                tmp = X[i, k] - X[j, k]
                d += tmp * tmp
            D[i, j] = np.sqrt(d)

Emscripten should allow you to expose and call your python -> llvm -> JS code as described here: https://github.com/kripken/emscripten/wiki/Interacting-with-code

Yauhen Yakimovich
  • 13,635
  • 8
  • 60
  • 67
  • Interesting idea. Shedskin may indeed be usable. I Googled for existing Emscripten Numba attempts and didn't find any. So would this require Python to be compiled to LLVM as well as Numba's output to be LLVM? That could be quite a big output couldn't it? – Luke Stanley Jul 23 '13 at 05:32