2

I want to plot CPU intensive functions with matplotlib. I would like to use pypy but it is not compatible with matplotlib. The plotting itself is not CPU intensive and does not need to be accelerated. I wonder if there is a way to call a pypy function from C-python. Could I (ab)use the multiprocessing module and say set_executable("/.../pypy") from C-python?

Olivier Esser
  • 431
  • 1
  • 4
  • 12

3 Answers3

2

You could write the output of your computation to stdout and pipe it to another program that reads the data to plot from stdin, like so:

pypy compute.py | python plot.py

As the intermediary format, you could use a format like JSON which are available in the standard library of both pypy and cpython and can convert from and to python primitives easily.

Alternatively, the compute.py could just use the subprocess module to start plot.py in CPython instead of relying on the pipe being arranged by the shell.

Alternatively, you could use pickle which can preserve more information about python objects, but be careful since pickle isn't a well standardized format.

If you use a recent enough version of CPython and Pypy, another alternative you can try is the multiprocessing.connection module's Listener and Client classes. Note that cross python implementation Listener and Client was broken in some older version of Python and Pypy (multiprocessing Listeners and Clients between python and pypy). The multiprocessing.manager module probably would also work across different cpython and pypy.

Community
  • 1
  • 1
Lie Ryan
  • 62,238
  • 13
  • 100
  • 144
  • These days we'd use cffi to interface with pypy code from cpython. Much faster and neater. https://cffi.readthedocs.io/en/latest/embedding.html – Eric Sep 04 '21 at 17:28
0

If you have CPU intensive functions the best bet is to:

A) See if numpy can help,

B) See if the functions can be optimised in python,

C) Consider writing C extension for your function.

D) Consider embedding cpython and matplotlib in pypy as demonstrated here.

Steve Barnes
  • 27,618
  • 6
  • 63
  • 73
  • A) No. These are algorithmic functions (I play with prime numbers). B) Pypy is usually strong in optimizing algorithmic functions. C) should work but it is time consuming and more difficult. I am in the situation when pypy should be fine to compute the function and C-python should be fine for the plotting. The problem is simply to connect the two. – Olivier Esser Dec 24 '13 at 07:41
  • Added link to possible solution. – Steve Barnes Dec 24 '13 at 07:56
  • But the first sentence of this is somewhat scaring. (Just a proof of concept, barely works, etc.). Moreover it seems you have to recompile pypy with some options disabled. Using the multiprocessing module should be fine but I am still learning python. If there are no caveats, surely someone has already done it. – Olivier Esser Dec 24 '13 at 08:16
0

You may use execnet. There are several exmpales of Connecting different Python interpreters

Mohammad Alhashash
  • 1,543
  • 1
  • 14
  • 30