0

I'm trying to compile PyPy for use on an OpenWrt configuration, but I am having a really hard time doing it.

My main problems are:

  • Each time I change the Makefile I am forced to start the translating process of PyPy again. Is there a way to avoid this?

  • Would copying just the compiled pypy-c and lib*.so binaries do it, or would I have to copy everything from the compiled files of PyPy?
    Here is the directory structure after running make on the files.

  • How do I specify the version of GCC to use? I've tried to do this without success.

  • How would I get the interpreter to run after installing it on OpenWrt?

  • One can also compile PyPy using PyPy itself, which gives a .tar file with a different structure (no Makefile, pypy executable etc). Can I use that instead of building it from the source?

Here is my Makefile.

include $(TOPDIR)/rules.mk

PKG_NAME:=PyPy
PKG_VERSION:=5.8.0
PKG_RELEASE:=1
PKG_MD5SUM:=504c2d522595baf8775ae1045a217a2b120732537861d31b889d47c340b58bd5

PKG_SOURCE_NAME:=pypy2
PKG_SOURCE_URL:=https://bitbucket.org/pypy/pypy/downloads/
PKG_SOURCE:=$(PKG_SOURCE_NAME)-v$(PKG_VERSION)-src.tar.bz2
PKG_BUILD_DEPENDS:=python
PKG_CAT:=bzcat


PKG_BUILD_DIR:=$(BUILD_DIR)/$(BUILD_VARIANT)$(PKG_SOURCE_NAME)-v$(PKG_VERSION)-src

include $(INCLUDE_DIR)/package.mk
$(call include_mk, python-package.mk)

define Package/PyPy
    SECTION:=utils
    CATEGORY:=Utilities
    TITLE:=PyPy
    URL:=https://pypy.org/index.html
    DEPENDS:=+libffi +libexpat +libunwind

endef

define Package/PyPy/description
    PyPy is an alternate implementation of the Python programming language written in Python. 
    This distribution is for Linux architecture, using Python 2.

endef

define Package/PyPy/install
    $(INSTALL_DIR) $(1)/usr/bin
    $(INSTALL_BIN) $(PKG_BUILD_DIR)/pypy-c $(1)/usr/bin
    $(CP) (PKG_BUILD_DIR)/*.so $(1)/usr/lib
    $(INSTALL_BIN) $(PKG_BUILD_DIR)/pypy/bin/*  $(1)/usr/bin

endef

define Build/Compile
    $(MAKE) -C $(PKG_BUILD_DIR)
endef

$(eval $(call BuildPackage,PyPy))
HGitere
  • 37
  • 2
  • 10

2 Answers2

0

I would need to see the entire process: the part of the Makefile above, the other pieces such as rules.mk, package.mk, command line arguments and environmental variables in order to understand what is not working

  1. The Makefile in question is not supported, the developers do not use it, and as you discovered it does not work well. As described in the build page, building PyPy from source is a four part process, and the Makefile mashes three of those stages together so any changes currently require restarting from 0. Note that the underlying build process uses PYPY_USESSION_DIR, PYPY_USESSION_BASENAME, and a numbering system to ensure that each time the first stage (translation from RPython to C) puts the results in a new, clean directory. For this reason if you restart the process you will lose your previous work.

  2. Python is both a binary interpreter and a stdlib of "battery included" modules. If you wish to use the binary interpreter, you need to install the binary and the stdlib support files together since they work as a unit. That is why we specify a fourth step in the build page, packaging. Please read that section carefully for methods of using the interpreter+stdlib

  3. PyPy does not really support cross compiling, one method that used to work is described in the documentation of RPython. PyPy does support native gcc and/or clang. What compiler are you trying to use on what hardware?

  4. Run the interpreter as you would any python interpreter; specify the path to the executable file, which (as stated above) needs to know how to find the support files such as the stdlib and the site-packages directory of third-party modules installed specifically for PyPy.

  5. Using PyPy to build PyPy should be no different than using CPython to build PyPy, except it will be much faster. The differences you see must be an artifact of how you are building.

I would recommend you NOT use the unsupported Makefile until you understand the build, compile, package, install process, and then once you have a working installation help the PyPy project improve the process until it can be automated into a Makefile

mattip
  • 2,360
  • 1
  • 13
  • 13
  • Here is [rules.mk](https://github.com/openwrt/openwrt/blob/master/rules.mk) and [package.mk.](https://github.com/openwrt/openwrt/blob/master/include/package.mk) I'm not sure how I could show you the entire process unless you do it yourself, but hopefully those help. As for number five, I followed the guide instead of using the Makefile, and after packaging into a .tar and unpacking it, it had a totally different directory structure like I described. Maybe I'm doing it wrong? :\. I had another iteration that was able to compile in OpenWrt (continued) – HGitere Aug 04 '17 at 06:33
  • until the linking phase where it could not locate certain OpenWrt specific libraries. I will look into both the binary distributions and the cross translating. I'm trying to use GCC 5.3 on Linux, and my hardware specs are higher than specified on the building guide page. – HGitere Aug 04 '17 at 06:40
  • the reason I'm trying to build is because my operating system (OpenWrt) is does not have it's own binary. – HGitere Aug 04 '17 at 08:03
  • Unpacking the tar file should give you a functional python environment, i.e. running ``path/to/bin/pypy`` should give you a python REPL. The directory structure is a subset of the complete source distribution. I assume you mean your hardware is X86 based? – mattip Aug 04 '17 at 08:50
  • yes, running `~/bin/pypy` does give me a python REPL. I think I understand what you mean. But I still cannot use the unpacked environment for OpenWrt due to the missing libraries I mentioned earlier. Trying to install an already build binary leads to an error that it's missing dependencies such as `libc.so.1, libssl.so.1` etc, which I understand are OS specific? Yes, my hardware is x86 based. – HGitere Aug 04 '17 at 11:14
0

If you are running OpenWRT on supported hardware and operating system, you would be much happier using a binary distribution and not compiling from source. In this case your Makefile should download a binary distribution and simply install it.

If you are using a different CPU, chances are PyPy will not work out-of-the-box on your hardware, you will have to run tests to make sure the JIT will actually make things faster.

mattip
  • 2,360
  • 1
  • 13
  • 13