0

I'm trying to build a Python 2.7 from source on a pretty minimal system (Ubuntu 16.04 docker image) which doesn't cause HTTPS downloads to fail due to <urlopen error unknown url type: https>. I don't want to use a package manager, but am aware that this would solve the problem easily.

I went through Unknown url type error in urllib2 and applied the different referenced patches to setup.py, set the SSL variable in Modules/Setup and Modules/Setup.dist to the installation prefix which I passed to ./config --prefix of OpenSSL 1.1.1 and set LD_LIBRARY_PATH to the OpenSSL installation prefix.

I suspect that one such a minimal system more things have to be fixed than suggested in the other answer. Why is there no --with-ssl switch for configure?

The build script is

apt-get update && apt-get install --yes wget xz-utils make gcc patch
wget http://www.cpan.org/src/5.0/perl-5.26.1.tar.gz && tar xf perl-5.26.1.tar.gz && cd perl-5.26.1 && ./configure.gnu && make -j16 && make install && cd ..
wget https://ftp.gnu.org/pub/gnu/gettext/gettext-0.19.8.1.tar.xz && tar xf gettext-0.19.8.1.tar.xz && cd gettext-0.19.8.1 && ./configure && make -j16 && make install && cd ..
wget https://www.zlib.net/zlib-1.2.11.tar.gz && tar xf zlib-1.2.11.tar.gz && cd zlib-1.2.11 && ./configure && make -j16 && make install && cd ..
wget https://www.kernel.org/pub/software/scm/git/git-2.13.3.tar.gz && tar xf git-2.13.3.tar.gz && cd git-2.13.3 && ./configure --with-perl=/usr/local/bin/perl && make -j16 && make install && cd ..
            # need `--with-perl` passed to `configure` because the `/usr/local/bin/perl` isn't picked up regardless where it's found in PATH, see https://stackoverflow.com/questions/48931247/cant-locate-extutils-makemaker-pm-in-inc-during-git-build for details
wget https://www.openssl.org/source/openssl-1.1.1-pre1.tar.gz && tar xf openssl-1.1.1-pre1.tar.gz && cd openssl-1.1.1-pre1 && ./config --prefix=/usr/local/openssl && make -j16 && make install && cd ..
            # need to install into explicit prefix different from `/usr/local` in order to make OpenSSL get picked up by Python's patched `configure` (specifying `OPENSSL_ROOT=/usr/local` doesn't work)
wget https://www.python.org/ftp/python/2.7.14/Python-2.7.14.tgz && tar xf Python-2.7.14.tgz && cd Python-2.7.14 && wget https://gist.githubusercontent.com/rkitover/2d9e5baff1f1cc4f2618dee53083bd35/raw/7f33fcf5470a9f1013ac6ae7bb168368a98fe5a0/python-2.7.14-custom-static-openssl.patch && patch -p1 <python-2.7.14-custom-static-openssl.patch && env OPENSSL_ROOT=/usr/local/openssl ./configure --disable-shared && make -j16 && make install && cd ..
git clone git://git.gnome.org/jhbuild && cd jhbuild && ./autogen.sh --prefix=/usr/local && make && make install && cd ..
env JHBUILD_RUN_AS_ROOT= jhbuild bootstrap

I chose jhbuild bootstrap as test command because that's the desired outcome, feel free to suggest a better one.

I created a SSCCE at https://gitlab.com/krichter/jhbuild-docker-build which allows to run the build script in GitLab CI and an example run including a full build log can be investigated at https://gitlab.com/krichter/jhbuild-docker-build/-/jobs/53948513.

Kalle Richter
  • 8,008
  • 26
  • 77
  • 177
  • 1
    When you configure all of the packages you are building, you should add an `RPATH` so the same libraries are used at runtime. I would expect to see all of the configure's to include something like `LDFLAGS += -L/usr/local/lib -Wl,-R,/usr/local/lib -Wl,--enable-new-dtags`. – jww Feb 23 '18 at 14:40
  • @jww Thank you for the hint. I started learning about rpath just now and am not capable yet of fully understanding what you mean by "I would expect...". Do you mean I have to patch `config` and `configure` of OpenSSL and Python? Or run `./configure --prefix=... LDFLAGS+='-L/usr/local/lib -Wl,-R,/usr/local/lib -Wl,--enable-new-dtags'`? – Kalle Richter Feb 23 '18 at 16:31
  • Do I need to add something to the patch of `Modules/Setup.dist`? – Kalle Richter Feb 23 '18 at 16:43
  • Checkout [Noloader | Build-Scripts](https://github.com/noloader/Build-Scripts). The build scripts do similar to what you are trying to do, but it does them with Git, SSH, Wget and several other programs. It even has to patch source files and configure files at times. It does not build PHP but it should be easy enough to do so. – jww Feb 23 '18 at 16:53
  • I'm doing the above to get GNOME's jhbuild running for which I already have some module scripts defining dependencies which I want to use. A working Python/Python capable of duing HTTPS downloads is the only thing left... It's a valuable link, though. Could it make any sense to add `--no-ssl2 --no-ssl3 --no-comp --shared -DNDEBUG --enable-ec_nistp_64_gcc_128` to OpenSSL's `config`? – Kalle Richter Feb 23 '18 at 17:01
  • 1
    You should consider joining [Build-Scripts](https://github.com/noloader/Build-Scripts) as a collaborator. We seem to have a lot of overlap. The scripts have been around for a while so they are fairly mature. They are also cross-platform. I think it would be a lot easier for you to add Python to the build scripts as opposed to reinventing and repairing the the wheel. For example, the missing runpaths and dtags, and missing options to OpenSSL and Git. – jww Feb 23 '18 at 17:36
  • @jww Good idea. I started with https://github.com/noloader/Build-Scripts/issues/1 (back to basics...). Thanks for the connection. – Kalle Richter Feb 23 '18 at 17:44

1 Answers1

0

I can get it to fail with <urlopen error [SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:661)> (which I deal with at FIXME) after

  • adding --prefix=/usr/local/ --openssldir=/usr/local to config of OpenSSL,
  • applying https://gist.githubusercontent.com/rkitover/2d9e5baff1f1cc4f2618dee53083bd35/raw/7f33fcf5470a9f1013ac6ae7bb168368a98fe5a0/python-2.7.14-custom-static-openssl.patch as explained in the referenced question above and

  • patching Modules/Setup.dist and Modules/Setup with

    --- Modules/Setup.dist  2018-02-23 01:12:46.537415323 +0100
+++ Modules/Setup.dist.d    2018-02-23 01:44:45.235092890 +0100
@@ -215,10 +215,10 @@

 # Socket module helper for SSL support; you must comment out the other
 # socket line above, and possibly edit the SSL variable:
-#SSL=/usr/local/ssl
-#_ssl _ssl.c \
-#  -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-#  -L$(SSL)/lib -lssl -lcrypto
+SSL=/usr/local
+_ssl _ssl.c \
+   -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
+   -L$(SSL)/lib -lssl -lcrypto

 # The crypt module is now disabled by default because it breaks builds
 # on many systems (where -lcrypt is needed), e.g. Linux (I believe).

    (patch -u -p0 <somename.patch)

A example build overcoming the initial error can be found at https://gitlab.com/krichter/jhbuild-docker-build/-/jobs/53978734.

Kalle Richter
  • 8,008
  • 26
  • 77
  • 177
  • You should add the additional information to the question; and not as an answer. – jww Feb 23 '18 at 14:37
  • This solves the issue. Since python's build process is pretty close to horrible, I expect a number of build issues following the new build issue and it's not worth to destroy the information I collected over one day by editing it away or making the question overly long. – Kalle Richter Feb 23 '18 at 14:41