2

I am working on an Oracle VirtualBox Fedora 64-bit VM with Python3.6. I have configured a venv to install apache-airflow. My pip3 is updated to 18.0, and I have installed a number of devel-tools with yum:

$ yum groupinstall "Development tools"
$ yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel python3-devel.x86_64 cyrus-sasl-devel.x86_64
$ yum install python3-psutil.x86_64
$ yum install libevent-devel

Even with these devel tools, I am getting a gcc error whenever I try to run the pip install:

$ pip3 install apache-airflow

The error and the terminal output preceding it looks like this:

running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/psutil
copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_compat.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_psosx.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/__init__.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_common.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_psposix.py -> build/lib.linux-x86_64-3.6/psutil
creating build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/runner.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.6/psutil/tests
running build_ext
building 'psutil._psutil_linux' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/psutil
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=442 -DPSUTIL_LINUX=1 -I/home/jsexauer/include -I/usr/include/python3.6m -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o
annobin: _psutil_linux.c: Error: plugin built for compiler version (8.0.1) but run with compiler version (8.1.1)
cc1: error: fail to initialize plugin /usr/lib/gcc/x86_64-redhat-linux/8/plugin/annobin.so
cc1: error: ‘-fcf-protection=full’ requires Intel CET support. Use -mcet or both of -mibt and -mshstk options to enable CET
error: command 'gcc' failed with exit status 1

It looks like it's an error with psutil or gcc, and that is why I ran the yum install for psutil, but that did not solve anything.

 $ yum install python3-psutil.x86_64

The most helpful information I found elsewhere was about a version mismatch between annobin and gcc 8.1.1. However, the annobin version I have seems to be the patched version.

I found nothing online about the '-fcf-protection=full' error. I am not sure how this would affect airflow, nor how to resolve the 'cc1: errors'.

Any help or guidance would be appreciated.

  • I had the same problem but got it fixed by installing gcc packages :- sudo yum -y install gcc gcc-c++ – sumeetgautam Oct 17 '18 at 16:06
  • So what did `yum install python3-psutil.x86_64` output? An error? If you tried `pip3 install apache-airflow` again did it show the same output you showed here? – dlamblin Oct 22 '18 at 06:20

3 Answers3

1

I had the same problem but got it fixed by Before install Apache Airflow install

sudo yum -y install gcc gcc-c++

For Me the Source was :-

https://medium.com/@0x0ece/installing-apache-airflow-on-centos-7-750c77b7aa35

sumeetgautam
  • 159
  • 1
  • 7
0

You're installing in to a virtual env, but the tool chain for building psutil is not working for you there. Also installing the pre-built psutil with yum install python3-psutil.x86_64 has no effect on your virtual env, on purpose.

You'll need to symlink in the system package of psutil you installed into your venv, approximately with: ln -s /usr/lib/python3.6/dist-packages/psutil* $VIRTUAL_ENV/lib/python*/site-packages

dlamblin
  • 43,965
  • 20
  • 101
  • 140
0

It works for me to update annobin by dnf update annobin -y, there is a related thread on bugzilla

whatacold
  • 660
  • 7
  • 15