Installing Airflow with pip

Question

I'm trying to install apache airflow with pip, so I enter "pip install apache-airflow". but somehow i got an error that i don't understand. Could you please help me with this?

for a little bit context, I'm using macOS catalina and python 3.8.2.

I have tried to upgrade my pip, but the error still there.

These are the error that appear

ERROR: Command errored out with exit status 1:
 command: /Users/muhammadsyamsularifin/airflow/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-install-ruyjyg1t/setproctitle/setup.py'"'"'; __file__='"'"'/private/tmp/pip-install-ruyjyg1t/setproctitle/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-8qj0qv0h/install-record.txt --single-version-externally-managed --compile --install-headers /Users/muhammadsyamsularifin/airflow/venv/include/site/python3.8/setproctitle
     cwd: /private/tmp/pip-install-ruyjyg1t/setproctitle/

Complete output (119 lines):
running install
running build
running build_ext
building 'setproctitle' extension
creating build
creating build/temp.macosx-10.14.6-x86_64-3.8
creating build/temp.macosx-10.14.6-x86_64-3.8/src
xcrun -sdk macosx clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DSPT_VERSION=1.1.10 -D__darwin__=1 -I/Users/muhammadsyamsularifin/airflow/venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c src/setproctitle.c -o build/temp.macosx-10.14.6-x86_64-3.8/src/setproctitle.o
In file included from src/setproctitle.c:14:
In file included from src/spt.h:15:
In file included from src/spt_python.h:14:
In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/limits.h:21:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h:63:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from src/setproctitle.c:14:
In file included from src/spt.h:15:
In file included from src/spt_python.h:14:
In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/limits.h:21:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h:64:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/limits.h:8:2: error: architecture not supported
#error architecture not supported
 ^
In file included from src/setproctitle.c:14:
In file included from src/spt.h:15:
In file included from src/spt_python.h:14:
In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from src/setproctitle.c:14:
In file included from src/spt.h:15:
In file included from src/spt_python.h:14:
In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:27:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_dev_t;         /* dev_t */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:62:9: error: unknown type name '__uint64_t'
typedef __uint64_t      __darwin_ino64_t;       /* [???] Used for 64 bit inodes */
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t'
typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__uint128_t'?
typedef __uint16_t      __darwin_mode_t;        /* [???] Some file attributes */
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:71:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_off_t;         /* [???] Used for file sizes */
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_pid_t;         /* [???] process and group IDs */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_sigset_t;      /* [???] signal set */
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_suseconds_t;   /* [???] microseconds */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_uid_t;         /* [???] user IDs */
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_useconds_t;    /* [???] microseconds */
        ^
note: '__uint128_t' declared here
In file included from src/setproctitle.c:14:
In file included from src/spt.h:15:
In file included from src/spt_python.h:14:
In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:43:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_wctype_t;
        ^
note: '__uint128_t' declared here
In file included from src/setproctitle.c:14:
In file included from src/spt.h:15:
In file included from src/spt_python.h:14:
In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:75:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_va_list.h:31:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/types.h:37:2: error: architecture not supported
#error architecture not supported
 ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command 'xcrun' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/muhammadsyamsularifin/airflow/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-install-ruyjyg1t/setproctitle/setup.py'"'"'; __file__='"'"'/private/tmp/pip-install-ruyjyg1t/setproctitle/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-8qj0qv0h/install-record.txt --single-version-externally-managed --compile --install-headers /Users/muhammadsyamsularifin/airflow/venv/include/site/python3.8/setproctitle Check the logs for full command output.

@hoefling : this worked for me when I exported the ARCHFLAGS env, thanks a lot — vdolez, Nov 16 '20 at 15:32
@hoefling : i'm unfortunatly still unable to run airflow --version — vdolez, Nov 16 '20 at 15:41
@vdolez did you run `airflow initdb` after initial installation? Check out my other answer here: https://stackoverflow.com/a/57515985/2650249 — hoefling, Nov 16 '20 at 17:05

score 1 · Answer 1 · answered Nov 17 '20 at 01:17

I looked into similar errors and here are a few possible fixes:

If you installed Python3.8 via Brew, try to uninstall it and install a new version that you build from source.
Try sudo python3.8 -m pip install apache-airflow.
Upgrade to Python 3.8.5 as per this post.
Export the environment variable export ARCHFLAGS="-arch x86_64" as per this post.

score 0 · Answer 2 · answered Nov 17 '20 at 14:05

I faced exact same issue, here's how I solved it.

I explicitly used python 3.8.5 as pointed in Meghdeep Ray's answer

I also exported the export ARCHFLAGS="-arch x86_64" environment variable still inspired by Meghdeep Ray's answer

But the important part for me was to install apache-airflow from pip using the following command :

pip install apache-airflow[]==1.10.12 \
     --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

The constraint is important as pointed in this github issue

Installing Airflow with pip

2 Answers2