File not found error when pip installing python-lzo

Question

I'm trying to install python-lzo on mac OS Sierra. I have used brew to install the c lzo library:

brew update
brew install lzo

I then try and install python-lzo using a virtual environment and pip:

$ virtualenv ./env
$ . ./env/bin/activate
$ pip install python-lzo

Which gives me the following output:

Collecting python-lzo
  Using cached python-lzo-1.11.tar.gz
Building wheels for collected packages: python-lzo
  Running setup.py bdist_wheel for python-lzo: started
  Running setup.py bdist_wheel for python-lzo: finished with status 'error'
  Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-wDRYjx/python-lzo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/tmp_ueOY4pip-wheel- --python-tag cp27:
  /usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running build_ext
  building 'lzo' extension
  creating build
  creating build/temp.macosx-10.11-x86_64-2.7
  clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
  lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
  #include <lzo1x.h>
           ^
  1 error generated.
  error: command 'clang' failed with exit status 1

  ----------------------------------------
  Running setup.py clean for python-lzo
Failed to build python-lzo
Installing collected packages: python-lzo
  Running setup.py install for python-lzo: started
    Running setup.py install for python-lzo: finished with status 'error'
    Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-wDRYjx/python-lzo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-8RUWxo-record/install-record.txt --single-version-externally-managed --compile:
    /usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
      warnings.warn(msg)
    running install
    running build
    running build_ext
    building 'lzo' extension
    creating build
    creating build/temp.macosx-10.11-x86_64-2.7
    clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
    lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
    #include <lzo1x.h>
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1

    ----------------------------------------

The lzo1x.h file does exist here: /usr/local/opt/lzo/include/lzo/ or with the brew command:

$ ls $(brew --prefix lzo)/include/lzo/

I have then tried to set the CFLAGS envronment variables, but still have the same error:

$ export EXTRA_CFLAGS=-I/usr/local/opt/lzo/include && export EXTRA_CXXFLAGS=-I/usr/local/opt/lzo/include && export EXTRA_LDFLAGS=-L/usr/local/opt/lzo/lib && pip install

And in desperation:

$ export EXTRA_CFLAGS=-I/usr/local/opt/lzo/include/lzo && export EXTRA_CXXFLAGS=-I/usr/local/opt/lzo/include/lzo && export EXTRA_LDFLAGS=-L/usr/local/opt/lzo/lib && pip install python-lzo

Still the same error message, any ideas?

-- Updated--
Also tried passing the variables directly into pip:

pip install --global-option='build_ext' --global-option='-I/usr/local/opt' --global-option='-I/usr/local/opt/lzo/include/lzo' --global-option='-L/usr/local/opt/lzo/lib' python-lzo

This time I get a slightly different error:

  cmdoptions.check_install_build_global(options)
Collecting python-lzo
  Using cached python-lzo-1.11.tar.gz
Skipping bdist_wheel for python-lzo, due to binaries being disabled for it.
Installing collected packages: python-lzo
  Running setup.py install for python-lzo ... error
    Complete output from command /Users/myuser/dir/devel/lzo_example/env/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-14vN3V/python-lzo/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" build_ext -I/usr/local/opt -I/usr/local/opt/lzo/include/lzo -L/usr/local/opt/lzo/lib install --record /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-Gq5pjs-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/emson/Dropbox/devel/aws/coursenut_bigdata/lzo_example/env/bin/../include/site/python2.7/python-lzo:
    /usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
      warnings.warn(msg)
    running build_ext
    building 'lzo' extension
    creating build
    creating build/temp.macosx-10.11-x86_64-2.7
    clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/opt/lzo/include/lzo -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
    In file included from lzomodule.c:35:
    /usr/local/opt/lzo/include/lzo/lzo1x.h:33:10: fatal error: 'lzo/lzoconf.h' file not found
    #include <lzo/lzoconf.h>
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1

Many thanks

-- SUCCESS ---

Big thanks @daphtdazz your solution (see below) worked! I have modified it slightly to use the brew --cellar command to get the location of the lzo files:

export C_INCLUDE_PATH=$(brew --cellar lzo)/2.09/include/lzo:$(brew --cellar lzo)/2.09/include/ export LIBRARY_PATH=/usr/local/lib pip install python-lzo

daphtdazz · Accepted Answer · 2016-11-08T13:37:29.523

I got the same results as you, and eventually this worked for me:

export C_INCLUDE_PATH=/usr/local/Cellar/lzo/2.09/include/lzo:/usr/local/Cellar/lzo/2.09/include 
export LIBRARY_PATH=/usr/local/lib 
pip install python-lzo

(Obviously you may need to tweak those depending on what version of the lzo brew distribution you have installed.)

Which answered the question for me, but I don't really understand why those aren't configured correctly...

I have the Xcode developer tools installed though as well, and I think they've conflicted with brew and pip before, so maybe that's it. I'll keep having a look and if I work it out I'll update this answer.

Edit

Having a look at the setup.py file for python-lzo it seems there's a bit of weirdness going on, in particular I'm concerned that on Windows both include paths are being added, which suggests the lzo source distribution isn't very well arranged. But there doesn't seem to be a designed way of achieving this on macOS with a brew install, so I think the include part of it at least is best solved with the extra environment variable.

Fantastic that works! Thanks so much. I modified your version slightly to use the `brew --cellar` command thus: `export C_INCLUDE_PATH=$(brew --cellar lzo)/2.09/include/lzo:$(brew --cellar lzo)/2.09/include/;export LIBRARY_PATH=/usr/local/lib;pip install python-lzo` — emson, Nov 09 '16 at 11:59

Soner from The Ottoman Empire · Answer 2 · 2022-10-01T11:34:42.337

Excerpted from here, on M1 chip

# Install lzo homebrew 
arch -arm64 brew install lzo

# Configure include path and lib path
export C_INCLUDE_PATH=/opt/homebrew/Cellar/lzo/2.10/include/lzo:/opt/homebrew/Cellar/lzo/2.10/include/
export LIBRARY_PATH=/opt/homebrew/lib

# Install
pip3 install python-lzo

Installing collected packages: python-lzo
  Running setup.py install for python-lzo ... done
Successfully installed python-lzo-1.12

Here is my log,

soner@Soners-MacBook-Pro ~ % arch -arm64 brew install lzo
==> Downloading https://ghcr.io/v2/homebrew/core/lzo/manifests/2.10
Already downloaded: /Users/soner/Library/Caches/Homebrew/downloads/d4aa5b0c239912c53bc857d1012c6b7feb4acb509618f5e100f95bf8521f08e7--lzo-2.10.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/lzo/blobs/sha256:e16072e8ef7a88
Already downloaded: /Users/soner/Library/Caches/Homebrew/downloads/2e2528259c080227ff9fcce6ab15d0f0415a85dad7f909d9a803d1f863f43960--lzo--2.10.arm64_monterey.bottle.tar.gz
==> Pouring lzo--2.10.arm64_monterey.bottle.tar.gz
  /opt/homebrew/Cellar/lzo/2.10: 31 files, 565.6KB
==> Running `brew cleanup lzo`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
soner@Soners-MacBook-Pro ~ % export C_INCLUDE_PATH=/opt/homebrew/Cellar/lzo/2.10/include/lzo:/opt/homebrew/Cellar/lzo/2.10/include/
soner@Soners-MacBook-Pro ~ % export LIBRARY_PATH=/opt/homebrew/lib
soner@Soners-MacBook-Pro ~ % pip3 install python-lzo
Collecting python-lzo
  Using cached python-lzo-1.14.tar.gz (14 kB)
  Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for python-lzo, since package 'wheel' is not installed.
Installing collected packages: python-lzo
  Running setup.py install for python-lzo ... done
Successfully installed python-lzo-1.14

score 0 · Answer 3 · answered Nov 04 '16 at 13:51

0

I don't use MacOS, but can you try giving the env. variables directly to PIP?

So, same commands as you did in the second try (without exports), but without the &&.

foo=bar pim=pam pip install ...

Also, have you checked that the missing file exists in the directories you include?

answered Nov 04 '16 at 13:51

Michael Hooreman

582
1
5
16

Thanks Michael.... I've tried passing them in as global-option parameters e.g. pip install --global-option='build_ext' --global-option='-I/usr/local/opt' --global-option='-I/usr/local/opt/lzo/include/lzo' --global-option='-L/usr/local/opt/lzo/lib' python-lzo but still get a similar error - SEE UPDATED ABOVE – emson Nov 04 '16 at 17:51

File not found error when pip installing python-lzo

3 Answers3