12

Operating System: macOS Monterey 12.6 Chip: Apple M1 Python version: 3.11.1

I try:

pip3 install gensim

The install process starts well but fatally fails towards the end while running 'clang'. The error message is:

clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -I/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/core/include -c gensim/models/word2vec_inner.c -o build/temp.macosx-10.9-universal2-cpython-311/gensim/models/word2vec_inner.o
      gensim/models/word2vec_inner.c:217:12: fatal error: 'longintrepr.h' file not found
        #include "longintrepr.h"
                 ^~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

This issue is raised in a couple of github postings and is attributed to some incompatibility between cython and python 3.11. However, no suggestion is forwarded as to what users should do until cython is updated. I may have misrepresented the details of the discussions on github but I think this is the gist of it.

Can anyone help me in installing gensim in the meantime?

Thanks.

I updated cython and aiohttp. The latter because I had seen a post where the aiohttp install failed for the same reason as mine (missing "longintrepr.h").

No improvement. "pip install gensim" still fails and fails with the same message as copied above.

Halim Gurgenci
  • 135
  • 1
  • 1
  • 5

3 Answers3

11

I also faced the same issue for gensim library on Windows laptop while using Python 3.11.1 Changing to the Python 3.10 worked for me.

Arya Stark
  • 205
  • 1
  • 11
7

It seems your issue may be due to the specifics of a fairly-new Python, and lagging library support, on a somewhat new system (a MacOS M1 machine) which has its own somewhat-unique build toolchains.

Unless you absolutely need to use Python 3.11.1, I'd suggest using Gensim within a Python environment with a slightly-older Python interpreter, where the various pacakges you truly need may be a little more settled. For example, on many OS/architecture/Python combinations, a standard pip install will grab precompiled libraries – so build errors of the type you're seeing can't happen.

That your installation is falling back to a local compilation (which hits a problem without an easy off-the-shelf solution) is a hint that something about the full configuration is still somewhat undersupported by one or more of the involved libraries.

If you use the conda 3rd-party system for managing Python virtual environments, it also offers you the ability to explicitly choose which Python version will be used in each environment. That is, you're not stuck with the exact version, and installed libraries, that are default/global on your OS. You could easily try Python 3.10, or Python 3.9, which might work better.

And, keeping your development/project virtual-environment distinct from the system's Python is often considered a "best practice" for other purposes, too. There's no risk that you'll cause damage to the system Python and any tools reliant on it, or face an issue where several of your Python projects need conflicting library versions. (You just use a separate environment for eah project.) And, the exercise of rigorously specifying what needs to be in your project's environment helps keep its prerequisites/dependencies clear for any future relocations/installations elsewhere.

When using the conda tool for this purpose, I usually start with the miniconda version, so that I have explicit control over exactly what packages are installed, and can thus keep each environment minimally-specified for its purposes. (The larger anaconda approach pre-installs tons of popular packages instead.)

gojomo
  • 52,260
  • 14
  • 86
  • 115
  • Further to my first comment, pursuing advice from @gojomo, I installed a virtual environment in miniconda and was able import gensim when I used Python 3.8. The pip did not work but I installed using setup.py. – Halim Gurgenci Jan 05 '23 at 03:28
4

I had a similar issue with Python3.11 while building Pycrypto on GNU/Linux. As a dirty workaround, I copied https://github.com/python/cpython/blob/main/Include/cpython/longintrepr.h to /usr/include/python3.11/longintrepr.h and it worked!

Srikant
  • 167
  • 7