I followed the steps on the website https://github.com/facebookresearch/fairseq to build the fairseq. When I tried to execute pip install --editable ./, I met the following error:
Obtaining file:///data_new/private/liuyadi/myenv/fairseq
Checking if build backend supports build_editable ... done
Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: cython in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (0.29.33)
Requirement already satisfied: regex in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (2022.10.31)
Requirement already satisfied: cffi in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (1.15.1)
Requirement already satisfied: tqdm in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (4.64.1)
Requirement already satisfied: numpy in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (1.24.2)
Requirement already satisfied: torch in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (1.10.1)
Requirement already satisfied: sacrebleu in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (2.3.1)
Requirement already satisfied: editdistance in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from fairseq==0.9.0) (0.6.2)
Requirement already satisfied: pycparser in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from cffi->fairseq==0.9.0) (2.21)
Requirement already satisfied: colorama in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from sacrebleu->fairseq==0.9.0) (0.4.6)
Requirement already satisfied: portalocker in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from sacrebleu->fairseq==0.9.0) (2.7.0)
Requirement already satisfied: lxml in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from sacrebleu->fairseq==0.9.0) (4.9.2)
Requirement already satisfied: tabulate>=0.8.9 in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from sacrebleu->fairseq==0.9.0) (0.9.0)
Requirement already satisfied: typing-extensions in /data_new/private/liuyadi/myenv/lib/python3.8/site-packages (from torch->fairseq==0.9.0) (4.5.0)
Building wheels for collected packages: fairseq
Building editable for fairseq (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building editable for fairseq (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [125 lines of output]
No CUDA runtime is found, using CUDA_HOME='/home/liuyadi/miniconda3'
running editable_wheel
creating /tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info
writing /tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/dependency_links.txt
writing entry points to /tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/entry_points.txt
writing requirements to /tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/requires.txt
writing top-level names to /tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/top_level.txt
writing manifest file '/tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file '/tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq.egg-info/SOURCES.txt'
creating '/tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq-0.9.0.dist-info'
creating /tmp/pip-wheel-l0im9_dq/.tmp-xmlbxq8o/fairseq-0.9.0.dist-info/WHEEL
running build_py
running build_ext
skipping 'fairseq/data/data_utils_fast.cpp' Cython extension (up-to-date)
skipping 'fairseq/data/token_block_utils_fast.cpp' Cython extension (up-to-date)
building 'fairseq.libbleu' extension
creating /tmp/tmpn5d7f_wt.build-temp/fairseq
creating /tmp/tmpn5d7f_wt.build-temp/fairseq/clib
creating /tmp/tmpn5d7f_wt.build-temp/fairseq/clib/libbleu
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/data_new/private/liuyadi/myenv/include -I/usr/include/python3.8 -c fairseq/clib/libbleu/libbleu.cpp -o /tmp/tmpn5d7f_wt.build-temp/fairseq/clib/libbleu/libbleu.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=libbleu -D_GLIBCXX_USE_CXX11_ABI=0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/data_new/private/liuyadi/myenv/include -I/usr/include/python3.8 -c fairseq/clib/libbleu/module.cpp -o /tmp/tmpn5d7f_wt.build-temp/fairseq/clib/libbleu/module.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=libbleu -D_GLIBCXX_USE_CXX11_ABI=0
fairseq/clib/libbleu/module.cpp:9:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/torch/utils/cpp_extension.py:381: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/torch/utils/cpp_extension.py:323: UserWarning:
!! WARNING !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++ 4.8.5) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 5.0 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.
See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 5 or higher.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! WARNING !!
warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler))
Traceback (most recent call last):
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
spawn(cmd, dry_run=self.dry_run, **kwargs)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
raise DistutilsExecError(
distutils.errors.DistutilsExecError: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 147, in run
self._create_wheel_file(bdist_wheel)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 328, in _create_wheel_file
files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 259, in _run_build_commands
self._run_build_subcommands()
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/command/editable_wheel.py", line 286, in _run_build_subcommands
self.run_command(name)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/dist.py", line 1221, in run_command
super().run_command(command)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 735, in build_extensions
build_ext.build_extensions(self)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
self._build_extensions_serial()
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
self.build_extension(ext)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
_build_ext.build_extension(self, ext)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
objects = self.compiler.compile(
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 490, in unix_wrap_single_compile
original_compile(obj, src, ext, cc_args, cflags, pp_opts)
File "/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
/data_new/private/liuyadi/myenv/lib/python3.8/site-packages/setuptools/command/editable_wheel.py:151: _DebuggingTips: An error happened while installing 'fairseq' in editable mode.
************************************************************************
The following steps are recommended to help debugging this problem:
- Try to install the project normally, without using the editable mode.
Does the error still persists?
(If it does, try fixing the problem before attempting the editable mode).
- If you are using binary extensions, make sure you have all OS-level
dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
- Try the latest version of setuptools (maybe the error was already fixed).
- If you (or your project dependencies) are using any setuptools extension
or customization, make sure they support the editable mode.
After following the steps above, if the problem still persist and
you think this is related to how setuptools handles editable installations,
please submit a reproducible example
(see https://stackoverflow.com/help/minimal-reproducible-example) to:
https://github.com/pypa/setuptools/issues
More information about editable installs can be found in the docs:
https://setuptools.pypa.io/en/latest/userguide/development_mode.html
************************************************************************
_DebuggingTips.warn(project)
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building editable for fairseq
Failed to build fairseq
ERROR: Could not build wheels for fairseq, which is required to install pyproject.toml-based projects
I updated the version of torch before to make it fit the official document's requirements. Could anyone help me to fix this problem? My environment: Python: 3.8.0 pip: 23.0 torch: 1.10.1+cu102 transformers: 4.26.1
