No module named lxml.html

Question

Running OS X 10.9.4, I'm trying to use Scrapy, but I get this error:

Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 3, in <module>
from scrapy.cmdline import execute
File "/Library/Python/2.7/site-packages/scrapy/cmdline.py", line 8, in <module>
from scrapy.crawler import CrawlerProcess
File "/Library/Python/2.7/site-packages/scrapy/crawler.py", line 6, in <module>
from scrapy.core.engine import ExecutionEngine
File "/Library/Python/2.7/site-packages/scrapy/core/engine.py", line 14, in <module>
from scrapy.core.downloader import Downloader
File "/Library/Python/2.7/site-packages/scrapy/core/downloader/__init__.py", line 13, in <module>
from .middleware import DownloaderMiddlewareManager
File "/Library/Python/2.7/site-packages/scrapy/core/downloader/middleware.py", line 7, in <module>
from scrapy.http import Request, Response
File "/Library/Python/2.7/site-packages/scrapy/http/__init__.py", line 11, in <module>
from scrapy.http.request.form import FormRequest
File "/Library/Python/2.7/site-packages/scrapy/http/request/form.py", line 9, in <module>
import lxml.html

And 'pip install lxml' only returns

/Users/username/.virtualenvs/scraper/lib/python2.7/site-packages

falsetru · Accepted Answer · 2014-08-10T01:42:03.007

3

It seems like you installed scrapy with system version of Python.

While you installed lxml in virtualenv version of Python.

Check your pip reference which python using following command:

pip -V

If you want install scapy in virtualenv, you need to uninstall scrapy first. Otherwise it will prevent running of virtualenv version because of PATH issue.

deactive  # deactive first, to use system version of python/pip
pip uninstall -y scrapy
hash -r   # refresh program location.

edited Aug 10 '14 at 01:42

answered Aug 10 '14 at 01:24

falsetru

357,413
63
732
636

I think you're right, it returns: pip 1.5 from /Users/username/.virtualenvs/scraper/lib/python2.7/site-packages (python 2.7) How do I resolve it? – mobius Aug 10 '14 at 01:25
@mobius, Which python do you want to use? – falsetru Aug 10 '14 at 01:26
@mobius, If you want virtualenv one, install `scrapy` there: `pip install scrapy` – falsetru Aug 10 '14 at 01:27
@mobius, I meant (system version / virtualenv version). – falsetru Aug 10 '14 at 01:27
@mobius, If you want to use system version, deactivate virtualenv first, then install lxml: `deactivate; pip install lxml` – falsetru Aug 10 '14 at 01:28
Ah, virtualenv version. So I uninstall Scrapy, then reinstalled while in my virtualenv (just to make sure), but I'm still getting "ImportError: No module named lxml.html". – mobius Aug 10 '14 at 01:29
@mobius, What do you get by issuing following command: `type scrapy` – falsetru Aug 10 '14 at 01:29
"scrapy is /usr/local/bin/scrapy". Also, I updated the output I get after installing Scrapy inside the virtualenv – mobius Aug 10 '14 at 01:30
1

@mobius, Uninstall `scrapy` from the system-wide version first. `deactive; pip uninstall -y scrapy; hash -r`. – falsetru Aug 10 '14 at 01:31
That worked! I think I understand what was going wrong – mobius Aug 10 '14 at 01:33

No module named lxml.html

1 Answers1