My requests-html does not work and gives error 'No module named email.utils.'

Question

I am learning web scraping and I installed requests-html. Now I ran this script I saw in a tutorial, and can't get it to work. I don't understand as it is a part of the standard library. I tried pip install email but it just returned another error:

'ERROR: No matching distribution found for email'.

I ran the code in a shell and chromium got installed for render() function as it was supposed to. pyppeteer was timing out for some reason so I had to declare html.render(timeout=60), it then worked but I still can't get it to work if I use script instead of IDLE shell.

This is the code below:

from requests_html import HTMLSession

session = HTMLSession()

url = 'https://www.youtube.com/channel/UC8tgRQ7DOzAbn9L7zDL8mLg/videos'

r = session.get(url)

r.html.render(sleep=1, keep_page=True, scrolldown=1)

#take the rendered html and find the element that we are interested in
videos = r.html.find('#video-title')

#loop through those elements extracting the text and link
for item in videos:
    video = {
        'title': item.text,
        'link': item.absolute_links
    }
    print(video)

This is the traceback:

Traceback (most recent call last):
  File "D:/Python/requests-render-demo.py", line 1, in <module>
    from requests_html import HTMLSession
  File "C:\Python39\lib\site-packages\requests_html.py", line 9, in <module>
    import pyppeteer
  File "C:\Python39\lib\site-packages\pyppeteer\__init__.py", line 15, in <module>
    from importlib_metadata import version
  File "C:\Python39\lib\site-packages\importlib_metadata\__init__.py", line 7, in <module>
    import email
  File "D:/Python\email.py", line 1, in <module>
    import smtplib
  File "C:\Python39\lib\smtplib.py", line 47, in <module>
    import email.utils
ModuleNotFoundError: No module named 'email.utils'; 'email' is not a package ```

You have a script named `email.py` (in D:\Python) that is shadowing the built-in `email` module. — jasonharper, Apr 13 '21 at 20:15
Thanks Jason! I can't believe I missed that. Also, can you explain why it ran in shell though? — Uttam Sharma, Apr 14 '21 at 07:48
I would guess that there is a different Python search path between the two environments, such that the working one either doesn't see the `email.py` file at all, or sees the real module before that file. — jasonharper, Apr 14 '21 at 13:03

score 0 · Answer 1 · answered Apr 13 '21 at 20:05

Looks like the problem you are facing is:

...I still can't get it to work if I use script instead of IDLE shell...

From the stack trace, it looks like requests_html did not get installed correctly (The traceback suggests it is not able to import requests_html)

Assuming you are running a fresh virtual environment, install the library again and then try out your script from within the virutal environment:

virtualenv new_virtual_environment
source new_virtual_environment/bin/activate # This will enable the virtual environemnt
pip install requests-html
python <<path_to_script file>>

Thanks Raghav but the problem was that I had a script with name email.py in D:/python which shadowed standard library and caused problem. — Uttam Sharma, Apr 14 '21 at 07:51

My requests-html does not work and gives error 'No module named email.utils.'

1 Answers1