4

Followed this tutorial about Web Scraping with Python and BeautifulSoup to learn the ropes - However Pycharm returns an error which I do not understand

Hi there!

Tried the above mentioned tutorial with an adjusted link as the actual link the tutorial expired (New link I used) However, when I click Run i get several errors Tried the type hint of PyCharm to no avail.


import requests
from bs4 import BeautifulSoup

r = requests.get('https://pyvideo.org/events/pycon-se-2018.html')

soup = BeautifulSoup(r.text, 'html.parser')
results = [a.attrs.get('href') for a in soup.select('h4.entry-title a[href^=/pycon]')]

print(results)

Expected was a list of the link. What I get though are a bunch of errors


Traceback (most recent call last):
  File "/Users/maxschmitt/PycharmProjects/tester2/tester.py", line 7, in <module>
    results = [a.attrs.get('href') for a in soup.select('h4.entry-title a[href^=/pycon]')]
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/bs4/element.py", line 1376, in select
    return soupsieve.select(selector, self, namespaces, limit, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/soupsieve/__init__.py", line 114, in select
    return compile(select, namespaces, flags, **kwargs).select(tag, limit)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/soupsieve/__init__.py", line 63, in compile
    return cp._cached_css_compile(pattern, namespaces, custom, flags)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/soupsieve/css_parser.py", line 209, in _cached_css_compile
    CSSParser(pattern, custom=custom_selectors, flags=flags).process_selectors(),
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/soupsieve/css_parser.py", line 1048, in process_selectors
    return self.parse_selectors(self.selector_iter(self.pattern), index, flags)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/soupsieve/css_parser.py", line 882, in parse_selectors
    key, m = next(iselector)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/soupsieve/css_parser.py", line 1035, in selector_iter
    raise SelectorSyntaxError(msg, self.pattern, index)
soupsieve.util.SelectorSyntaxError: Malformed attribute selector at position 16
  line 1:
h4.entry-title a[href^=/pycon]

Do you have an idea what I did wrong? Any help would be appreciated!

Thank you very much!

JohnDoe
  • 97
  • 5
  • 12

1 Answers1

7

You need to wrap /pycon in "" or escape it with \

import requests
from bs4 import BeautifulSoup

r = requests.get('https://pyvideo.org/events/pycon-se-2018.html')

soup = BeautifulSoup(r.text, 'html.parser')
results = [a.attrs.get('href') for a in soup.select('h4.entry-title a[href^="/pycon"]')]

print(results)

Or

results = [a.attrs.get('href') for a in soup.select('h4.entry-title a[href^=\/pycon]')]
QHarr
  • 83,427
  • 12
  • 54
  • 101
  • 1
    I was getting the same error with `tbl.select_one("a[href*=doi.org]")` and I had to escape the `.` too thus: `tbl.select_one("a[href*=doi\.org]")`. Thanks! – Rafs May 06 '21 at 11:33