Multi find beautiful soup

Question

I have a list of item like this: (the number of item lists can vary)

<h3>My title</h3>
<a href="http://myurl.com">http://myurl.com</a>
<span class="t">text</span>

<h3>My title</h3>
<a href="http://myurl.com">http://myurl.com</a>
<span class="t">text</span>

...

How with beautiful soup I could get all these data so I can put all of them is a list to have a result like this : [{'title': h3, 'url': url, 'title': title}, {'title': h3, 'url': url, 'title': title}, ...] ?

thank you

Could you show the exact format of the list? Is it a nested list? Or just a plain list? — whackamadoodle3000, Jun 25 '18 at 16:33
It would be helpful to know the website that you are pulling this information from, since you need a way to select/identify what you want to extract — rahlf23, Jun 25 '18 at 16:48

score 0 · Accepted Answer · answered Jun 25 '18 at 16:55

You could iterate over your HTML's contents like so (assuming that your data is held in html_data):

import bs4

soup = BeautifulSoup(html_data)
my_list = []
for i in range(len(soup.body.contents), step=3):
    my_list.append({'title1': soup.body.contents[i], 'url': soup.body.contents[i+1], 'title2': soup.body.contents[i+2]})

This of course only works under the premise that your data resides on the same level and is not nested in any way. If it's not, then you should post a valid chunk of your test data and its structure.

So did this help you? Marking this as answered in such case would be constructive. — Arno-Nymous, Jul 10 '18 at 10:49

Multi find beautiful soup

1 Answers1