0

I want to get the href (https://www.dcard.tw/forum/popular) from https://www.dcard.tw/

this href is under <a href="/forum/popular">

my code:

from bs4 import BeautifulSoup
import requests
url = "https://www.dcard.tw/"
soup = BeautifulSoup(requests.get(url).text,'lxml')

for link in soup.find_all('a'):
    print(link.get('href'))
Ismaili Mohamedi
  • 906
  • 7
  • 15
ericfang
  • 1
  • your are not getting the html as requests are blocked, explore to put header etc so its looks like more like real requests. or move to selenium, the response you are getting is: ```Just a moment... www.dcard.tw Checking if the site connection is secure Enable JavaScript and cookies to continue www.dcard.tw needs to review the security of your connection before proceeding. Ray ID: 76565d82bd4d3ff1 Performance & security by Cloudflare``` – simpleApp Nov 05 '22 at 14:47

1 Answers1

0

You can try the examples from this thread, they got some successful options.

How can I get href links from HTML using Python?

Regards, Alex

Alexandru Tudorie
  • 15
  • 6