-2

i want to get CNN news article all link

for example

in this link 

  https://edition.cnn.com/search/?q=%20news&size=10

i can show lastest 10 news

to get news link i tried two methods.

   html_page = urlopen(url)
   soup = BeautifulSoup(html_page, "lxml")

   cnn_paper = newspaper.build(url, memoize_articles=False)  # ~15 seconds
   n_list = []
   for article in cnn_paper.articles:
      n_list.append(article.url)

and 

req = Request(url)
html_page = urlopen(req)

soup = BeautifulSoup(html_page, "lxml")
links = []
for link in soup.findAll('a'):
    links.append(link.get('href'))

but i can`t get news link

If you go to the next page, i can only get the same link

Visa De
  • 21
  • 3

1 Answers1

0

try this instead:

for link in soup.find_all('a'):
    links.append(link.get('href'))
Sushant Kathuria
  • 123
  • 8
  • 2
    While this code may answer the question, providing additional context regarding how and/or why it solves the problem would improve the answer's long-term value. – Nic3500 Sep 18 '18 at 13:17