2

I have a CSV file full of candidate names and I want to extract their LinkedIn Profile URLs. I have automated login using selenium and taken name input from CSV and clicked the search button but am unable to access the profile URLs on the page. For eg: I want to scrape profile URLs on the following Linkedin page but it is returning nothing when I query the following:

The link: https://www.linkedin.com/search/results/all/?keywords=April%20Siose&origin=GLOBAL_SEARCH_HEADER

driver.find_element_by_class_name("search-result__result-link ember-view").get_attribute("href")

aim_bee.1
  • 77
  • 8

2 Answers2

0

It seems the site requires you to login in before it shows the desired page. The driver libary may have a auth feature or something similar to log in automatically

Ovoid
  • 33
  • 6
  • So, I kind of automated the login part using selenium but having an issue at this stage. – aim_bee.1 May 12 '20 at 20:41
  • I found a useful [post](https://stackoverflow.com/a/25755483/10625989) it should still work. If it doesn't LinkedIn may have changed the structure again and you may need to modify the parameters – Ovoid May 12 '20 at 20:49
  • I edited my comment with a new link as the top post is now outdated. Apologies – Ovoid May 12 '20 at 20:54
  • Can u suggest a way without using requests?? It will add on to the timeout complications which I wanted to avoid. – aim_bee.1 May 12 '20 at 21:09
0
linkedin_urls = [url.get_attribute("href") for url in linkedin_urls]

this is the command in selenium to extract the LinkedIn profile's URLs.

Fabio Veronese
  • 7,726
  • 2
  • 18
  • 27
Saba Ghani
  • 27
  • 2