I am trying to find a company on a website and then click the link the search returns. After working a while to debug the code - it turned out my capitalization was off - the script runs through smoothly without any errors but it does not open the link in the browser like I thought it would.
Here is the code:
import webbrowser
import csv
import mechanize
import string
br = mechanize.Browser()
def open_page(url):
webbrowser.open(url)
def name_to_url(name):
words = name.split(" ")
url = "http://solicitors.lawsociety.org.uk/search/results?Name="
end_of_url = "&Type=0&IncludeNlsp=True&Pro=True"
for word in words:
url += "%s+" % word
url += "%s" % end_of_url
return url
with open('/home/seb/company-list/data/test.csv', 'rb') as f:
reader = csv.reader(f)
for row in reader:
company_name = string.capwords(row[0])
url_go = name_to_url(company_name)
br.open(url_go)
br.find_link(text=company_name)
req = br.click_link(text=company_name)
br.open(req)
The URL it leads to is correct 100% (triple-checked in Pycharm) and the company_name is also correct.