0

I have a selenium script with python 3. I need to run it every day at 7AM for 5 minutes but I cannot leave my laptop on 24/7 just for it. What would be the easiest/best alternative?

I checked Lambda AWS but I am unable to install chrome (or the headless version of it) there and I have no experience with servers.

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
user3507584
  • 3,246
  • 5
  • 42
  • 66
  • Are you able to provisioning an EC2 instance? Is your script compatible with headless browsers like phantomjs? – JRichardsz Feb 13 '19 at 02:32

2 Answers2

2

AWS Lambda is a viable solution for your goal - especially if the script is going to have a small runtime. They have recently increased the maximum execution time to 15 minutes, so you should be ok.

Getting headless Chrome in Lambda (in Python; it's full of JS/node.js solutions out there :)) is doable - I myself have used successfully this project in the past - https://github.com/21Buttons/pychromeless

To schedule the execution at your desired time you could use Amazon's CloudWatch.

Todor Minakov
  • 19,097
  • 3
  • 55
  • 60
  • In my script I use functions like `find_element_by_xpath` or `click`. A headless chrome would be able to work with these selenium functions? – user3507584 Feb 13 '19 at 23:34
  • 1
    Yes, absolutely; pretty much everything in the selenium library is available to a headless browser. – Todor Minakov Feb 14 '19 at 04:53
0

To execute a Selenium script with Python 3 without installing a browser you can use the GhostDriver.

GhostDriver

Ghost Driver is an implementation of the Remote WebDriver protocol using PhantomJS as back-end. GhostDriver is designed to be integral part of PhantomJS itself with JavaScript API.

Additional WebDriver Capabilities through GhostDriver

  • phantomjs.page.settings.SETTING = VALUE
  • phantomjs.page.customHeaders.HEADER = VALUE
  • phantomjs.page.whitelist
  • phantomjs.page.blacklist
  • unhandledPromptBehavior
  • loggingPrefs
  • phantomjs.binary.path
  • phantomjs.ghostdriver.path
  • phantomjs.cli.args
  • phantomjs.ghostdriver.cli.args

Main Advantages of GhostDriver

  • Screen Capture
  • Page Automation
  • Network Monitoring
  • To run Unit tests on command line
  • Combined with QUnit for the test suite

An Example

  • Code Block :

    from selenium import webdriver

driver = webdriver.PhantomJS(executable_path='/path/to/phantomjs')
driver.get('https://www.google.com/')
print(driver.title)
driver.quit()

  • Console Output:

    Google
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • So, how does a GhostDriver commerical answers the question "how & where to schedule daily recurring script executions"; and OP asking for AWS lambda you instruct him to set a driver executable that's for Windows? – Todor Minakov Feb 13 '19 at 07:36