0

I'm working on a scraper for a website which closes its site/its database every night from about eleven in the evening to five in the morning. Since the scraper I'm working on takes about 70hours to complete, and I am in a rush to get it done, I'd have to wake up at 4.30 every morning...... maybe there's a better way..

Using a ruby gem such as 'whenever' you can set the time to start the code, like:

every :day, :at => ['5am'] do

and then the code

However, this doesn't pause the code. How would you make the code pause at 11pm to 5am the next day, continue to 11pm, pause untill 5am the next day, etcetera?

Seeb
  • 199
  • 4
  • 16

1 Answers1

0

If you gave a few more details, like what framework, or methods you are using, it would be easier to help you.

However, if you are using something based on EvenMachine, then it's possible to pause EventMachine::Connection's. You could the just go through all you connections, with "whenever", and pause all you outgoing connections, and the later on go through them again and resume the connections.

jbr
  • 6,198
  • 3
  • 30
  • 42
  • Hi Jbr, I'm working with simply ruby and the rubygems nokogiri, open-uri, watir-webdriver, csv and whenever. All i want to be able to do is run my ruby code from a certain time at the day and stop a certain time of the day, and continue the next day. Automated. It takes multiple days to run my code, and it has to stop every night, and I want to automate that process by saying to the code: run from 5 am to 11 pm every day. – Seeb Apr 25 '13 at 11:36