0

I'm trying to pause scrapy after x seconds and resume until any key is pressed i have this code in middlewares.py

class RepeatTimer(Timer):
    def run(self):
        while not self.finished.wait(self.interval):
            self.function(*self.args, **self.kwargs)


CMD = '''
on run argv
  display notification (item 2 of argv) with title (item 1 of argv)
end run
'''

.....
   # passed objects.

    def notify(title, text):
        subprocess.call(['osascript', '-e', CMD, title, text])
        # pasue scrapy
        
        # while True:
        #     inputt = input("Press Enter/Return Key to continue...")
        #     if inputt == "":
        #         break
    
    timer = RepeatTimer(40, notify, args=("GetBooks Timer", "Change Your IP Address It Has Been 900 Seconds GETSBOOK"))
    timer.start()

    @classmethod
    def from_crawler(cls, crawler):
        # This method is used by Scrapy to create your spiders.
        s = cls()
        crawler.signals.connect(s.spider_opened, signal=signals.spider_opened)
        return s

I have found this and this answer but it doesn't work for me.

Thanks You For Type of Help!!!!

Adarsh
  • 25
  • 4
  • I would suggest you pass a defer into scrapy `engine.py` where the crawler is called from the middleware, and then call the JOBDIR command when it matches the condition of the defer. – Emil11 Aug 01 '22 at 12:03
  • Secondly the suggested answers from the link will not work for what you're after. The reactor when started will not wait for the conditions of a timer-set, and a delay to a call in the middleware will likely not impede the engine process. – Emil11 Aug 01 '22 at 12:20

0 Answers0