0

I have written a Python program that scrapes information from a website using regex. My goal is to create a cron job to run this scraper each month.

I have gone into the Linux terminal, typed in crontab -e, and added to the bottom of the crontab file:

**

#!/usr/bin/python chmod +x
30 8 1 * * /home/pi/Nikita/The_Scraper/thescraper.py
PATH=/home/pi/Nikita/The_Scraper/thescraper.py
MAILTO="myemailaddress@gmail.com"

**

I am wondering:

  1. If this is the correct text to include in the crontab file

  2. How to verify if my scraper program with run each month and the cron job I created is working

AndyG
  • 39,700
  • 8
  • 109
  • 143
user3482411
  • 1

1 Answers1

1

Where do I start:

1st: Cron starts the script via your shell. So /home/pi/Nikita/The_Scraper/thescraper.py have to have execute permissions.

2nd: PATH ist the name of the environment variable where the shell searches for your script if specified without PATH. It should contain only directories.

3rd: The crontab is read from top to bottom. It should be enough to use

MAILTO="myemailaddress@gmail.com"
30 8 1 * * /home/pi/Nikita/The_Scraper/thescraper.py

This should run your script every 1st day of the month on 8:30.

The MAILTO setting specifies external mail address. You should have a properly configured MTA (program that delivers the mail) running. The content of that mail is STDOUT and STDERR of the mail.

To test you could specify a time some near time (5 minutes in the future) and see what happens. Also you can redirect the OUTPUT to a file then you see if the job has been run an what its OUTPUT was if sending a mail does not work.

30 8 1 * * /home/pi/Nikita/The_Scraper/thescraper.py > /some/test/file/were/your/user/can/write

If you have access to the system logs you are able to see if the cronjob has been executed.

abraXxl
  • 64
  • 4