0

I have set my windows power settings such that my computer never turns off and never goes to sleep. I've setup the following python script to run as a scheduled task, which runs fine until almost 1/2 hour after I've logged off my computer. It mysteriously stops. There is no error message in the Events log. The memory used by this process doesn't appear to be spiking. I don't know what's going on. Please help!

import datetime
from urllib.request import urlopen
from bs4 import BeautifulSoup
import csv
import logging
import shutil

def get_soup(theurl):
    html = urlopen(theurl)
    return BeautifulSoup(html.read(), "lxml")

def output_data(soup, filename, fieldnames, rows_with_info, supervision_row_count):
    with open(filename, 'a', newline='') as csvfile:
        mywriter = csv.DictWriter(csvfile, fieldnames=fieldnames)

        mydict = {};

        # Scraping first table         
        offender_table = soup.find(lambda tag: tag.name=='table' and tag.has_attr('id') and tag['id']=="offender-search-details")
        rows = offender_table.findAll(lambda tag: tag.name=='tr')
        mydict[fieldnames[0]]= clean_data(name_and_id[0])
        mydict[fieldnames[1]]= clean_data(name_and_id[1])          

        #lots of similar code removed for sake of brevity

        mywriter.writerow(mydict)   


start_id = 10
max_id = 199999

for the_id in range(start_id, max_id):
    logger.info('running with id-' + str(the_id))

    theurl=thebaseurl + str(the_id)

    soup = get_soup(theurl)
    sentence_table = soup.find(lambda tag: tag.name=='table' and tag.has_attr('id') and tag['id']=="offender-search-sentence-info")

    if sentence_table:
        logger.info('found data for id-' + str(the_id))

        sentence_rows = sentence_table.findAll(lambda tag: tag.name=='tr')

        supervision_row_count = 0

        for the_row_index in range(0, len(sentence_rows)):
            col_count = sentence_rows[the_row_index].find_all('td')
            if (len(col_count) == 2):
                supervision_row_count = supervision_row_count + 1

        supervision_row_count = supervision_row_count -1
        rows_with_info = len(sentence_rows) - 4 - supervision_row_count 

        output_data(soup, filename, fieldnames, rows_with_info, supervision_row_count)

logger.info('finished-' + str(datetime.datetime.now()))
Najla August
  • 81
  • 1
  • 12
  • Without digging too deep, could it possibly be that your script just finishes its loop from 10 to 200K and exits normally? – Organis Dec 13 '16 at 16:16
  • My script is outputting a line for each ID, like this: 2016-12-13 10:43:44,648 INFO running with id-42272. Therefore, I know it stops randomly, because it stops outputting ID numbers when it dies. – Najla August Dec 13 '16 at 16:26
  • What are the task's [settings](https://msdn.microsoft.com/en-us/library/windows/desktop/aa381843(v=vs.85).aspx)? Anything else happening after you logout? – dhke Dec 13 '16 at 16:48
  • The task is set to run whether user is logged or not, and with the highest privileges. What else should I look for that might be happening after logout? – Najla August Dec 13 '16 at 18:59
  • I just noticed this checkbox is selected: "Stop if the computer ceases to be idle". I don't fully understand what this means. – Najla August Dec 13 '16 at 19:16

0 Answers0