Why do seemingly valid Python scripts fail to run when initiated via the Windows Task Scheduler?

Question

I have been writing scripts in Python which I want to automatically run on a Windows server on a scheduled basis. I tried working with batch files very briefly, but have enthusiastically moved on to Python scripts instead and am doing fine with them. The scripts I write are easy to write and they do exactly what I want them to do when I manually execute them. I have written many different scripts now, mostly dealing with copying, deleting, and renaming files or moving directories around.

The problem is when I try to schedule Python scripts using Windows Task Scheduler, many of them fail to run (Task Scheduler says "Last Run Result = 0x1"). This happens all the time. I have had similar experiences with batch files as well (batch files which can be run manually fail to run when scheduled). Given my limited experience thus far, I would have to say this is certainly a Windows Task Scheduler problem and not a Python problem.

Here is an example Python script:

#import modules
import os, shutil, datetime, subprocess

#global variables
zip_dir = 'Y:\7z'
zip_dir_misc = 'X:\7z'
zip_extension = '.7z'

def newest_zip_file(directory, extension = zip_extension): 
  return max(
    (os.path.join(dir_name, file_name)
    for dir_name, dir_names, file_names in os.walk(directory)
    for file_name in file_names
    if file_name.endswith(extension)),
    key=lambda fn: os.stat(fn).st_mtime)  

def copy_zip_file(src_dir_p, temp_dir_p):
  src_file = newest_zip_file(src_dir_p)
  new_file = temp_dir_p + '\\' + os.path.basename(src_file)
  shutil.copyfile(src_file, new_file)

copy_zip_file(zip_dir, zip_dir_misc)

This script copies a .7z file from one server to another via a networked folder (represented as the X: partition). This script works when run manually, but not when scheduled. However, when the script is changed to copy the same .7z file to another directory on the same server (not a networked folder), the script will work flawlessly, either when executed manually or scheduled.

If I am doing something programmatically incorrect in the script above (perhaps I am referring to a networked folder improperly) then I can fix this script this one time (although I have already tried every combination I could imagine for defining the networked folder using things like the full server name). But I keep running into the same problem with totally different Python scripts which behave the same way, which brings me to the real question:

Why do seemingly valid Python scripts fail to run when initiated via the Windows Task Scheduler?

My Windows Task Scheduler configuration:

Run whether user is logged on or not
Run with highest privileges
Actions (Python scripts) called directly (Start a program: abc.py), not sent as a argument to Python.exe, although I have definitely tried the Python.exe approach many times as well. I am not convinced that either approach is better than the other (as far as making the scheduled task execute), so I just simply call the script directly.

I am looking mostly for suggestions and best practices for solving this general problem, and not so much specific fixes to the example script posted above.

do you use IDLE? I know I had to set IDLE to run as an administrator when using Task Scheduler. — JasonBK, Aug 13 '15 at 18:11
Nope. I was not using IDLE at the time, and I still do not use it. I do not think it should be a requirement. — nairware, Aug 13 '15 at 20:53

score 6 · Accepted Answer · answered Aug 21 '14 at 04:26

6

You mention networked folders, this is a known issue with the task scheduler. For tips on dealing with it in Python, see this post.

Generally:

Make sure that you are setting the "Start in" property to the script directory in the Action via Task Scheduler GUI
Do full file references within the script, including the directory
If necessary, run the task as an authenticated user and "run with highest privileges" in the General settings
For various reasons you may have better luck running the task as a batch file that calls python.exe to run your script

answered Aug 21 '14 at 04:26

Erica Kane

3,137
26
36

Your first point made it work for me. Could you explain what "Start in" means? – Ray Jul 31 '17 at 16:47
The [post you linked to](http://coredumpelf.wordpress.com/2011/04/04/windows-2008-task-scheduler-issues-with-mapped-drives/) is essential reading for anyone using mapped drives. – pianoJames Aug 14 '17 at 13:33
1

@Ray: I think that if you DON'T specify a "Start in", then the script will run relative to C:\Windows\System32 or something similar. [This](https://serverfault.com/questions/426519/why-is-start-in-needed-for-windows-scheduled-tasks) might have more info. – pianoJames Aug 14 '17 at 14:14

score 2 · Answer 2 · answered Jun 21 '12 at 14:55

I'm not very familiar with Windows (or its Task Scheduler), but it sounds like the typical cron problems you run into when trying to run Python scripts on an automated base on Unix systems.

The main reason for the cron problems are a different environment and a different user the scripts are run in and under. The main reason why they are hard to fix is because the developer is missing the error messages when the scripts are run automatically. To fix that I typically use a wrapper script which logs all the output of the Python script into an error log file (just to be able to see the error messages after the fail).

Fixing the original problem typically also can be done by a wrapper script setting up the correct environment (variables and stuff) for the Python script.

As I said, my experience is based on Unix cron, not on Windows Task Scheduler, but the trouble sounds very much alike.

score 1 · Answer 3 · answered Jun 21 '12 at 15:27

A couple of notes:

Make sure the user which the scheduler is using to launch the python files has the python directory in its PATH;

Secondly, make sure that user also has networking permissions, and the mount exists for that user as well

Third, ensure you are launching from the proper folder. I suggest either modifying the scripts to change to a working directory where the .py file is stored, or launch using a .bat file that will change to the working directory before launching the python script.

score 1 · Answer 4 · answered Sep 26 '16 at 16:49

I run into this problem on Windows Server 2012 R2 and I think I found the reason behin this: It seems that the environmnet variable are loaded at the creation of a task. Therefore if you are using your own module and if you set up the PYTHONPATH variable after the task, the task will always fail trying to load the new python modules. The solution I found is simply to to recreate the task and to use a batch to launch the python script. On my sever it works, I hope it will do the trick for you

score -1 · Answer 5 · answered Jun 21 '12 at 15:00

First of all, unless you set windows up properly, windows knows nothing of python or .py files. Make sure you're doing that.

Second of all, try this with a completely minimal python script - for example one that simply touches a file in a directory. This will tell you if your scripts are being run at all.

Thirdly, if you think this is a windows problem, then it might well be better to ask at Superuser or Serverfault.

Oleg SH · Answer 6 · 2017-03-29T06:35:39.587

-1

My script runs like this: Start a program: cmd.exe /c c:\Python27\python.exe "c:\path\script.py" Run with highest privileges

edited Mar 29 '17 at 06:35

answered Mar 28 '17 at 12:19

Oleg SH

443
5
11

Why do seemingly valid Python scripts fail to run when initiated via the Windows Task Scheduler?

6 Answers6