Why can't I delete a directory created in ProgramData?

Question

I' trying to create a directory and then delete it (for testing purposes, which I will ommit, but can give details if needed).

Like this:

>>> import os
>>> os.makedirs('C:\\ProgramData\\dir\\test')
>>> os.remove('C:\\ProgramData\\dir\\test')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [WinError 5] Access is denied: 'C:\\ProgramData\\dir\\test'

I always get access denied, although I'm running the interpreter as an admin. Also I have no problem manually deleting the directory.

score 3 · Answer 1 · answered Apr 08 '15 at 11:12

3

Use os.rmdir to delete a folder. os.remove is for deleting files.

answered Apr 08 '15 at 11:12

khelwood

55,782
14
81
108

Can you elaborate on why this happens? Keep in mind that, if it's not obvious, my Python knowledge is extremely limited, at best :) Is there documentations somewhere that says why `os.remove` should not be used on directories? Also if this is mandatory shouldn't the method throw a "better" exception? – Simeon Apr 08 '15 at 11:44
2

@Simeon, on Windows, `os.remove` is implemented by calling [`DeleteFile`](https://msdn.microsoft.com/en-us/library/aa363915). When the target is a directory, the resulting error is [`ERROR_ACCESS_DENIED`](https://msdn.microsoft.com/en-us/library/ms681382#ERROR_ACCESS_DENIED) -- hence `PermissionError` in Python. This stems from the underlying [NT status code](https://msdn.microsoft.com/en-us/library/cc704588) `STATUS_FILE_IS_A_DIRECTORY`. `os.rmdir` is implemented by calling [`RemoveDirectory`](https://msdn.microsoft.com/en-us/library/aa365488). – Eryk Sun Apr 08 '15 at 12:08
2

@Simeon, at a lower level the cause of the error when trying to delete a directory via `DeleteFile` is that it calls the system service [`NtOpenFile`](https://msdn.microsoft.com/en-us/library/ff567011) with `FILE_NON_DIRECTORY_FILE` set in `OpenOptions`, whereas `RemoveDirectory` uses `FILE_DIRECTORY_FILE`. That's the main difference. After that both APIs call [`NtSetInformationFile`](https://msdn.microsoft.com/en-us/library/ff567096) to set the `FileDispositionInformation` to delete the file or directory. – Eryk Sun Apr 08 '15 at 12:27
@eryksun Thanks a lot. This is what I needed, could you add an answer for more visibility? – Simeon Apr 08 '15 at 13:35

Eryk Sun · Accepted Answer · 2015-04-09T09:38:29.863

Use os.rmdir to remove a directory. On Windows this is implemented by calling the WinAPI function RemoveDirectory. os.remove is implemented by calling DeleteFile, which is only meant for deleting files. If the filename argument is a directory, the call fails and it sets the last error code to ERROR_ACCESS_DENIED, for which Python 3 raises a PermissionError.

In this case the access denied error is based on the NTSTATUS code STATUS_FILE_IS_A_DIRECTORY, i.e. RtlNtStatusToDosError(0xC00000BA) == 5. Often the kernel's status code is more informative than the corresponding WinAPI error, but not always, and there isn't always a simple mapping from one to the other, depending on the division of labor between the kernel, system processes, services, environment subsystems, and the application. In this case I think the kernel status code is undeniably more informative than a generic access denied error.

At a lower level the cause of the error when trying to delete a directory via DeleteFile is that it calls the system service NtOpenFile with the FILE_NON_DIRECTORY_FILE flag set in OpenOptions, whereas RemoveDirectory specifies FILE_DIRECTORY_FILE. Subsequently both functions call NtSetInformationFile to set the FileDispositionInformation to delete the file or directory.

Just to be a contrarian, let's implement the entire sequence using only file operations on an NTFS file system.

>>> import os, pathlib
>>> base = pathlib.Path(os.environ['ProgramData'])

Create the 'dir' directory:

>>> dirp = base / 'dir::$INDEX_ALLOCATION'
>>> open(str(dirp), 'w').close()
>>> os.path.isdir(str(dirp))
True

By manually specifying the stream type as $INDEX_ALLOCATION, opening this 'file' actually creates an NTFS directory. Incidentally, you can also add multiple named $DATA streams to a directory. Refer to the file streams topic.

Next create the 'test' subdirectory and call os.remove to delete it:

>>> test = base / 'dir' / 'test::$INDEX_ALLOCATION'
>>> open(str(test), 'w').close()
>>> os.path.isdir(str(test))                       
True
>>> os.remove(str(test))
>>> os.path.exists(str(test))
False

You may be surprised that this worked. Remember the filename in this case explicitly specifies the $INDEX_ALLOCATION stream. This overrules the FILE_NON_DIRECTORY_FILE flag. You get what you ask for. But don't rely on this since these streams are an implementation detail of NTFS, which isn't the only file system in use on Windows.

Thanks :) This is extremely exhaustive. I wish I could give it more votes. — Simeon, Apr 09 '15 at 09:55
@Simeon, Except I skipped showing how to run Python under a [debugger](https://msdn.microsoft.com/en-us/library/ff551063) (`cdb python`), set a breakpoint on the target function (`bp kernel32!DeleteFileW`), go (`g`), call `os.remove`, step through by calls (`pc`), print 6 arguments for an x64 call (`r rcx,rdx,r8,r9; dq @rsp+20 l2`), jump to the function return using `pt`, and print the thread environment block ([`!teb`](https://msdn.microsoft.com/en-us/library/ff565433)) to see the `LastErrorValue` (Windows) and `LastStatusValue` (NT). — Eryk Sun, Apr 09 '15 at 10:30
:D I know my "thanks" comments are just spam, but answers of this quality are rarely seen on SO, so thanks again. — Simeon, Apr 10 '15 at 14:17

Why can't I delete a directory created in ProgramData?

2 Answers2