1

In a Python3 program running in Windows 10, os.system() is not able to find a file created in the APPDATA directory using with open, even after that same file can be successfully read by a subsequent with open.

QUESTION FOR THIS OP:

What specific syntax needs to be changed in the Python3 code below in order to successfully create a file in the APPDATA directory that can be found visually by users of the Windows 10 Explorer UI and also programmatically by the os.system() command below which is failing?

CURRENT CODE:

The following code is what we are currently trying:

print(os.getenv('APPDATA'))
terraformRC = os.path.join( os.getenv('APPDATA'), "terraform.rc")
print("terraformRC is: ", terraformRC)
print("About to write terraformRC. ")
try: 
  with open(terraformRC, 'w') as f:
    f.write('provider_installation {\n')
    f.write('  filesystem_mirror {\n')
    f.write('    path    = "' + providersPath + '"\n')
    f.write('    include = ["*/*"]\n')
    f.write('  }\n')
    f.write('\n')
    f.write('  direct {\n')
    f.write('    exclude = ["*/*"]\n')
    f.write('  }\n')
    f.write('}\n')
except (Exception) as e:
  print(e)

print("About to read the terraformRC we just wrote.  ")
with open(terraformRC, 'r') as lines:
  for line in lines:
    print(line)

print("About to disable settings for folder so that the terraformRC file can be unhidden.  ")
removeSettingsCmd = 'attrib -h -s ' + terraformRC
os.system(removeSettingsCmd)

CURRENT FAILED RESULTS:

The following output is printed by Windows CMD when we call the above Python3 code from Windows CMD.

C:\path\to\AppData\Roaming
terraformRC is:  C:\path\to\AppData\Roaming\terraform.rc
About to write terraformRC.
About to read the terraformRC we just wrote.
provider_installation {
  filesystem_mirror {
    path    = "C:\path\to\dependencies\terraform\providers"
    include = ["*/*"]
  }
  direct {
    exclude = ["*/*"]
  }
}

About to disable settings for folder so that the terraformRC file can be unhidden.
File not found - C:\path\to\AppData\Roaming\terraform.rc

THE PROBLEM:

As you can see from the output above, Python seems to successfully find the APPDATA directory. Then Python seems to successfully write a terraformRC file to APPDATA. Then Python seems to successfully read the terraformRC file that it just seems to have written to APPDATA.

The problem is that the os.system(removeSettingsCmd) then fails with a message File not found - C:\path\to\AppData\Roaming\terraform.rc stating that it cannot find the terraformRC file at the correct location. And also, a human user is not able to view the terraformRC file when looking for it in APPDATA using Windows Explorer.

CodeMed
  • 9,527
  • 70
  • 212
  • 364
  • @lit: No, that's not relevant. – martineau Aug 09 '21 at 23:04
  • What happens if you try to manually enter the `attrib` command for the file on the command line? – martineau Aug 09 '21 at 23:07
  • 1
    I think you may need quotes around the filename: ```removeSettingsCmd = 'attrib -h -s "' + terraformRC + '"'``` – sj95126 Aug 09 '21 at 23:11
  • That's only a possibility of the real filenpath has any space characters in it. – martineau Aug 09 '21 at 23:14
  • @martineau it's obvious that `path\to\AppData` is completely made up, so the actual file name might have a space. – Mark Ransom Aug 09 '21 at 23:17
  • @MarkRansom: Yes, obviously, I didn't say anything contrary to that — in fact, just the opposite. – martineau Aug 09 '21 at 23:18
  • This question looks nearly identical to one that was posted a couple of days ago. I was unable to reproduce it. – Mark Ransom Aug 09 '21 at 23:22
  • The error message apparently includes the full file path, which implies that it's not a space issue. However, the reported output includes an extra space in the `terraformRC is:`, so the reported output is maybe suspect. – jamesdlin Aug 09 '21 at 23:23
  • @jamesdlin: The path in the error message was probably changed to match the rest of the question. As are all the other things containing it (so there might even be an extra space which proves nothing). – martineau Aug 09 '21 at 23:29
  • @CodeMed I don't know what your point is. I meant that that `print` call contains only one space after the colon, yet the reported output contains two spaces, which is impossible. So the (obviously doctored) output doesn't match the code and seems untrustworthy. – jamesdlin Aug 09 '21 at 23:30
  • Try `print(os.getenv('APPDATA')==os.getenv('APPDATA').strip())`. – Mark Ransom Aug 09 '21 at 23:36
  • Don't know if it's relevant, but on my Windows computer the only thing in my `C:\Users\Me\AppData\Roaming` directory is other subdirectories — there are no files of any kind (and yes, I have show hidden and system files enabled). – martineau Aug 09 '21 at 23:39
  • @martineau same here, except for the file that I created there trying to reproduce the duplicate problem. And it was more than a couple of days ago, because my file is dated 2021-07-20. I'm on Windows 7 though, that might make a difference. – Mark Ransom Aug 09 '21 at 23:43
  • Make a [mcve]. After I added `import os` and removed the reference to `providersPath`, the code worked, the file was created, and I could see it in Explorer. Myabe you've left out other code as well. The `attrib` wasn't needed, because the file only had the `A` (archive) bit set. – Mark Tolonen Aug 09 '21 at 23:48
  • 1
    @CodeMed BTW, "a human user is not able to view the terraformRC file when looking for it in APPDATA using Windows Explorer" isn't a useful data point given that the file (somehow) started off with the hidden and/or system attributes. (Why does it have those attributes at all?) – jamesdlin Aug 09 '21 at 23:51
  • @jamesdlin the default separator for `print` is space. – Mark Tolonen Aug 10 '21 at 01:35
  • Try putting a `sleep(60)` just before opening the file for read. Maybe something in your system is monitoring that directory and deleting anything that pops up. – Mark Ransom Aug 10 '21 at 02:26
  • 1
    @CodeMed that’s not what I was referring to. `print('a ',b)` will adds a space between the two items, which is why your prints have two spaces, and explains what @janesdlin pointed out – Mark Tolonen Aug 10 '21 at 06:05
  • Do you have the same issue with any directory? E.g. "C:\Users\youruser\Documents"? – gimix Aug 10 '21 at 09:31
  • No, Your code worked for me in APPDATA and the file was visible in Explorer with only the A attribute. There's something else going on that we can't repeat. – Mark Tolonen Aug 10 '21 at 22:35
  • Use a tool like [SysInternals Process Monitor](https://learn.microsoft.com/en-us/sysinternals/downloads/procmon) to watch the file accesses. It really doesn't make sense that python can write and read the file but os.system with a full path can't find it. – Mark Tolonen Aug 10 '21 at 22:59
  • Filter on Process=attrib.exe would be a place to start – Mark Tolonen Aug 10 '21 at 23:17
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/235855/discussion-between-mark-tolonen-and-codemed). – Mark Tolonen Aug 10 '21 at 23:32

1 Answers1

3

It appears you have installed Python from the Microsoft Store. Store apps are Universal Windows Platform (UWP) apps and redirect AppData storage on a per-user basis to a unique user-specific area that a native app such as attrib is unaware of. For more information see UWP - store and retrieve settings and other app data.

I recommend uninstalling the UWP version of Python and installing an official native binary from python.org, which will work as you expect.

Mark Tolonen
  • 166,664
  • 26
  • 169
  • 251