0

I have a process that I have to communicate with through pipes. It works correctly in Windows XP SP3 compatibility mode on Windows 10 but not in other modes.

But it seems that subprocess doesn't care about compatibility mode even if shell=True is used. I thought if shell starts it instead of going directly to kernel with the request the CM would be preserved, but it does not. When started from cmd.exe it works in XP compatibility mode as expected.

How can I force subprocess to start it in a compatibility mode? Or is it a question of using shell=True with added some special environment variable?

The process is Cygwin's ssh.exe and it has big issue with key permissions being to open on Windows 10. Using it in XP CM tricks it to ignore ACLs that otherwise enter into posix version of permissions mode. Thing is that I can not expect users to set correct permissions for each key they add through Properties and doing it automatically in a program is complicated because the program might not have permissions to do so. It's a big mess. Any ideas are appreciated.

Dalen
  • 4,128
  • 1
  • 17
  • 35
  • One solution would be to use the Microsoft Application Compatibility Toolkit to create an explicit compatibility fix (a .sdb file) and then install the fix alongside your program. That might be a little more flexible than using a registry entry. – Harry Johnston Jun 11 '17 at 02:03
  • ... are you using the 32-bit version of Python? It might be worth explicitly selecting the 64-bit version of `cmd.exe`, i.e., `c:\windows\sysnative\cmd.exe /c myapp.exe` and see if that makes any difference. – Harry Johnston Jun 11 '17 at 02:04

1 Answers1

1

You can try setting __COMPAT_LAYER before invoking ssh.exe:

import os
import subprocess

os.environ.update({"__COMPAT_LAYER": "WinXP"})  # set it to WinXP

your_process = proc = subprocess.Popen(["ssh", "whatever"], shell=True, env=os.environ)
# ...
zwer
  • 24,943
  • 3
  • 48
  • 66
  • Why not just `os.environ["__COMPAT_LAYER"] = "WinXP"`? – phd Jun 10 '17 at 14:40
  • @phd - no particular reason, a matter of preference, I usually set multiple environment variables so it's kind of a reflex. – zwer Jun 10 '17 at 15:10
  • Doesn't work. It seems that __COMPAT_LAYER is set by the OS to tell your app that it does run in compatibility mode. So that your app can act upon it if it needs to. But +1 for a good try anyway. – Dalen Jun 10 '17 at 15:30
  • @Dalen - how are you deploying your app? You can do a registry entry to force `ssh.exe` to run in compatibility mode, but it requires elevated execution at least for the first time. – zwer Jun 10 '17 at 15:36
  • Elevated execution is, I supose, running as admin? I got an OSError raised complaining about not being run in elevated mode after setting ssh.exe's CM to XP in properties. So subprocess perhaps does start it in CM after all. But why then it works from CMD, but in Python still asks for password and shouts about too opened permissions if my parent process is run as admin, thus freezing my whole app on Windows 10. – Dalen Jun 10 '17 at 15:46
  • This stupid issue is driving me crazy. And I cannot just use paramico or twisted because other spawned processes are using ssh. Baah! Which reg entry do I have to add and where to enforce CM to ssh.exe and others? Shouldn't properties app do so for you? – Dalen Jun 10 '17 at 15:52
  • @Dalen - add a key in `HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers` with name set to the full path to your `ssh.exe`, and a value set to `WINXPSP3`. You can do that programmatically when installing your app and you shouldn't need to run your Python in any special way afterwards to have it respected. – zwer Jun 10 '17 at 16:03
  • If you don't want a variable set in the current process, start with a copy via `environ = os.environ.copy()`. Also, some misbehaving programs are particular about the case of variable names, in which case you can start with `environ = nt.environ.copy()` to use the initial process environment. – Eryk Sun Jun 11 '17 at 01:57