-1

To communicate some production system from Python I developed an extension using C++ and Pybind11. The functionality itself working fine. But(!) the python.exe process itself hangs when script finished. Currently script is very simple, I'm using it just to validate if all required functionality is working. Like this:

import xyz1 as x
x.XyzEnvironment.systemInfo()

#set connection string
xyz = x.XyzClient("H:XyzDevVm2012:20203;m:ABX;p:P001")

xyz.API.connect()
xyz.login("ADMIN", "forget")

xyz.API.beginTx()
xyz.API.execSQL("update NumSrx set nrlast=nrlast+1 where task='MYTASK-X'")
xyz.API.commitTx()

ds = xyz.API.openSQL("select * from USR")
ds.RowsCount
ds.FieldsCount
ds.Fields
ds.Rows
ds.Rows[1].getString(2)

print("press [enter]...")
s = input()

But here is a trick: all API code for target system is in .NET/C#, so to minimize efforts I developed in C++ only a wrapper which opens a named pipe to .NET/C# app which actually do all the work. So, instead of porting lot of code from C# to C++ I wrote some minimal set of C++ wrapper code which communicate C# app. (just to compare - 50k code lines in C# components required for communications vs. 3k code lines in C++ wrapper for it and 2k lines in C# wrapper. So, of course that worth it)

The functionality itself is working fine. BUT(!) - when it prints "press [enter]..." and I press [enter] the script is not finished. Python process keep hanging forever! Only when I manually kills a .NET process which was used to communicate a prod-system the python process also exits. I tried to add to the end of my python script calls like these: exit(), sys.exit(), os_exit(0), raise SystemExit, etc - but all of them do not guarantee correct closure of python process. Also quite often I see that my .NET app already disappeared from list of running processes but the python process still hanging.

Thus questions are:

  • what could be reasons of such strange hanging after script finished?
  • is it possible to handle script finished event from my C++ extension? So, I can close all opened handles, processes and release all allocated resources.
  • is it possible to enforce exit python process? (note: I already tried exit(), sys.exit(), os_exit(0), raise SystemExit - these not always work)

Note:

  • it is Windows 11 Pro for WS
  • Python 3.11.4 x64 installed from MS Store (the best method for Windows as I know from experience)
dmitry_bond
  • 378
  • 1
  • 3
  • 15
  • If you're going to be doing all the work over a named pipe, why even bother with C++ instead of native Python? – Botje Aug 25 '23 at 12:37
  • As for debugging the issue, well, use a debugger to grab a stack trace of your process 'hanging'. That should give us a clue how to get it unstuck properly. Also provide a bit more information about your C++ wrapper. Does it spawn a thread that is not getting stopped, perhaps? – Botje Aug 25 '23 at 12:38
  • 3
    The reason for the "strange hanging" would be a bug in your C++ wrapper or the C#/.NET code that gets called that results in undefined behavior and memory corruption. It will be necessary to locate the bug and fix it. Attempting to compensate for the bug, like calling `exit` directly, instead of figuring out the real bug and fixing it, will always end in tears. – Sam Varshavchik Aug 25 '23 at 12:41
  • (1) No. Wrapper does not create any threads! It is quite straight-forward - it writes something to named-pipe only when received a call from Python. Also it reads from named-pipe only after it wrote something to it. There is no any unsolicited or background I/O. (2) I need a C++ wrapper to have more convenient and simple API. We have server-side on C++ and client on C#, so I can reuse quite a lot of C++ code to make things simpler. I prefer to avoid writing wrapper on Python, just to minimize efforts. – dmitry_bond Aug 25 '23 at 13:08
  • Interesting thing - I exposed a new static method into a Python from XyzClient class - XyzClient.releaseAll(). And inside that method I simply closed a named pipe and pipe-server process. Now python correctly closed without any hanging. – dmitry_bond Aug 26 '23 at 06:31
  • "what could be reasons of such strange hanging after script finished?" - maybe it has something to do with the custom wrapper you told us about, but aren't showing us? "(1) No. Wrapper does not create any threads! It is quite straight-forward - it writes something to named-pipe only when received a call from Python." - **and, so what?** Why should that guarantee that this isn't where the problem is? The Python code *doesn't* hang when you use it directly from Python, right? So there aren't a lot of *other* places the problem could be, are there? – Karl Knechtel Aug 26 '23 at 10:06

0 Answers0