In Pythonnet BeginAllowThreads crashes when calling C# object from Python

Question

Using Pythonnet I'm calling a long-running method on a CLR object from within a Python script. I want to use BeginAllowThreads to allow other threads obtain the GIL as described here: Pythonnet Threading.

However the call to BeginAllowThreads crashes the whole application with a 'no thread state' error. The calling thread is a CLR worker thread and has the GIL, so I believe it should have a thread state.

Here is a summary of the code sequence:

// In the main thread Pythonnet is initialised like this:
public void Initialise() 
{
    PythonEngine.Initialize();
    m_outerScope = Py.CreateScope( "outerscope" )
    ...
    m_mainThreadState = PythonEngine.BeginAllowThreads();
}

...
// A python script is run on a worker thread like this. 
// pyObj is a CLR object with methods the script can call, created 
// with clrobj.ToPython():
void runScript( string script, PyObject pyObj, string objName)
{
    using( Py.GIL() ) {
        using( PyScope scope = m_outerScope.NewScope() ) {
            try {
                scope.Set( objName, pyObj );
                scope.Exec( script );
            } catch( Exception exc ) {
                ...
            }
        }
    }
}
...
// Within the Python script we have something like:
objName.SomeMethod( 'message')
...

// The callback method on the CLR object looks like:
public void SomeMethod( string msg )
{
    var state = PythonEngine.BeginAllowThreads();
    someLongRunningFunction( msg );
    PythonEngine.EndAllowThreads( state );
}

But the no-state error occurs when BeginAllowThreads is called. Without the Begin/EndAllowThreads calls the code works fine but other threads block waiting for the GIL.

It seems as if the worker thread has no thread state within the PythonEngine, but it has the GIL. What am I missing here?

score 0 · Answer 1 · answered Jun 20 '22 at 14:49

0

When you call a C# method from Python, Python.NET calls BeginAllowThreads. The crash you see is because you can't call BeginAllowThreads twice.

If other threads block waiting for GIL, the GIL must be locked somewhere else.

answered Jun 20 '22 at 14:49

LOST

2,956
3
25
40

I suspected that that was the case but I haven't found any documentation that mentions it. However because I found another thread waiting on the GIL while the worker thread was in the 'someMethod' code above it seemed that it couldn't be the case. The problem is that it causes a deadlock if the Windows GUI thread is waiting on the GIL and 'someMethod' makes a synchronous Invoke call on the Windows thread. The GIL should be released but it doesn't seem to be. – cjm1905 Jun 21 '22 at 16:55

In Pythonnet BeginAllowThreads crashes when calling C# object from Python

1 Answers1