I have several classes written in C# that I want to consume from an application written in Python using Python for .NET.
Having previously done something similar using Python4Delphi and Python's C API I know how crucial it is to hold the Global Interpreter Lock (GIL) for everything that interacts with Python, while releasing it during long-running operations so other Python threads may run.
For acquiring the GIL, Python for .NET comes with the handy Py.GIL()
utility and I wrote NoGil
to make it just as easy to release the GIL.
I had assumed that whenever Python code called the C# code, the GIL was held for the duration of the call. It seems, though, that the GIL is not held during method calls, while it is held during a constructor call, as illustrated by the example below.
Here is the C# class and my NoGil
utility class with some logging. For simplicity I did not implement the Disposable pattern in full.
using Python.Runtime;
using System;
namespace PythonNet
{
public class Class1
{
public Class1()
{
using (new NoGil("constructor"))
{
Console.WriteLine("executing constructor");
}
}
public void Method()
{
using (new NoGil("method"))
{
Console.WriteLine("executing method");
}
}
}
public class NoGil: IDisposable
{
private string _message;
private IntPtr _state = IntPtr.Zero;
public NoGil(string message)
{
_message = message;
Console.WriteLine("Before calling BeginAllowThreads from " + message);
_state = PythonEngine.BeginAllowThreads();
Console.WriteLine("After calling BeginAllowThreads from " + _message);
}
public void Dispose()
{
if (_state == IntPtr.Zero)
{
Console.WriteLine("B_state == IntPtr.Zero in " + _message);
}
else
{
Console.WriteLine("Before calling EndAllowThreads from " + _message);
PythonEngine.EndAllowThreads(_state);
Console.WriteLine("After calling EndAllowThreads from " + _message);
}
}
}
}
It is used from Python (after having installed the pythonnet package via pip) like this:
import clr
clr.AddReference("PythonNet.dll")
from PythonNet import Class1
c = Class1()
c.Method()
The output is
Before calling BeginAllowThreads from constructor
After calling BeginAllowThreads from constructor
executing constructor
Before calling EndAllowThreads from constructor
After calling EndAllowThreads from constructor
Before calling BeginAllowThreads from method
Fatal Python error: PyEval_SaveThread: NULL tstate
Current thread 0x0000124c (most recent call first):
File "test.py", line 5 in <module>
I tried this with Python 2.7 and 3.6 (both 64 bit), the latest version 2.3.0 of Python for .NET and both .NET 4.0 and 4.6.1 as my target framework.
Questions:
- Is this expected behavior in Python for .NET or should I file a bug?
- If it is expected, what exactly are the situations where I can assume that the GIL is being held when calling .NET code? I did not find any documentation on this.
- Or should I never assume anything about the GIL and always acquire it when necessary (i.e. when calling into Python code via delegates)? Then how do I make sure I don't hold it during a lengthy non-Python operation?