2

I have several classes written in C# that I want to consume from an application written in Python using Python for .NET.

Having previously done something similar using Python4Delphi and Python's C API I know how crucial it is to hold the Global Interpreter Lock (GIL) for everything that interacts with Python, while releasing it during long-running operations so other Python threads may run.

For acquiring the GIL, Python for .NET comes with the handy Py.GIL() utility and I wrote NoGil to make it just as easy to release the GIL.

I had assumed that whenever Python code called the C# code, the GIL was held for the duration of the call. It seems, though, that the GIL is not held during method calls, while it is held during a constructor call, as illustrated by the example below.

Here is the C# class and my NoGil utility class with some logging. For simplicity I did not implement the Disposable pattern in full.

using Python.Runtime;
using System;

namespace PythonNet
{
    public class Class1
    {
        public Class1()
        {
            using (new NoGil("constructor"))
            {
                Console.WriteLine("executing constructor");
            }
        }

        public void Method()
        {
            using (new NoGil("method"))
            {
                Console.WriteLine("executing method");
            }
        }
    }

    public class NoGil: IDisposable
    {
        private string _message;
        private IntPtr _state = IntPtr.Zero;

        public NoGil(string message)
        {
            _message = message;
            Console.WriteLine("Before calling BeginAllowThreads from " + message);
            _state = PythonEngine.BeginAllowThreads();
            Console.WriteLine("After calling BeginAllowThreads from " + _message);
        }

        public void Dispose()
        {
            if (_state == IntPtr.Zero)
            {
                Console.WriteLine("B_state == IntPtr.Zero in " + _message);
            }
            else
            {
                Console.WriteLine("Before calling EndAllowThreads from " + _message);
                PythonEngine.EndAllowThreads(_state);
                Console.WriteLine("After calling EndAllowThreads from " + _message);
            }
        }
    }
}

It is used from Python (after having installed the pythonnet package via pip) like this:

import clr
clr.AddReference("PythonNet.dll")
from PythonNet import Class1
c = Class1()
c.Method()

The output is

Before calling BeginAllowThreads from constructor
After calling BeginAllowThreads from constructor
executing constructor
Before calling EndAllowThreads from constructor
After calling EndAllowThreads from constructor
Before calling BeginAllowThreads from method
Fatal Python error: PyEval_SaveThread: NULL tstate

Current thread 0x0000124c (most recent call first):
  File "test.py", line 5 in <module>

I tried this with Python 2.7 and 3.6 (both 64 bit), the latest version 2.3.0 of Python for .NET and both .NET 4.0 and 4.6.1 as my target framework.

Questions:

  1. Is this expected behavior in Python for .NET or should I file a bug?
  2. If it is expected, what exactly are the situations where I can assume that the GIL is being held when calling .NET code? I did not find any documentation on this.
  3. Or should I never assume anything about the GIL and always acquire it when necessary (i.e. when calling into Python code via delegates)? Then how do I make sure I don't hold it during a lengthy non-Python operation?
denfromufa
  • 5,610
  • 13
  • 81
  • 138
PersonalNexus
  • 1,292
  • 1
  • 9
  • 25
  • Can you repost the output, code-formatted instead of quote-formatted, so we can be sure of *exactly* what the code produced? – user2357112 Mar 16 '18 at 19:27
  • 1
    [It looks like pythonnet *does* auto-release the GIL for method calls](https://github.com/pythonnet/pythonnet/blob/8f258acbf6a605ada4045391dc14f21bfc79fda6/src/runtime/methodbinder.cs#L514), for some reason. – user2357112 Mar 16 '18 at 19:33
  • Reformatted the output. Thanks for the link. Looks like unless a method is marked with the `[ForbidPythonThreads]` attribute (which unfortunately is internal), the GIL will be released before the call. Not so for constructors, however. – PersonalNexus Mar 16 '18 at 20:04
  • Allow threads and acquire GIL are a bit different, e.g. see this PR discussion: https://github.com/pythonnet/pythonnet/pull/419 – denfromufa Mar 20 '18 at 03:11

0 Answers0