1

I'm adding python scripting support to an application.

This application has an API which is not thread safe, and I cannot change this aspect.

One requirement I have is being able to run multiple independent scripts, thus I have to run sub-interpreters in separate threads.

Although, due to the GIL in CPython, no more than one thread runs concurrently, whatever thread holds the GIL will still run concurrently with the main thread, and this will cause problems due to the thread-unsafe API of the application.

To summarize: I'm looking for a way to run all python code (__main__, threads, every sub-interpreter) in the main thread.

How can this be solved?

Should the main thread always hold the GIL, and have a function that -in a cooperative-multitasking fashion- would release it and reacquire it x milliseconds later, thus allowing the interpreter to do some work? This doesn't look right: such function will consume x milliseconds also when python has no work to do.

fferri
  • 18,285
  • 5
  • 46
  • 95
  • "Although, due to the GIL in CPython, no more than one thread runs concurrently, whatever thread holds the GIL will still run concurrently with the main thread" doesn't make any sense – Mad Physicist Dec 06 '19 at 14:34
  • Can you add a wrapper-layer around your non-thread-safe API, such that a mutex is locked before the API is accessed, and unlocked afterwards? That could allow multiple threads to access the API safely. – Jeremy Friesner Dec 06 '19 at 16:08
  • No because I'm not in control of that, then I would be the only one accessing the API with the mutex, the application would access the API without the mutex, making the mutex pointless. – fferri Dec 06 '19 at 19:21
  • 1
    How about making the Python API not make any changes itself, but feeds commands into [a queue](https://docs.python.org/3/library/queue.html). Then the main thread (only) handles the queue. – DavidW Dec 07 '19 at 10:25

0 Answers0