0

Python has its GIL and is ever and anon single-threaded. On the other hand there is no clear reason why a single JVM instance could not hand off the Gateway responsibilities to each of N threads - each one handling a separate the socket communication with a separate python process . Is this actually possible with py4j ?

WestCoastProjects
  • 58,982
  • 91
  • 316
  • 560

1 Answers1

1

A single GatewayServer instance on the Java side can handle N python processes.

The server thread accepts connections on a thread and then spawn a new thread for each new connection/python process.

You can read more about the threading model used in Py4J here.

Barthelemy
  • 8,277
  • 6
  • 33
  • 36
  • Thanks for those docs. It is not clear to me yet how to get the multiple _py4j_ processes. Note that section 3.4. _Using Single Threading Model (pinned thread)_ is for _python_ initiated processes (the opposite of what I want). Section 3.5 _Initiating the communication from the Java side_ is what I need but does not clearly state how to do the multiple processes. Do you see how to apply _3.5_ to get the multiple py4j processes? – WestCoastProjects Mar 23 '21 at 14:06