2

First of all, I should say that this might more a design question rather than about code itself.

I have a network with one Server and multiple Clients (written in Twisted cause I need these asynchronous non-blocking features), such server-client couple it's just only receiving-sending messages.

However, at some point, I want one client to run a python file when received a certain message. That client should keep listening and talking to the Server, also, I should be able to stop that file if needed, so my first thought is starting a thread for that python file and forget about it.

At the end it should go like this: Server sends message to ClientA, ClientA and its dataReceived function interprets the message and decides to run that python file (which I don't know how long will take and maybe contains blocking calls), when that python file finishes running should send the result to ClientB.

So, questions are:

  • Would it be starting a thread a good idea for that python file in ClientA?
  • As I want to send the result of that python file to ClientB, can I have another reactor loop inside that python file?

In any case I would highly appreciate any kind of advice as both python and twisted are not my specialty and all these ideas may not be the best ones.

Thanks!

josepmeg
  • 67
  • 5

1 Answers1

2

At first reading, I though you were implying twisted isn't python. If you are thinking that, keep in mind the following:

Twisted is a python framework, I.E. it is python. Specifically it's about getting the most out of a single process/thread/core by allowing the programmer to hand-tune the scheduling/order-of-ops in their own code (which is nearly the opposite of the typical use of threads).

While you can interact with threads in twisted, its quite tricky to do without ruining the efficiency of twisted. (for longer description of threads vs events see SO: https://stackoverflow.com/a/23876498/3334178 )

If you really want to spawn your new python away from your twisted python (I.E. get this work running on a different core) then I would look at spawning it off as a process, see Glyph's answer in this SO: https://stackoverflow.com/a/5720492/3334178 for good libraries to get that done.

Processes give the proper separation to allow your twisted apps to run without meaningful slowdown and you should find all your starting/stoping/pausing/killing needs will be fulfilled.

Answering your questions specifically:

Would it be starting a thread a good idea for that python file in ClientA?

I would say "No" its generally not a good idea, and in your specific case you should look at using processes instead.

Can I have another reactor loop inside that python file?

Strictly speaking "no you can't have multiple reactors" - but what twisted can do is concurrently manage hundreds or thousands of separate tasks, all in the same reactor, will do what you need. I.E. run all your different async tasks in one reactor, that is what twisted is built for.

BTW I always recommend the following tutorial for twisted: krondo Twisted Introduction http://krondo.com/?page_id=1327 Its long, but if you get through it this kind of work will become very clear.

All the best!

Community
  • 1
  • 1
Mike Lutz
  • 1,812
  • 1
  • 10
  • 17
  • Thanks Mike! I'll definitely take a look to these processes (and the twisted tutorial), seems that may work for me. I know that it might haven't looked as I meant, I just tried to show that what I want to run is something "independent" even though is a reaction to an event that my client will handle. At least, I do think they follow different paths. Anyway, referring to the second answer, then I should have the factories and so for the second connection (between A and B) in that second python file while calling "reactor.connect" (for that connection) from the main code? Thanks Mike! – josepmeg Jun 16 '15 at 18:19
  • @josepmeg: I'm confused how you want to get the "result of that python file" (which is run on clientA) to clientB. Do you want to send it to the server and then have the server send it to clientB? Or do you want clientA to directly connect to clientB ... and BTW what protocol is this all happening via? TCP? UDP? serial? other? (I'll try and help you in the comments but if this gets too complicated you may need to open a new question with some code snippets) – Mike Lutz Jun 16 '15 at 18:32
  • So, my main concern is that I don't want ClientA to be worried about that python file, just start/stop it. When that file is done should send its result to the server (might be better than doing it directly to B, I can have Server redirecting things). But might be easy to do so using just an independent socket so I don't have to use the reactor for that...I'm just trying to figure out which one is the best way to proceed and have a good understanding of everything before start writing the code. I'm doing everything using TCP by now. Thanks! – josepmeg Jun 16 '15 at 18:54
  • 1
    @josepmeg: If I imagine how I would make the logic on clientA, it would be: (all within twisted) clientA connects to server, holding a TCP connection open. Via messageReceived the client wakes up to process incoming messages. In the case of a massage that tells the client to spawn "that python file", the client will call twisted's spawnProcess with the python to run. Upon `processExited`, clientA will package up the results and `write` in the Server TCP connection object the results. To be any clearer, code will be needed, see the last sample on: http://stackoverflow.com/a/30399317/3334178 – Mike Lutz Jun 16 '15 at 19:23