0

I have an application built with Python / Twisted Matrix which uses methods from a SOAP client in order to send some messages. Problem is that sometimes i want to send a lot of messages and when that happens i would like to do it in multiple threads. For example if i have to send 100 messages i would like this broken into groups of 20 messages and to create 5 threads to send the messages in parallel.

What should i look for ? Any ideas ? I would also like the threads to be able to report back with the gathered data

P.S. given the fact that probably working with SOAP clients is more of a problem of waiting around ... do you think that threading is not the best approach to solve this ? Can the callbacks of the soap client be used to create some sort of "pool" of senders and have the senders somehow as for new stuff to send as soon as they are free ? Ideas ?

Liviu
  • 1,023
  • 2
  • 12
  • 33

1 Answers1

0

The best approach is probably defined by the distribution of the SOAP services and methods your accessing.

My first suggestion would be to not use OS Threading, but use micro-threading generator-coroutines with inlineCallbacks and deferredSemaphores.

But you might want to tune things to reuse connections for the same server and/or retain server cookies.

MattH
  • 37,273
  • 11
  • 82
  • 84
  • `inlineCallbacks` and `DeferredSemaphore` are not "micro-threading", they are generator coroutines; not quite the same thing. Micro-threading looks like regular threading (no "yield" statements) but provides more efficient thread-switching in the background. See eventlet for a microthreading system that uses Twisted for I/O. – Glyph Oct 08 '12 at 16:25