How to make more than one xmlrpc calls to different xmlrpc servers at a time

Question

Is there any way to make multiple calls from an xmlrpc client to different xmlrpc servers at a time.

My Server code looks like this: (I'll have this code runnning in two machines, server1 & server2)

class TestMethods(object):
    def printHello(self):
        while(1):
            time.sleep(10)
            print "HELLO FROM SERVER"
            return True

class ServerThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.server = SimpleXMLRPCServer(("x.x.x.x", 8000))
        self.server.register_instance(TestMethods())
    def run(self):
        self.server.serve_forever()

server = ServerThread()
server.start()

My Client code looks like this:

import xmlrpclib

client1 = xmlrpclib.ServerProxy("http://x.x.x.x:8080") # registering with server 1
client2 = xmlrpclib.ServerProxy("http:/x.x.x.x:8080") # registering with server 2
ret1 = client1.printHello()
ret2 = client2.printHello()

Now, on the 10th second I'll get a response from server1 and on the 20th second I'll get a response from server2 which is unfortunately not what I want. I'm trying to make calls to two machines at a time so that I get the response back from those two machines at a time.

PLease help me out, THanks in advance.

Answer 1

There a a few different ways to do this.

python multiprocessing

Is the built-in python module for running stuff in parallel. The docs are fairly clear. The easiest & most extensible way using this method is with a 'Pool' of workers that you can add as many to as you want.

from multiprocessing import Pool
import xmlrpclib

def hello_client(url):
    client = xmlrpclib.ServerProxy(url)
    return client.printHello()

p = Pool(processes=10)  # maximum number of requests at once.

servers_list = ['http://x.x.x.x:8080', 'http://x.x.x.x:8080']
# you can add as many other servers into that list as you want!

results = p.map(hello_client, servers_list)

print results

twisted python

twisted python is an amazing clever system for writing all kinds of multithreaded / parallel / multiprocess stuff. The documentation is a bit confusing.

Tornado

Another non-blocking python framework. Also very cool. Here's an answer about XMLRPC, python, and tornado.

gevent

A 'magic' way of allowing blocking tasks in python to happen in the background. Very very cool. And here's a question about how to use XMLRPC in python with gevent.