python multiprocessing agents and udp listeners

Question

I have a master class (server for want of a better term) that creates multiple clients (from a client class) usingthge mulitproccessing library.

class mantransact:
def __init__(self,runMode,f_xml):  
    #call the build nodes function
    self.buildNodes()
    sockLisProcess = multiprocessing.Process(target=self.sockListener())
    sockLisProcess.start()
    self.initiateTransactions()

def buildNodes(self,):
    n_source = self.f_xml.getElement("nodeSource")
    self.log.addToLog ("node source is - %s" % n_source)
    self.n_file = load_source.load_source(n_source,"csv")
    #remove header from node list
    del self.n_file.data_list[0]    
    self.nodes = [self.mkproc(node, l) for l in self.n_file.data_list]
    self.log.addToLog(self.nodes)

def mkproc(self, func, line):
    l = "-".join(line)
    p = multiprocessing.Process(target=func, args=(l, self.f_xml))
    p.start()
    return (line[0], p)    

def sockListener(self):
    self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    server_addresss = ('localhost',10099)
    self.sock.bind(server_addresss)
    while True:
        self.log.addToLog("server is waitin")
        data, address = self.sock.recvfrom(1024) 
        self.log.addToLog(data, address)   

def sockSender(self,client_address,d):
    self.sock2 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    self.sock2.bind(('localhost',10098))
    recip = ('localhost',int(client_address))
    self.sock2.sendto(str(d),recip)
    self.sock2.close() 

def initiateTransactions(self,):
    #loop through transaction and then loop node list to match at match transmit transaction         
    #using UDP ports
    for t in self.t_file.data_list:
        for n in self.nodes:
            if t[0] == n[0]:
                for l in self.n_file.data_list:
                    if l[0] == n[0]:
                        self.log.addToLog ("gonna transmit UDP transaction to node - %s" % n[0])
                        client_address = l[1]
                        pip = n[2]
                        t.insert(0, "nTransaction")
                        self.sockSender(client_address, t)

I am trying to create UDP listeners at both the client and the nodes:

class node:
def __init__(self,child_conn, line, f_xml):
   l = line.split("-")
   """extract calues from array and use f_xml for config"""
   self.proofProcess = multiprocessing.Process(target=self.runProof(self.waitingTransactions))
    self.proofProcess.start()
   self.listenProcess = Multiprocessing.Process(target=self.udpListener())
   self.listenProcess.start()

def udpListener(self):
    lsock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    lsock.bind(("localhost",int(self.IP)))
    while 1 ==1:
        data, addr = lsock.recvfrom(1024)
        print ("received message", data)
        """do some things with data"""

I have two issues:

1 with my server I want my code to kick off these processes and then continue on instantiating or performing other tasks but the code just hangs waiting for the listener to receive a packet. Am i instantiating processing incorrectly:

2 with my clients they are performing a task to solve a problem and don't start the listeners until that task is complete. Can they not start their task an listen in parallel? The listener is meant to interrupt the calculation if another clietn solves it first and then it will receive a new task from the server

score 0 · Answer 1 · answered Jun 27 '15 at 12:25

0

I found the solution.

By putting the multiprocessing element into a seperate process:

def loopProcesses(self,procedureName):

    processX = multiprocessing.Process(target=procedureName)
    processX.start()
    return processX

And putting the names of the processes to be used into an array which is looped through to call the loopProcesses() process both processes were started in parallel.

m_processes = [self.sockListener(), self.initiateTransactions()]
l_processes = [self.loopProcesses(mp) for mp in m_processes]

answered Jun 27 '15 at 12:25

p0815

13
5

I have recently discovered that this doe not work. it will not start the second until the first is finnished. An alternative solution – p0815 Jul 04 '15 at 16:13
The above did not work as the functions called are in a continuous loop until they have found a number of solutions. A problem was occuring when the first function was called it would start without the start command. I later found that I have to call the function without using the '()' and then the funciton will wait. ammended code is: – p0815 Jul 11 '15 at 17:48

score 0 · Accepted Answer · edited May 23 '17 at 11:43

The above did not work as the functions called are in a continuous loop until they have found a number of solutions. A problem was occuring when the first function was called it would start without the start command. I later found that I have to call the function without using the '()' and then the funciton will wait. ammended code is:

p = [multiprocessing.Process(target=self.sockListener),multiprocessing.Process(target=self.initiateTransactions)]
for prc in p:
    prc.start()

I found this after a lot of seraching and came accross this: Socketserver multiprocessing.Process is starting without calling start()

python multiprocessing agents and udp listeners

2 Answers2