Distributed System

Question

I am looking to create a distributed framework in Java and need some help sorting out the implementation of a client/manager/worker situation as described in my pseudocode below.

Manager
BEGIN
    WHILE(true)     
        RECEIVE message FROM client
            IF (worker_connections > 0) THEN            
            FOR (i=0;i<worker_connections;i++) 
                SEND message TO worker[i]
            FOR (i=0;i<worker_connections;i++) 
                RECIEVE result[i] FROM worker[i]             
            SEND merge(result[]) TO client
            ELSE            
            SEND "No workers available" TO client
            END IF
    END WHILE
END

Client
BEGIN
    RECEIVE message FROM user
    SEND message TO manager
    RECEIVE message FROM manager
END

Worker
BEGIN
    WHILE(true)
        RECEIVE message FROM manager
         result = doSomething(message)
         SEND result TO manager
    END WHILE
END

So far I have implemented the client as described in the pseudocode but i'm having problems with the manager/worker part so at the moment the client just receives the message that no workers are available. The manager can accept multiple connections, each connection is run as a thread but how do I differentiate between a client connection and worker connection? How do I keep track of the number of workers currently connected?

The client, manager and workers are representations of different machines however I am only developing on one machine (in Java).

Can we see some code of what you have tried to do? – Marcelo Nov 25 '11 at 17:59 — Marcelo, Nov 25 '11 at 17:59

score 2 · Accepted Answer · answered Nov 25 '11 at 18:03

2

Q. How do I differentiate between a client connection and worker connection?
A. Use two ports, one to listen to connections from workers and one from clients.
Q. How do I keep track of the number of workers currently connected?
A. Have a registration mechanism (maybe in a list of workers) to register/deregister each time a worker connects/disconnects.

answered Nov 25 '11 at 18:03

Raihan

10,095
5
27
45

How do I use listen on two ports, my manager code is looping waiting for new client connections.. new connection_thread(serverSocket.accept()).start(); – Chris Seymour Nov 25 '11 at 18:26
2

You can provide a port number when creating a new ServerSocket `public ServerSocket(int port)`. Have two threads opening two ServerSocket. – Raihan Nov 25 '11 at 18:35
Thank you, my manager can now accept multiple connection from both clients and workers on different ports! – Chris Seymour Nov 25 '11 at 19:06

score 1 · Answer 2 · answered Mar 04 '12 at 03:53

Q: The manager can accept multiple connections, each connection is run as a thread but how do I differentiate between a client connection and worker connection?

I guess you can define some protocol, by which I mean the format/semantic of messages, msg type may help. Also, as for distributed system, the FIFO can not be vaulted, I guess you should use some mechanism as sequence num, logic clock to take care of the match between request and response.

Q:How do I keep track of the number of workers currently connected?

First, login and logout can be taken into account, however, not always possible, especially logout; second, connect error using timeout, as to crash after sending, there can be some threshold, or "getResult" message; third, as to a DS, I think you had better apply some election algorithm to make sure there is always worker.

Distributed System

2 Answers2

Linked