I implemented a chat using gRPC and 3 servers with load balance. The first thing to achieve the load balance is to use a ManagedChannel with defaultLoadBalancingPolicy. In my case I used round_robin policy. And create the channel using a MultiAddressNameResolverFactory with the host and ports of the three servers. Here I create a client chat for Alice. Then You copy this class and create a client chat for Bob. This should already do the load balance that you asked.
public class ChatClientAlice {
private NameResolver.Factory nameResolverFactory;
private ManagedChannel channel;
public static void main(String[] args) {
ChatClientAlice chatClientAlice = new ChatClientAlice();
chatClientAlice.createChannel();
chatClientAlice.runBiDiStreamChat();
chatClientAlice.closeChannel();
}
private void createChannel() {
nameResolverFactory = new MultiAddressNameResolverFactory(
new InetSocketAddress("localhost", 50000),
new InetSocketAddress("localhost", 50001),
new InetSocketAddress("localhost", 50002)
);
channel = ManagedChannelBuilder.forTarget("service")
.nameResolverFactory(nameResolverFactory)
.defaultLoadBalancingPolicy("round_robin")
.usePlaintext()
.build();
}
private void closeChannel() { channel.shutdown(); }
private void runBiDiStreamChat() {
System.out.println("creating Bidirectional stream stub for Alice");
EchoServiceGrpc.EchoServiceStub asyncClient = EchoServiceGrpc.newStub(channel);
CountDownLatch latch = new CountDownLatch(1);
StreamObserver<EchoRequest> requestObserver = asyncClient.echoBiDi(new StreamObserver<EchoResponse>() {
@Override
public void onNext(EchoResponse value) { System.out.println("chat: " + value.getMessage()); }
@Override
public void onError(Throwable t) { latch.countDown(); }
@Override
public void onCompleted() { latch.countDown(); }
});
Stream.iterate(0, i -> i + 1)
.limit(10)
.forEach(integer -> {
String msg = "Hello, I am " + ChatClientAlice.class.getSimpleName() + "! I am sending stream message number " + integer + ".";
System.out.println("Alice says: " + msg);
EchoRequest request = EchoRequest.newBuilder()
.setMessage(msg)
.build();
requestObserver.onNext(request);
// throttle the stream
try { Thread.sleep(5000); } catch (InterruptedException e) { }
});
requestObserver.onCompleted();
System.out.println("Alice BiDi stream is done.");
try {
// wait for the time set on the stream + the throttle
latch.await((5000 * 20), TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
On the server service you will have to use a singleton to store the StreamObservers every time that you receive a new request from new clients. Instead of returning the message to a single observer responseObserver.onNext(response); you will iterate all observers and send the message to all singletonObservers.getObservers().forEach(..... Although this has nothing to do with the load balance strategy I thought that it is worthwhile to post because if you don't implement it well your clients will not receive messages from other clients.
public class ChatServiceImpl extends EchoServiceGrpc.EchoServiceImplBase {
private final String name;
private final SingletlonChatStreamObserver singletonObservers;
ChatServiceImpl(String name) {
this.name = name;
this.singletonObservers = SingletlonChatStreamObserver.getInstance();
}
@Override
public StreamObserver<EchoRequest> echoBiDi(StreamObserver<EchoResponse> responseObserver) {
System.out.println("received bidirectional call");
singletonObservers.addObserver(responseObserver);
System.out.println("added responseObserver to the pool of observers: " + singletonObservers.getObservers().size());
StreamObserver<EchoRequest> requestObserver = new StreamObserver<EchoRequest>() {
@Override
public void onNext(EchoRequest value) {
String msg = value.getMessage();
System.out.println("received message: " + msg);
EchoResponse response = EchoResponse.newBuilder()
.setMessage(msg)
.build();
// do not send messages to a single observer but to all observers on the pool
// responseObserver.onNext(response);
// observers.foreach...
singletonObservers.getObservers().forEach(observer -> {
observer.onNext(response);
});
}
@Override
public void onError(Throwable t) {
// observers.remove(responseObserver);
singletonObservers.getObservers().remove(responseObserver);
System.out.println("removed responseObserver to the pool of observers");
}
@Override
public void onCompleted() {
// do not complete messages to a single observer but to all observers on the pool
// responseObserver.onCompleted();
// observers.foreach
singletonObservers.getObservers().forEach(observer -> {
observer.onCompleted();
});
// observers.remove(responseObserver);
System.out.println("removed responseObserver to the pool of observers");
}
};
return requestObserver;
}
}
and this is my SingletlonChatStreamObserver to have only one of this object for all 3 servers:
public class SingletlonChatStreamObserver implements Serializable {
private static volatile SingletlonChatStreamObserver singletonSoleInstance;
private static volatile ArrayList<StreamObserver<EchoResponse>> observers;
private SingletlonChatStreamObserver() {
//Prevent form the reflection api.
if (singletonSoleInstance != null) {
throw new RuntimeException("Use getInstance() method to get the single instance of this class.");
}
}
public static SingletlonChatStreamObserver getInstance() {
if (singletonSoleInstance == null) { //if there is no instance available... create new one
synchronized (SingletlonChatStreamObserver.class) {
if (singletonSoleInstance == null) {
observers = new ArrayList<StreamObserver<EchoResponse>>();
singletonSoleInstance = new SingletlonChatStreamObserver();
}
}
}
return singletonSoleInstance;
}
//Make singleton from serializing and deserialize operation.
protected SingletlonChatStreamObserver readResolve() {
return getInstance();
}
public void addObserver(StreamObserver<EchoResponse> streamObserver) {
observers.add(streamObserver);
}
public ArrayList<StreamObserver<EchoResponse>> getObservers() {
return observers;
}
}
I will commit the complete code on my explore-grpc project.
