Given two or more finagle clients that have different destination names, if those names happen to resolve to the same inet address how to I get finagle to only maintain a single pool of connections to that endpoint?
Overly Simple Sample Code
The below code registers a very simple (and mostly useless) Resolver that always resolves to the same address. In practice this would be more like a many to one relationship (instead of all to one). There's another simple application that starts a server and two clients that both use the resolver to find the address to talk to the server.
// Registered with finagle Resolver via META-INF/services
class SmartResolver extends AbstractResolver {
@Override public String scheme() { return "smart"; }
@Override public Var<Addr> bind(String arg) {
return Vars.newConstVar(Addrs.newBoundAddr(Addresses.newInetAddress(
// assume this is more complicated and maps many names to
// one address
new InetSocketAddress("127.0.0.1", 9000)))));
}
}
class Main {
public static void main(String[] args) {
// Create a server, we record stats so we can see how many connections
// there are
InMemoryStatsReceiver stats = new InMemoryStatsReceiver();
ListeningServer server = Thrift.server()
.withStatsReceiver(stats)
.serveIface(":9000", (EchoService.ServiceIface) Future::value);
// create a few clients that all connect to a resolved server, the
// resolver ensures that they all are communicating with our server
EchoService.ServiceIface c1 = Thrift.client()
.newIface("smart!c1", EchoService.ServiceIface.class);
EchoService.ServiceIface c2 = Thrift.client()
.newIface("smart!c2", EchoService.ServiceIface.class);
// make sure any lazy connections have been opened
Await.result(c1.echo("c1"));
Await.result(c2.echo("c2"));
// I'm not sure how to see how many physical connections there are
// incoming to the server, or if it's possible to do this.
assertEquals(1, stats.counter(JavaConversions.asScalaBuffer(Arrays.asList("connects"))))
Await.result(server.close());
}
}
// echo.thrift
service EchoService {
string echo(1: string quack);
}
All the details
Where I work we have a microservice architecture using finagle and thrift, we use Consul for service discovery. Some of the external systems we interact with are very restrictive about the number and frequency of tcp connections they accept, for this reason some service instances are 'assigned responsibility' for these connections. To make sure that requests that need to use specific connections are sent to the correct service, that service registers a new service name in consul representing the connection it is responsible for. Clients then lookup the service by the connection name instead of the service name.
To make that clearer: Say you have a device-service that opens TCP connections to a configured list of devices, the devices only support a single connection at a time. You might have a few instances of this device-service with some scheme in place to divide the devices between the device-service instances. So instance A connects to devices foo and bar, instance B connects to baz.
You now have another service, say poke-service that needs to talk to specific devices (via the device-service). The way I've solved this issue is to have the device-service instance A register foo and bar, and instance B register baz, all against their own local address. So looking up foo resolves to the address for device-service instance A instead of the more generic cluster of all device-service instances.
The problem I'd like to solve is an optimisation problem, I'd like finagle to recognise that both foo and bar actually resolve to the same address and to reuse all the resources that it allocates and maintains as part of the connection. Additionally if foo and bar get re-assigned to different device-service instances, I'd like everything to just work based on the information in Consul which would reflect this change.
