We are doing a POC to prove that Vitess can scale massively and meet our requirements. We are using around 40 application servers, 15 VTGates and 30 shards (each shard contains master, replica and rdonly). However we were able to scale up to a point and above that point getting a flat line.
Main dark point for us is connecting application server and multiple VTGates. We have tried loadbalancer (AWS nlb) in between them and seen increased QPS but much lower TPS (~15000 QPS, ~ 1500-2000 TPS). Then we tired each application use JDBC connection pooling to connect VTGate without loadbalancer. We got similar results. Then we tried without connection pooling. Then we were able to increase TPS, however saw massive dips in QPS which affect the TPS.
As you can see we have hit a certain roadblock and need few brilliant ideas to overcome this. Really appreciate valuable inputs