The recommendation is to use just one GremlinClient
object and to reuse that across your application because the client uses a connection pool so the same connection can be used again for different requests instead of having to create and tear down one connection for each request. The driver also supports request pipelining since version 3.4.0 which means that the same connection can be used for different requests in parallel which reduces the number of required connections. That can of course also only work if you reuse the client.
The problem you describe however could be caused by a bug in the Gremlin.Net driver. If you are using version 3.4.0 of Gremlin.Net, then it could be this bug that can lead to a huge number of created connections. In that case, you can avoid it by downgrading to a 3.3.5 until we release a fix with version 3.4.1. If you also see the problem in version 3.3.5, then please create an issue in TinkerPop's issue tracker to describe the problem.
There could of course also be another reason for the server to close the connection. That would need to be investigated in more depth to understand the reason, e.g., with a tool like Wireshark by inspecting the network traffic between the driver and Cosmos DB.