We are evaluating ISTIO for one of the projects. In the project, we have around 200 containers and around 200 services. Each container might be brought up with replica set to 2. So, there could be 400 containers in total and hence 400 ISTIO side cars.
With default settings, each ISTIO side car is using up 300Mbytes. With 400 side cars, this will result to 120Gbytes (300Mbutes * 400) of memory for side cars. That seems to be even higher than the memory requirements needed for the application.
There are two observations
- Memory usage seems to be going up with number of cores. On the system, we are trying, has 88 cores. Current understanding that it takes around 1.5 to 2Mbutes for each core.
- Memory usage seems to be going up with increasing number of services. In the project we are working on, we expect to see around 200 services. It seems to be occupying around 3Mbytes for each service.
On (1), we have found (from googling) a solution. Solution is to set the envoy concurrency to smaller number.
On (2), is there any solution? In our project, it is not that every container talks to every other container/service. A given container can talk to set of destination services (typically around 5 or so). Is it possible for side car of a given container to allocate memory for services that the container uses?
I am wondering whether anybody has similar issue with ISTIO and how they got around the memory consumption issues.
