Tomcat Performance with Huge String Responses

Question

I've a springRest web service endpoint that returns a string text of size 4MB. As we do load test of this endpoint we constantly see heap spikes and ultimately the system crashes. I'm thinking - as we make requests each request is serviced by a thread separately. My hypothesis is: Because the string is saved in a global static variable, each thread takes a copy of the 4MB and after around 3000 requests the heap is all consumed and the system crashes because 3000 threads taking each 4MB is around 12GB and hence the system goes out of memory. But this is my hypothesis.

My question: doesn't tomcat reclaim the memory after each thread that processes a request has done it's job? Is this related to GC (garbage collection)? In the request life cycle - as a request comes, a thread is created (per that request) does the thread get it's own copy of the response or it just references the response? if that huge string response is copied to each thread then may be that's why the heap spike is showing. When the response is given back to the client how does tomcat reclaim the resources of that thread? when does it do it? is claiming request threads related to GC?

Another aspect that i observed is: delay on the method socketWrite0() - this takes from 70-95% of the response time. It is a bottle neck i think. So in the flow of request response - who writes to the socket? the thread? or the thread hands the response to tomcat and tomcat writes it?

If any of you could give me a hint or an aspect to look at that relates memory spikes with huge string responses, i'd really appreciate it. thanks guys!

rose

score 2 · Answer 1 · answered Oct 08 '12 at 18:32

2

If you have declared a static final String the variable should be shared between threads. Without any code it's pretty much impossible to say what's going on.

Have you tried using a profiling tool like JProfiler or mat?

answered Oct 08 '12 at 18:32

Tim Lamballais

1,056
5
10

Thanks Tim. I'm using VisualVM for profiling. We also do dynatrace to do the load/performance tests. I'm reading the memory spikes from dynatrace report. – Rose Oct 08 '12 at 21:09
I seem to remember VisualVM allowing you to inspect the heap. So at the time of a peak you should see lots of large String Objects in memory. – Tim Lamballais Oct 09 '12 at 13:18

score 2 · Answer 2 · answered Oct 08 '12 at 18:33

I assume what you do is

response.getWriter().write(hugeString);

if that causes problem, tomcat is to blame, it probably never expected such huge strings.

You can try to cut the string into smaller ones and write only one small string each time, e.g.

int N = hugeString.length();
int CHUNK = 8*1024;
for(int i=0; i<N; i+=CHUNK)
    int end = Math.min(i+CHUNK, N);
    writer.write( hugeString.substring(i, end) );

score 0 · Accepted Answer · answered Oct 24 '12 at 01:29

Thanks guys for the comments given. The problem was very cryptic because it comes only when the system is stressed with multiple users on performance test. We tried different changes to cache client, caching strategies and cache time to live, connection time out.

We tried changing the algorithm. Caching more data to avoid expensive calls to legacy systems.

We tried changing the data structure from JSONArray to String.

All didn't help!!

Finally we figured out it was the load balancer that was creating the issue. We disabled it and it all worked fine.

If i'm given a chance to say one thing: performance/scalability of an enterprise system is bottle necked by the less performant component, so do a careful empirical testing and attack the problem after spending time figuring out the correct pain point. DO NOT RUN TO CHANGE YOUR CODE.

Tomcat Performance with Huge String Responses

3 Answers3