1

Many a carefully crafted piece of Java code has been laid to waste by java.lang.OutOfMemoryError. There seems to be no relief from it, even production class code gets downed by it.

The question I wish to ask is: are there good programming/architecture practices where you can avoid hitting this error.

So the tools at a Java programmers disposal seem to be:

  1. java.lang.Runtime.addShutdownHook(Thread hook) -- shutdown hooks allow for a graceful fall.
  2. java.lang.Runtime.freeMemory() -- allows us to check the memory available to the VM

So the thought I had is: could one write factory methods which before the creation of objects check if the system has adequate memory left before attempting to allocate memory? For example in C, the malloc would fail and you would know that you had run out of memory, not an ideal situation but you wouldn't just drop dead from an java.lang.OutOfMemoryError aneurism.

A suggested approach is to do better memory management or plug memory leaks or simply allocate more memory -- I do agree these are valuable points but lets look at the following scenarios:

  1. I'm running on an Amazon micro instance
  2. I can allocate very little memory to my VM say 400M
  3. My Java process processes jobs in a multi-threaded fashion, each thread consumes a variable amount of memory depending on the parameters of the computational task
  4. Let's assume that my process has no memory leaks
  5. Now if I keep feeding it jobs before they are complete it will eventually die of memory starvation
  6. If I set -Xmx too high -- I'll get swapping and possibly thrashing on the OS
  7. If I set an upper concurrently limit -- that might not be optimal as I could be limiting the accepting of a job which could be executed with the available RAM or worse accept a job that requires a LOT of memory and end up hitting java.lang.OutOfMemoryError anyway. X. Hope that helps explain the motivation of the question -- I think the standard responses are not mutually exclusive to seeking a fault tolerant approach to the problem.

Thanks in advance.

user1172468
  • 5,306
  • 6
  • 35
  • 62
  • 1
    Switch to a 64-bit JVM and install loads and loads of memory? You're running out of memory because you're allocating too much (on purpose or due to a leak). First find out if you're leaking, if not try to be more efficient. Or switch to a 64-bit JVM with loads of memory. – zmbq Sep 27 '12 at 22:10
  • Hi @zmbq, thanks for the response, I'll include notes based on your response. – user1172468 Sep 27 '12 at 22:11
  • 4
    Even if there's a way to see if you're about to run out of memory, how is the logic of your app going to handle it? The problem is as @zmbq says: your app needs a lot of RAM, in which case you either need to make it so it doesn't use so much RAM, put in more RAM, or go 64-bit; or you're leaking memory in which case you should fix your code -- even with more RAM you'll eventually run out. I don't think this is a very good question, but it can be improved if, for example, you really do use a lot of RAM, you can explain what it's for and maybe we can offer suggestions how to reduce usage. – gregmac Sep 27 '12 at 22:13
  • 1
    what are the performance tradeoffs of constantly performing the leftover memory check upon every object creation? i seem to be more inclined to reduce the exposure to the error during the design phase, load testing, as well as implementing limits to data payload that could lead to the error. but this is an eternally great subject. look forward to reading more responses. – amphibient Sep 27 '12 at 22:19
  • Hi @gregmac, if you look at my updated question -- I hope that will explain the scenario. So for example if I am RAM bound I could poll the available RAM before allocating a new object, if the available RAM is less than some set parameter either do a lazy spin lock or reject the request. The idea is that you have objects eating up RAM that are transient through the system, they will eventually be released but just at this second I don't want to allocate more objects. – user1172468 Sep 27 '12 at 22:21
  • @foampile, I honestly do not know the answer to that, it's a good point -- I do not know how expensive the freeMemory() method is. Now if we compare it to malloc in C -- malloc will return a 0 if it cannot allocate memory and this is done at every dynamic memory allocation which happens a LOT in any non-trivial piece of code. – user1172468 Sep 27 '12 at 22:22
  • as far as i know, there is no way to check how much usable memory is left due to garbage collection. GC usually run when it is near the limit or depending on the scheme. – gigadot Sep 27 '12 at 22:23
  • sorry - I reread the question and see you have an absolute memory limit. So what I wrote doesn't really apply. – Sam Goldberg Sep 27 '12 at 22:25
  • Actually - it occurred to me that you thought this could be handled in C. So how would you design your multithreaded app in C to prevent running out of memory? Just checking malloc return wouldn't help app keep running, since any of the threads could hit the memory limit at any random point in time, and then you would need to code some kind of recovery mechanism. I think the answer might be to design this in C, and see if same approach could work in Java. I don't think it's a trivial solution even in C. – Sam Goldberg Sep 28 '12 at 13:15

2 Answers2

2

We handled JVM memory more as a tuning parameter, than as something to manage actively with the application. We have a MemoryInfo class (which wraps several of the Runtime memory info methods).

While application is running, we track free memory in the application as:

 Runtime.getMaxMemory() - Runtime.getTotalMemory() + Runtime.getFreeMemory();

Max memory is the -Xmx jvm arg, total memory is what JVM has already allocated to the application heap, and free memory is how much of the allocated heap memory still available. (If your -Xms parameter is the same as your -Xmx parameter, then getFreeMemory() is all you need to check).

If we get above 70% memory usage, we send alerts to our monitoring system. At that point we make a decision whether we can limp through the rest of the day, or whether adjust the -Xmx parameter and restart. Although this seems a bit messy, in practice, once we have tuned a system, we never run into memory problems after that. (Once you get above 90% max memory used, the JVM will GC extremely frequently to try to prevent running out of memory).

I think the approach of managing memory with every construction is draconion, but if you need absolute control, then maybe it makes sense. Another approach is to make sure that any memory caches you use have an LRU or Expiration and reload mechanism, so you can better limit the number objects preserved in memory.

That said, our approach is to keep as much as possible in memory, and just allocate plenty of RAM. Our big systems have 28G RAM allocated (we use betwen 40-60% of that on average).

Sam Goldberg
  • 6,711
  • 8
  • 52
  • 85
  • thanks for the response -- here are my comments: 1. I do agree, when I'm running on my big Iron -- I tune allocate lots of RAM -- then pray (is the part I'm not very comfortable with) ... 2. I agree checking freemem at every object allocation may be unrealistic 3. However alternatives could be implemented such as a house keeping tread sets a static variable in a utility class to stopAllocatingNewObjects=true when you hit a certain threshold ... and when you fall below another threshold you flip it back to flase. Factories check that flag before creating objects. – user1172468 Sep 27 '12 at 22:28
  • Further, the question really aroze when I'm working with stream job processing on lots of AWS micro instances. Thanks. – user1172468 Sep 27 '12 at 22:32
  • @user1172468: I wrote the response before I saw your limitations. Based on what you said, I would try a simple approach of trying to determine a minimum free memory required before starting a job, or else some other kind of object counting which gives you an idea of whether the thread needs to sleep until more is free. – Sam Goldberg Sep 27 '12 at 22:32
  • I do agree with what you've said. My intention was to see if we can get the community to enumerate established approaches to this problem. Thanks again for the engaging discussion. – user1172468 Sep 27 '12 at 22:35
  • @SamGoldberg -- in which part of your app do you perform this check? just curious where you have it situated and how frequently you perform it. thanks – amphibient Sep 28 '12 at 00:01
  • 1
    @foampile: We have a background thread runnning which performs periodic statistics report on the app. The memory stats are reported once a minute (which is good enough for our app). The key of course is generating an alert if we pass 70% JVM memory usage. – Sam Goldberg Sep 28 '12 at 13:08
  • thanks, Sam -- i thought it was something to that tune. i like your setup. – amphibient Sep 28 '12 at 13:48
1

OK, so the solution, as I have previously suggested, is to require less memory. There's no point in running an unlimited amount of threads because your process gets multiple requests. Limit the number of threads in each process, and handle at most that number of requests concurrently. The rest of the requests will simply wait.

Since you don't have an unlimited number of cores, too many threads are a bad idea, anyway.

zmbq
  • 38,013
  • 14
  • 101
  • 171
  • hi @zmbq, so the process need not be CPU bound ... for example a scraper, the thread is going to be blocked most of the time, the memory may be depended on the size of the page it is scraping, so limiting the number of threads guarantees neither: 1. that you will not run into memory exhaustion --OR-- 2. that you will optimally use your computational resources. – user1172468 Sep 28 '12 at 02:02
  • 1
    Well, if you don't know how much memory a single thread would require, you can run out of memory by processing just one request. – zmbq Sep 28 '12 at 06:07
  • @user1172468: Have you done any profiling (e.g. using Jprobe or MyKit) see how memory is increased at various points in your app. It sounds from your last comment that the page size being scraped might be a good indicator of how much memory is needed for a job. Can you use page size as a simple metric to forecast memory need? Some times statistical probability is good enough (and absolute certainty not needed). – Sam Goldberg Sep 28 '12 at 13:06
  • @Sam, so here what I've thought of based on your comment. 1: I'm going to try to put together a simulator -- the proof is in the pudding. 2: I think commonly used approach is robust when you are dealing with large memory sizes in relation to your workload -- I suspect it is not as robust (as I'm finding out from some Cloud based apps I'm working on now). 3: Using forecasting you can't run your system to near optimal levels, you have to leave a lot of head room for redundancy. – user1172468 Sep 28 '12 at 16:53