2

I have an application using POE which has about 10 sessions doing various tasks. Over time, the app starts consuming more and more RAM and this usage doesn't go down even though the app is idle 80% of the time. My only solution at present is to restart the process often.

I'm not allowed to post my code here so I realize it is difficult to get help but maybe someone can tell me what I can do find out myself?

perlit
  • 329
  • 4
  • 13
  • 1
    See [Common Perl memory/reference leak patterns?](http://stackoverflow.com/questions/2223721/common-perl-memory-reference-leak-patterns) and [Perl memory usage profiling and leak detection](http://stackoverflow.com/questions/1359771/perl-memory-usage-profiling-and-leak-detection) and [How can I find memory leaks in long-running Perl programs?](http://stackoverflow.com/questions/429254/how-can-i-find-memory-leaks-in-long-running-perl-program) and [Are there any tools for finding memory leaks in my Perl program?](http://stackoverflow.com/questions/295385) – Ether Dec 09 '10 at 17:15

5 Answers5

2

Don't expect the process size to decrease. Memory isn't released back to the OS until the process terminates.

That said, might you have reference loops in data structures somewhere? AFAIK, the perl garbage collector can't sort out reference loops.

Are you using any XS modules anywhere? There could be leaks hidden inside those.

mu is too short
  • 426,620
  • 70
  • 833
  • 800
  • So if memory isn't released back to the OS then the usage'll continue to grow? There could be XS modules in the dependencies of one of the modules I'm using. I'm also using HTML::TreeBuilder whose documentations states I should explicitly call the delete() method to destroy it from memory which I'm doing already. – perlit Dec 09 '10 at 06:44
  • The size won't necessarily grow without limit. If there aren't any leaks then it would probably grow to a certain size and level off; unless of course memory gets too fragmented to be properly reused. Does it keep growing without limit or do you kill it when it gets too large for comfort? – mu is too short Dec 09 '10 at 07:18
  • It's running on a VPS, so once it uses about about 100 MB RAM, I end up killing it. I need to do this about 8 to 10 times a day. – perlit Dec 09 '10 at 07:26
  • Do you have a testing environment where you can isolate each of the POE tasks and see which one chews up all your memory? At least then you could narrow it down a bit. – mu is too short Dec 09 '10 at 07:58
1

A guess: your program executes a loop for as long as it is running; in this loop it may be that you allocate memory for a buffer (or more) each time some condition occurs; since the scope is never exited, the memory remains and will never be cleaned up. I suggest you check for something like this. If it is the case, place the allocating code in a sub that you call from the loop and where it will go out of scope, and get cleaned up, on return to the loop.

slashmais
  • 7,069
  • 9
  • 54
  • 80
1

Looks like Test::Valgrind is a tool for searching for memory leaks. I've never used it myself though (but I used plain valgrind with C source).

Dallaylaen
  • 5,268
  • 20
  • 34
1

One technique is to periodically dump the contents of $POE::Kernel::poe_kernel to a time- or sequence-named file. $poe_kernel is the root of a tree spanning all known sessions and the contents of their heaps. The snapshots should monotonically grow if the leaked memory is referenced. You'll be able to find out what's leaking by diff'ing an early snapshot with a later one.

You can export POE_ASSERT_DATA=1 to enable POE's internal data consistency checks. I don't expect it to surface problems, but if it does I'd be very happy to receive a bug report.

Rocco Caputo
  • 104
  • 3
1

Perl can not resolve reference rings. Either you have zombies (which you can detect via ps axl) or you have a memory leak (reference rings/circle)

There are a ton of programs to detect memory leaks. strace, mtrace, Devel::LeakTrace::Fast, Devel::Cycle

jami
  • 190
  • 2
  • 14