0

I have daemon with children on perl. For daemonize i use Proc::Daemon. Control daemon and children connected to DB(DBI lib), childs collect mail via imap (Mail::IMAPClient lib) too. I undef all variables in code of children, but over time, a child who works more than anyone, consume a lot of memory. ps output:

user   16521  0.6  1.6 135560 16516 ?        S    10:47   0:54 perl remote_imap.pl
user   16523  0.2 20.0 331976 201764 ?       S    10:47   0:21 perl remote_imapd.pl 16521
user   16525  0.1  3.0 157792 30720 ?        S    10:47   0:09 perl remote_imapd.pl 16521
user   16527  0.1  3.0 157796 30704 ?        S    10:47   0:08 perl remote_imapd.pl 16521
user   16529  0.1  3.0 157796 30572 ?        S    10:47   0:09 perl remote_imapd.pl 16521
user   16531  0.1  3.0 157792 30612 ?        S    10:47   0:08 perl remote_imapd.pl 16521

for sleep i use usleep from Time::HiRes lib. In remote_imap.pl usleep(100000), In remote_imapd.pl - usleep(500000).

Why memory is not released, because I did andef for all variables (except id_connet to DB)? If you want, I'll add code.

Main daemon remote_imap.pl Child remote_imapd.pl

Kostiuk Aleksandr
  • 509
  • 2
  • 14
  • 2
    add *minimal* code which is enough to reproduce the problem. – Karoly Horvath Dec 19 '14 at 11:29
  • 1
    Yup would need to see the code to understand where memory is allocated in the first place. – Chris Doyle Dec 19 '14 at 11:30
  • 4
    Check http://stackoverflow.com/questions/1359771/perl-memory-usage-profiling-and-leak-detection and http://learn.perl.org/faq/perlfaq3.html#How-can-I-free-an-array-or-hash-so-my-program-shrinks. – mpapec Dec 19 '14 at 11:39
  • Also you should be aware that assigning undef to an array doesn't work the same as with a scalar. @array = undef is not doing the same as undef(@array) – Chris Doyle Dec 21 '14 at 01:11

1 Answers1

2

Perl uses reference counting to manage it's memory space. When you declare a variable, a small amount is set aside. When you use it, more memory is occupied. Perl may reuse the memory space once a variable is no longer referenced at all.

That's why it's quite important to keep lexical scopes small, and avoid global variables - something global is 'in scope' throughout your code, even if it's only used once.

The gotcha is circular references. A circular reference never becomes unreferenced, so perl can't 'release' it automatically. That's also fairly easy to end up with inside objects - if part of an object has a reference to another part of the object.

That's why you have things like weaken().

I'll note in the examples you give above - the child process (why did you call the files the same thing?) you've lexically scoped:

my (@array, @is_ok, @params, $user_id, $user_remote_id, $is_delete, $error_count, $is_first, $server, $port, $login, $password, $is_ssl, $no_error, $error_text, $pid, $date, $date_up, $exists, $q, $sth, $rv);

Outside your while loop, and tried to 'tidy up' by:

(@array, @is_ok, @params, $user_id, $user_remote_id, $is_delete, $error_count, $is_first, $server, $port, $login, $password, $is_ssl, $no_error, $error_text, $pid, $date, $date_up, $exists, $q, $sth, $rv) = undef;

Out of interest - why are you doing this? Could you not scope them within your while loop, and then you won't need to reuse them? I don't know that's necessarily your problem though - it could quite easily be in one of the libraries you import. The fact that it's happening with the busier child implies that each loop something is getting to an array or object that's persisting.

Sobrique
  • 52,974
  • 7
  • 60
  • 101
  • I do not remember where, but I read that it is better to use global variables and do andef every cycle of `while`, before I use local variables. But the result is the same. I can add all of your code, with Lib.pm, if need. – Kostiuk Aleksandr Dec 19 '14 at 12:48