1

How can I tell Perl, built with glibc, to release the memory back to the operating system thus shrinking the resident size?

I seem to recall the being possible, but do not remember how it was done.

Can I for instance, nuke a typeglob and free all the space allocated inside it?

Looking at the perl faq it says,

You usually can't. Memory allocated to lexicals (i.e. my() variables) cannot be reclaimed or reused even if they go out of scope. It is reserved in case the variables come back into scope. Memory allocated to global variables can be reused (within your program) by using undef() and/or delete(). On most operating systems, memory allocated to a program can never be returned to the system. That's why long-running programs sometimes re- exec themselves. Some operating systems (notably, systems that use mmap(2) for allocating large chunks of memory) can reclaim memory that is no longer used, but on such systems, perl must be configured and compiled to use the OS's malloc, not perl's.

So I want to make sure my perl is compiled with glibc and that I have the conditions to do it. The docs say "you usually can't", well I have glibc and I want to be able to do it.

perl -V | grep -i malloc
    usemymalloc=n, bincompat5005=undef
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
Evan Carroll
  • 78,363
  • 46
  • 261
  • 468
  • 1
    I thought that one can't do that? The memory for lexicals is used by the interpreter as it pleases (retained for "later needs"). Found [this post](https://stackoverflow.com/a/1242849/4653379) with an excellent quote so far.. – zdim Oct 13 '17 at 19:57
  • Afaik, It's *possible* if you're using glibc, I don't know how or the drawbacks. – Evan Carroll Oct 13 '17 at 19:58
  • 1
    I just tried the following one-liner under Ubuntu, with a perl-5.26 usemymalloc=n (a vanilla Perlbrew): `perl -E'sub mem{system qq(ps u --pid $$)}; mem; { my $x = q(x)x(2**30); mem; undef $x; }; mem'`. This allocates a 1GB string at compile time (constant folding), then copies the string, then releases the variable. Without the `undef` the memory is kept until the end, but with the `undef` the copied string is freed. – amon Oct 13 '17 at 20:07
  • @amon Just (bravely) tried that on a nice server here with v5.16.3 ... after `undef`-ing RSS did drop but it still kept a lot. With your example it dropped by about a half (?), with `2**10` by about 25%. – zdim Oct 13 '17 at 20:13
  • @zdim The string exists twice: once as the string literal `q(x) x (2**30)` which was expanded by constant folding, then the copy. When the copy is `undef`ed, the copy is freed but the literal still resides in memory. So we'll see at most a 50% drop for that example. The `q(x) x (2**10)` is merely 1KB, so the few KB memory overhead of perl itself reduces the visibility of the effect. – amon Oct 13 '17 at 20:18
  • @amon Good point, thanks. So it does release it. This is still at interpreter's mercy, no? Don't know how more complex data structures would fare. (Tried with a simple hashref and it worked.) – zdim Oct 13 '17 at 20:21

1 Answers1

3

Going off of the Build and Installation guide Install, under Malloc issues it says

To build without perl's malloc, you can use the Configure command

sh Configure -Uusemymalloc

The build/test/install outline is given right at the top and this would replace its first line.

There is also the module Perl::Configure to ease and automate the process.

Also note experimentation by amon in comments, who found that the interpreter does release a good deal of memory from undef-ed variables. In a modified test with a hashref even just dropping out of scope, without undef, gets the same memory relief. (See a comment below.)

I don't know whether (or how much) any of this can be relied on, in particular with complex data structures, as (in my understanding) lexicals' allocations are at the interpreter's mercy.

Community
  • 1
  • 1
zdim
  • 64,580
  • 5
  • 52
  • 81
  • “Also, just dropping out of scope, without undef, gets the same memory relief.” – No, the above experiment demonstrates that the memory is only released when the variable is undef'd. Remove the `undef`, and the memory remains allocated until the Perl interpreter exits. – amon Oct 14 '17 at 19:40
  • 2
    @amon I did (would've not written that just so). However, as I re-tested now I see that I tried without `undef` _with a hashref_. It drops the same w/ or w/o `undef`: `perl -we'sub mem{system qq(ps u --pid $$)}; mem; { my $x = { k => q(x)x(2**30) }; mem } sleep 1; mem'`. I tested on a combination of v5.10.1, v5.16.3, and v5.27.2 on a desktop, laptop and server. With a scalar it indeed needs the `undef`. I am updating the misleading statement, thank you. – zdim Oct 15 '17 at 00:33