7

Consider the following snippet:

#include <stdio.h>
#include <stdlib.h>
#include <readline/readline.h>

int main() {
    for (;;) {
        char *buf = readline(">>> ");
        if (!buf)
            break;

        free(buf);
    }
}

Compiling with -lreadline, executing the program under valgrind and inputting some lines result in an enormous memory leak, on my system valgrinds verdict looks something like this:

==7651== LEAK SUMMARY:
==7651==    definitely lost: 0 bytes in 0 blocks
==7651==    indirectly lost: 0 bytes in 0 blocks
==7651==      possibly lost: 0 bytes in 0 blocks
==7651==    still reachable: 213,455 bytes in 217 blocks
==7651==         suppressed: 0 bytes in 0 blocks

Running with --show-leak-kinds=all results something like this (the whole thing is several hundred lines long, I'll only show the beginning):

==7693== 5 bytes in 1 blocks are still reachable in loss record 1 of 57
==7693==    at 0x483777F: malloc (vg_replace_malloc.c:299)
==7693==    by 0x48CE409: xmalloc (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A72E6: rl_set_prompt (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A87E6: readline (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x10915C: main (in /home/.../a.out)
==7693== 
==7693== 5 bytes in 1 blocks are still reachable in loss record 2 of 57
==7693==    at 0x483777F: malloc (vg_replace_malloc.c:299)
==7693==    by 0x48CE409: xmalloc (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48B95BC: ??? (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48B9D25: rl_expand_prompt (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A7309: rl_set_prompt (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A87E6: readline (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x10915C: main (in /home/.../a.out)
==7693== 
==7693== 8 bytes in 1 blocks are still reachable in loss record 3 of 57
==7693==    at 0x483777F: malloc (vg_replace_malloc.c:299)
==7693==    by 0x496C49E: strdup (in /usr/lib/libc-2.28.so)
==7693==    by 0x4AEEDCD: _nc_trim_sgr0 (in /usr/lib/libncursesw.so.6.1)
==7693==    by 0x4AE7EA2: tgetent_sp (in /usr/lib/libncursesw.so.6.1)
==7693==    by 0x48C39BC: _rl_init_terminal_io (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A851C: rl_initialize (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A87EC: readline (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x10915C: main (in /home/.../a.out)

Now despite the fact that the memory is not lost, the fact that readline simply does not free huge chunks of memory before program exit seems completely absurd to me. Am I missing something? Should I manually call some poorly documented cleanup functions? Is this a bug? Does this occur on every system?

There seem to be several similar issues floating around the Internet but I was surprised to see that this happens even in the simplest possible use case.

EDIT: because there has been a lot of discussion I will clarify a little: bruno's answer is of course correct, it's not a memory leak in the traditional sense and on almost all platforms it won't matter at all (I included the Linux tag, which was a mistake, I have removed it now), but I'd still like to know if this is really intentional or if happens because the memory is freed only after valgrind produces its statistics and if there is any way to get around this (or at the very least make valgrind ignore it so that it does not obscure missing free calls in the rest of my code)

chqrlie
  • 131,814
  • 10
  • 121
  • 189
Peter
  • 2,919
  • 1
  • 16
  • 35
  • _enormous memory leak_ where ? there is no memory leak – bruno Mar 16 '19 at 12:01
  • 3
    "Still reachable" is just memory that wasn't freed when the program exited. It's not a leak. – Shawn Mar 16 '19 at 12:11
  • 2
    I wouldn't call 200kB "enormous". – melpomene Mar 16 '19 at 12:31
  • @melpomene it was when being on a 48k memory (souvenir of "Goupil with a 6502" ) ^^ – bruno Mar 16 '19 at 12:40
  • @melpomene If average line is 16 bytes, then it leaks 1280000% of the required memory. –  Mar 16 '19 at 12:54
  • @StaceyGirl What do you mean, line? The valgrind output shows the memory is used to store the prompt and terminal settings (by curses). – melpomene Mar 16 '19 at 13:00
  • @melpomene OP just reads input line by line. Since people prefer to type less, 16 byte is reasonable. –  Mar 16 '19 at 13:05
  • 4
    "*the fact that readline simply does not free huge chunks of memory before program exit seems completely absurd to me*" - What seems absurd to me is programs trying to free memory when they're about to exit anyway. The OS will throw away your whole address space; why bother setting some bits to mark blocks as "free" first? – melpomene Mar 16 '19 at 13:05
  • @StaceyGirl None of the memory shown is even used for input. What you're saying is irrelevant. – melpomene Mar 16 '19 at 13:05
  • @melpomene Because the code the question is minimal example. I wonder if you are one of those people who like to complain about browsers consuming too much memory, yet 200kB to process dozen of bytes is somehow negligible. –  Mar 16 '19 at 13:07
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/190145/discussion-between-melpomene-and-staceygirl). – melpomene Mar 16 '19 at 13:08

1 Answers1

15

GNU readline: enormous memory leak

==7651== LEAK SUMMARY:
==7651==    definitely lost: 0 bytes in 0 blocks
==7651==    indirectly lost: 0 bytes in 0 blocks
==7651==      possibly lost: 0 bytes in 0 blocks

There is no memory leak

==7651==    still reachable: 213,455 bytes in 217 blocks
==7651==         suppressed: 0 bytes in 0 blocks
...

A reachable memory is not a memory leak

==7693== 5 bytes in 1 blocks are still reachable in loss record 1 of 57
==7693==    at 0x483777F: malloc (vg_replace_malloc.c:299)
==7693==    by 0x48CE409: xmalloc (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A72E6: rl_set_prompt (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x48A87E6: readline (in /usr/lib/libreadline.so.8.0)
==7693==    by 0x10915C: main (in /home/.../a.out)
...

getline used memory and still references it, do not forget it manages a history, and the history can be freed calling void rl_clear_history (void), add that function call in your program and redo a test

Function: void rl_clear_history (void)
  Clear the history list by deleting all of the entries, in the same
  manner as the History library's clear_history() function. This differs
  from clear_history because it frees private data Readline saves in the history list.
bruno
  • 32,421
  • 7
  • 25
  • 37
  • 8
    If amount of leaked memory grows with input size, reachable memory IS a memory leak too. –  Mar 16 '19 at 12:09
  • 1
    @StaceyGirl no because it is reusable by _getline_ for next calls – bruno Mar 16 '19 at 12:10
  • 2
    Irrelevant, leakage that grows linearly with input is a leak because it will prevent process from working in the end. –  Mar 16 '19 at 12:11
  • 2
    @StaceyGirl that readline is not as simple as _gets_, it manages historic, so _of course_ it uses memory to save the historic, if you don't need that just do not use it ;-) – bruno Mar 16 '19 at 12:18
  • 1
    Then this is exactly what OP was asking about. He clearly understands the concept of reachable memory. In its current form, your answer doesn't answer the question. –  Mar 16 '19 at 12:21
  • 4
    @StaceyGirl again the title is about (enormous) memory leaks and there is no memory leak – bruno Mar 16 '19 at 12:26
  • 1
    OP asks how to release that memory because he doesn't need it. Clearly he doesn't want to use lower-level APIs like `fgets` that require managing memory manually. –  Mar 16 '19 at 12:28
  • What constitutes a memory leak is up to interpretation in this case. There are certainly cases when not releasing memory before program exit can have negative consequences, depending on the system in question (and even if it is not harmful it is certainly unclean imo, why can readline not internally register an exit handler and guarantee that all memory allocated is freed again?). `rl_clear_history` does not seem to influence this behaviour. – Peter Mar 16 '19 at 13:58
  • 1
    To elaborate on my first remark: the C standard does NOT guarantee that reachable memory will be released when the program terminates, it is entirely up to the OS to take care of that and assuming that this will always happen is imo dangerously naive if you consider embedded platforms etc. – Peter Mar 16 '19 at 14:02
  • @Peter sorry but for me you are out of subject, and even what about the tag linux in the question ? – bruno Mar 16 '19 at 14:08
  • 1
    Yeah you're right, considering the tag I should not have gone on about other platforms. On Linux this should not be a problem (I knew that), although I still don't really like it from an aesthetic standpoint. – Peter Mar 16 '19 at 14:10