dmalloc output "not freed: '0x7f2e20d36808|s1' (1182 bytes) from 'unknown'" when using 'printf' and 'fgetc'

Question

We have been using dmalloc as part of our tools set to verify that our core libraries are free of memory leaks. However, recently we have discovered that using printf or fgetc would cause dmalloc to throw the following warnings in dmalloc.log.

not freed: '0x7f2e20d36808|s1' (1182 bytes) from 'unknown'

To demonstrate the issue, below is a very simple program that we have used to reproduce the errors:

#include <stdio.h>
#include <stdlib.h>
#include "dmalloc.h"

void main() {
        dmalloc_debug_setup("log-stats,log-non-free,check-fence,log=dmalloc.log");
        printf("Hello World\n");
        fgetc(stdin);
}

Compile the program with:

gcc test.c -ldmalloc -g -o test

Run and I get the following results:

1630587465: 2: Dmalloc version '5.5.2' from 'http://dmalloc.com/'
1630587465: 2: flags = 0x403, logfile 'dmalloc.log'
1630587465: 2: interval = 0, addr = 0, seen # = 0, limit = 0
1630587465: 2: threads enabled, lock-on = 0, lock-init = 2
1630587465: 2: starting time = 1630587464
1630587465: 2: process pid = 2079
1630587465: 2: Dumping Chunk Statistics:
1630587465: 2: basic-block 4096 bytes, alignment 8 bytes
1630587465: 2: heap address range: 0x7f343a6e0000 to 0x7f343a712000, 204800 bytes
1630587465: 2:     user blocks: 4 blocks, 8192 bytes (33%)
1630587465: 2:    admin blocks: 2 blocks, 8192 bytes (33%)
1630587465: 2:    total blocks: 6 blocks, 24576 bytes
1630587465: 2: heap checked 0
1630587465: 2: alloc calls: malloc 2, calloc 0, realloc 0, free 0
1630587465: 2: alloc calls: recalloc 0, memalign 0, posix_memalign 0, valloc 0
1630587465: 2: alloc calls: new 0, delete 0
1630587465: 2:   current memory in use: 8192 bytes (2 pnts)
1630587465: 2:  total memory allocated: 8192 bytes (2 pnts)
1630587465: 2:  max in use at one time: 8192 bytes (2 pnts)
1630587465: 2: max alloced with 1 call: 4096 bytes
1630587465: 2: max unused memory space: 8192 bytes (50%)
1630587465: 2: top 10 allocations:
1630587465: 2:  total-size  count in-use-size  count  source
1630587465: 2:           0      0           0      0  Total of 0
1630587465: 2: Dumping Not-Freed Pointers Changed Since Start:
1630587465: 2:  not freed: '0x7f343a6f0008|s1' (4096 bytes) from 'unknown'
1630587465: 2:  not freed: '0x7f343a710008|s1' (4096 bytes) from 'unknown'
1630587465: 2:  total-size  count  source
1630587465: 2:           0      0  Total of 0
1630587465: 2: ending time = 1630587465, elapsed since start = 0:00:01

Remove the fgetc I got only one warning with memory not being freed.

1630587578: 1: Dumping Not-Freed Pointers Changed Since Start:
1630587578: 1:  not freed: '0x7f939f4e0008|s1' (4096 bytes) from 'unknown'
1630587578: 1:  total-size  count  source
1630587578: 1:           0      0  Total of 0

Remove the printf then I got:

1630587655: 0: Dumping Not-Freed Pointers Changed Since Start:
1630587655: 0:  memory table is empty
1630587655: 0: ending time = 1630587655, elapsed since start = 0:00:00

The gcc version being used here is 9.3.0 and the dmalloc is 5.5.2 which comes with most latest Linux distros when install using their package manager.

Can anyone experienced with dmalloc help to point me out if this is a known problems with dmalloc or I'm missing some obvious settings here?

Update

This is also observed when compiled with g++.
Some people have pointed out that the library 5.5.2 is quite old. I have build the latest version 5.6.5 and test with gcc 11.2.0 and the problem still persists.

1630589888: 2: Dmalloc version '5.6.5' from 'http://dmalloc.com/'
1630589888: 2: flags = 0x403, logfile 'dmalloc.log'
1630589888: 2: interval = 0, addr = 0x0, seen # = 0, limit = 0
1630589888: 2: starting time = 1630589886
1630589888: 2: process pid = 11185
1630589888: 2: Dumping Chunk Statistics:
1630589888: 2: basic-block 4096 bytes, alignment 8 bytes
1630589888: 2: heap address range: 0x7ff82bd98000 to 0x7ff82bd9b000, 12288 bytes
1630589888: 2:     user blocks: 1 blocks, 2048 bytes (16%)
1630589888: 2:    admin blocks: 2 blocks, 8192 bytes (67%)
1630589888: 2:    total blocks: 3 blocks, 12288 bytes
1630589888: 2: heap checked 0
1630589888: 2: alloc calls: malloc 2, calloc 0, realloc 0, free 0
1630589888: 2: alloc calls: recalloc 0, memalign 0, valloc 0
1630589888: 2: alloc calls: new 0, delete 0
1630589888: 2:   current memory in use: 2048 bytes (2 pnts)
1630589888: 2:  total memory allocated: 2048 bytes (2 pnts)
1630589888: 2:  max in use at one time: 2048 bytes (2 pnts)
1630589888: 2: max alloced with 1 call: 1024 bytes
1630589888: 2: max unused memory space: 2048 bytes (50%)
1630589888: 2: top 10 allocations:
1630589888: 2:  total-size  count in-use-size  count  source
1630589888: 2:           0      0           0      0  Total of 0
1630589888: 2: Dumping Not-Freed Pointers Changed Since Start:
1630589888: 2:  not freed: '0x7ff82bd9a008|s1' (1024 bytes) from 'unknown'
1630589888: 2:  not freed: '0x7ff82bd9a808|s1' (1024 bytes) from 'unknown'
1630589888: 2:  total-size  count  source
1630589888: 2:           0      0  Total of 0
1630589888: 2: ending time = 1630589888, elapsed since start = 0:00:02

Added link to a issue raised with dmalloc. See here

Agreed. It looks like the library has just been updated last year. That's probably why the later version hasn't been included as the default on most Linux distros. — Tung Pham, Sep 02 '21 at 13:20
Some library functions dynamically allocate memory and then hold on to it until program exit. That's out of your control. You might consider trying Valgrind as an alternative to dmalloc. Valgrind has a non-intrusive usage model, and your installation package will likely know how to suppress most reports arising from benign standard library behaviors. — John Bollinger, Sep 02 '21 at 13:25
Thanks John. We have already been using valgrind in our tools set. However, it is more of a manual step. In our CI we use dmalloc so that it can run nicely in Debug mode and the output can be easily verified. However, with these issues,, we are seeing whether dmalloc is still the tool for the task or we should just stop using it and switch to something else. — Tung Pham, Sep 02 '21 at 13:35
@user1810087: I have updated my question with further details on how it also failed in the latest version of dmalloc. — Tung Pham, Sep 02 '21 at 13:45
The error here is likely in the implementation of the library, not that of dmalloc. If the library does not free memory before the program exits, that *is* an error. — DevSolar, Sep 02 '21 at 13:45
@DevSolar: Thanks. However, dmalloc is for Linux/Unix like system and if basic standard functions like printf/fgetc are not being considered, how can we trust the dmalloc output. Surely, we don't do printf or fgetc unless we are implementing an interactive application but there are potentially many other standard functions that might have been implemented a similar way. Going through all of the false positive warnings until getting to the actual cause must be very ineffective way of detecting memory leaks if there actually are. — Tung Pham, Sep 02 '21 at 13:54
@DevSolar: No, that's a bit nonsensical. `printf` is part of the same library as `malloc`, and the library writer may assume that they cooperate. That includes relying on behavior that generic code cannot rely on. If you replace half of that library, the resulting issues are your own fault. — MSalters, Sep 02 '21 at 13:55
My point is that, using that library implementation, you *do* habe a memory leak. It is not a *false* positive just because the leak happens in a standard function... — DevSolar, Sep 02 '21 at 13:56
@MSalters: I see no way that a custom implementation of `malloc()` could turn the error in `printf()` into a non-error. — DevSolar, Sep 02 '21 at 13:59
@DevSolar: Even talking about "error" does not make sense. The Standard Library is **the** top-level library; it supports `atexit` handling for instance, `printf` must work in exit handlers, and `atexit` registration may require `malloc`. There's no error, nor a leak in the Standard Library code - it returns all its memory to the OS, **after** `atexit` handling. **You** must call `free` to return memory to the Standard Library, potentially **during** `atexit` handling. `dlmalloc` runs into problems because there is no "after atexit" phase. — MSalters, Sep 02 '21 at 14:05
@MSalters: Neither `printf()` nor `atexit()` *require* malloc. And I'd much easier believe that the library assumes memory protection and memory space cleanup at exit than I'd believe in an `atexit()` cleanup step. Also, "after `atexit()`" is when the last bets for `printf()` not being bugged are off, because after that the streams that `printf()` could be used for close. — DevSolar, Sep 02 '21 at 14:55

score 0 · Accepted Answer · answered Sep 09 '21 at 21:47

I will answer my own question here. After more testing, I can confirm that this behaviour is also observed from other libraries such as jemalloc or mtrace. The conclusion is that, certainly the implementation of 'libc' is different on different platforms, and in this scenario the implementation for Linux (observed on my Debian and Ubuntu for Windows) does not free the allocated buffer before the last point that these libraries can check.

Thus, this is not a bug in dmalloc. Other tool such as 'valgrind' will be a better suit for this case.

dmalloc output "not freed: '0x7f2e20d36808|s1' (1182 bytes) from 'unknown'" when using 'printf' and 'fgetc'

1 Answers1