jemalloc not detecting memory corruption

Question

The following program does not trigger an assert failure:

int main(int argc, char **argv)
{
  int * n = (int *)malloc(100);
  //malloc_stats_print(nullptr, nullptr, "gablh");
  free(n);
  *n += 1;
  std::cerr << *n << std::endl;
  for (int i = 0; i != 10; ++i) {
    std::cerr << *(n+i) << std::endl;
  }
}

When I run the program MALLOC_CONF="quarantine:32,abort:true,stats_print:true" ex_stats_pr

I get:

Is there a way to trigger an abort failure with jemalloc?

Amitabha, Is there any chance of jemalloc to do any checks? Your program do free, then tries to corrupt memory, print something and then exits, without any additional calls to jemalloc. For example, quarantine option should be used with valgrind, not just in plain run (valgrind may do some checks): http://linux.die.net/man/3/jemalloc "The quarantined memory is not freed until it is released from quarantine, ...This feature is of particular use in combination with Valgrind [2] , which can detect attempts to access quarantined objects." — osgx, Jun 09 '16 at 17:43

score 0 · Answer 1 · answered Jun 09 '16 at 17:26

This is not quite a direct answer to your question, but...

Undefined behavior is undefined. Playing with memory that has been freed falls into this camp. Depending on your implementation of your memory handlers, you may have a "validate memory" type function that walks along your free memory list to see if there has been a corruption of some sort, but even that isn't going to catch everything (not familiar with jemalloc in particular myself). It could be that your above code hits memory that no one care about so therefore won't be caught. Heck, your std::cerr statement is doing undefined behavior as well, so you cannot even trust its values (think threads and OS grabbing and changing memory, etc.)

This is one of the reason that you don't use pointer directly in C++ whenever possible. Smart pointers and containers that manage lifetime automatically prevent nearly all of these types of bugs.

score 0 · Answer 2 · answered Sep 16 '17 at 20:50

You expect that jemalloc detects one write into freed memory and some reads of freed memory.

But the jemalloc library doesn't have this capability. Its debug mode only detects a limited set of errors that lead to memory corruption. For example double-frees.

This isn't an arbitrary limitation, as a library like jemalloc just can't detect any kind of memory access error. That means as a library, it can easily overload malloc()/free() etc. and install an exit handler. A debug mode implementation thus can effectively implement a limited set of checks. And of course, each debug mode implementation chooses its own trade-offs. For example, jemalloc also doesn't detect simple buffer-overflows during free although other libraries with debugging capabilities (e.g. Solaris' libumem) implement a lightweight mechanism where the integrity of some special trailing bytes is checked.

For a library like jemalloc to detect reads/writes into freed memory it would have to install debugger-style watches into each freed region which would be quiet complicated and yield significant runtime overhead - if that would scale for many and large allocations, at all.

The point is: jemalloc is the wrong tool for detecting writes into freed memory (A) and reads of freed memory (B).

For example, the Address Sanitizer (-fsanitize=address) that comes with GCC and Clang is able to detect (A), but not (B). And Valgrind (valgrind --tool=memcheck) is able to detect both (A) and (B) and reports those issues as invalid reads/write into a freed block. Both tools certainly have higher runtime overhead than a simple debugging mode of a allocator library. And since valgrind's approach is to emulate a CPU its overhead is much higher than the one of Address Sanitizer.

jemalloc not detecting memory corruption

2 Answers2