0

I've written a c/c++ memory manager for heap allocations (overloaded new/delete and malloc/realloc/free, based on Doug Lea's malloc but designed to be wait free) and seem to be having some trouble with fragmentation. Are there any good resources out there that detail good strategies for avoiding fragmentation via the manager?

Please note that I can't rearrange memory that has already been allocated (not using smart pointers with GUIDs) and re-writing the system to use pools instead of heap allocations is unfeasible.

Thanks,
Grant

Grant Peters
  • 7,691
  • 3
  • 45
  • 57
  • BTW, how do you make dlmalloc wait-free without a major rewrite? I can understand making the small allocs in dlmalloc lock free using SLIST (Atomic list) primitives since they are just size-binned freelists but waitfree is a bit more difficult. Also, you have to be very carefully if you are trimming (returning memory to the OS) with lock-free and wait-free methods unless you use hazard pointers or garbage collection to avoid the race condition where you can dereferencing linked nodes that have been freed back to the OS. – Adisak Oct 26 '09 at 00:13
  • The system is just based of Doug Lea's concept (didn't want to go through and learn the code then alter it), but if you give each thread its own bins and blocks to allocate from (if you just try to do bins alone, then you will have problems combining consecutive blocks of memory together), then the allocation functions are already wait free. All you need to do then is handle deallocations in a wait free manner (which I did by just placing them all in a list to be properly deallocated by the owner thread). – Grant Peters Oct 26 '09 at 03:44
  • To request/return memory from the OS, I just used a mutex. This is the only part that isn't wait free, thus the only time threads can block each other is when they both request new memory from the OS at the same time (if something wants to release to the OS, but another thread is interacting with the OS, then it just skips this and will try again at a later point in the app). – Grant Peters Oct 26 '09 at 03:48

2 Answers2

2

You may want to get some inspiration from jemalloc (http://people.freebsd.org/~jasone/jemalloc/bsdcan2006/jemalloc.pdf) - this allocator is used in the new Firefox explicitly because of its anti-fragmentation capabilities.

Ana Betts
  • 73,868
  • 16
  • 141
  • 209
0

Have a look at how more mature projects like glibc do it.

A quick Google finds this, with a stack of references.

Adam Hawes
  • 5,439
  • 1
  • 23
  • 30
  • FWIW, dlmalloc is quite mature. Doug Lea has been working on it since 1987. Furthermore, he was the primary maintainer of the GNU C++ library (libg++). If you use the "standard" malloc in many C and C++ libraries, you are already using dlmalloc. – Adisak Oct 26 '09 at 00:05