4

Can anybody who has some basic idea on glibc malloc code please tell me how can i iterate over all the arenas and find out what are the chunks who are not freed i.e their inuse bit is set. This i have to do at the time of exit of the process.

or

more deterministically, if we have an arena, can we access the first chunk allocated in it?

thanks everybody for taking your time and responding back. I posted this question quite a long time back. 'Phrack' has some hacking techniques listed in there issues. I was benefitted from that.

regards, Kapil

casperOne
  • 73,706
  • 19
  • 184
  • 253
Kapil
  • 836
  • 2
  • 8
  • 20
  • Just for clarity: I presume the desire here is to determine unfreed and in-use regions of memory _without writing a separate allocator on top of glibc?_ The knee-jerk solution here is obvious: write (or borrow) an overlay memory manager and use that to handle the situation. – MrGomez Apr 09 '12 at 22:52
  • How low level do you want to get? The memory management APIs and system calls are different across compilers and platform. Windows, Linux, Mac OS X, etc... Heap allocation functions such as malloc are built on top of these system calls. – Jason Huntley Apr 13 '12 at 15:11

3 Answers3

5

All the major c code profilers usually have some kind of wrapper around malloc for memory tracking purposes. You will probably have to do the same if not something similar for tracking memory and having it platform independent.

Here are some examples:

keeping track of how much memory malloc has allocated

Simple C implementation to track memory malloc/free?

You will have to add additional structures for storing allocated memory references so you can go back and iterate over them. I imagine you will want to read up on algorithms related to memory cleanup. Mark N Sweep and Reference-Counting are the most popular algorithms used today. The JVM uses Mark N Sweep. You will also have to research strong and weak linking scenarios and how they could apply to your GC.

Otherwise, if you want to save time and not write your own wrapper, tools like valgrind and gprof can be used to profile and evaluate memory usage.

Honestly I would check out out Boehm-Demers-Weiser Garbage Collecter. It's already written and is based of extensive research.

Just noticed BDWGC moved over to GitHub.

Community
  • 1
  • 1
Jason Huntley
  • 3,827
  • 2
  • 20
  • 27
  • Hm. This is a more useful overview of [garbage collection strategies in general](https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)), but I'm concerned (as in my own answer) that the OP and the bounty provider want something that uses the glibc allocator directly. I like where this is going, but I think there's room to expand on this answer. Would you be especially alarmed, in the nature of good competition, if I did so in my own? – MrGomez Apr 13 '12 at 23:35
1

Based on the formulation of this question, it appears you're trying to reinvent an algorithm not unlike mark-sweep for garbage collection using glibc. The effort is noble, but garbage collectors exist to meet this need quite tidily, and referring to them will save you a tremendous amount of reimplementation effort if this is indeed your ultimate goal.

Meanwhile, the functionality you desire doesn't exist in the C specification, and implementing it in glibc is somewhat difficult and hacky. You'll want to consult your local glibc's version of malloc/malloc.c to determine the correct strategy if you desire to continue moving forward with your implementation, on account of different versions providing very different allocator-level guarantees. It can be done by piggybacking code on the allocator, but this doesn't seem to be the ideal solution to your expressed problem.

Despite being steeped in C++, this thread contains a treasure trove of information on how to write your own memory manager and how to evaluate a good reference implementation, which is a more workable strategy if you don't want to tether yourself to the internals of glibc. I would highly advise this strategy, because it future-proofs your application and abstracts away the functionality you desire (thus allowing you to drop in a different C library in the future).

Best of luck with your application(s).

Community
  • 1
  • 1
MrGomez
  • 23,788
  • 45
  • 72
0

Here is a BlackHat paper with a good description of glibc's internal structures used for memory heap allocation.
arena.c and malloc.c sources are another place worth looking at.

In the code, main_arena global is the root. main_arena.next points to the next arena (if any), main_arena.bins describe the bins with memory chunks.

VladV
  • 10,093
  • 3
  • 32
  • 48