17

I thought that I couldn't retrieve the length of an allocated memory block like the simple .length function in Java. However, I now know that when malloc() allocates the block, it allocates extra bytes to hold an integer containing the size of the block. This integer is located at the beginning of the block; the address actually returned to the caller points to the location just past this length value. The problem is, I can't access that address to retrieve the block length.

#include <stdlib.h>
#include <stdio.h>
int main(void)
{
        char *str;
        str = (char*) malloc(sizeof(char)*1000);
        int *length;
        length = str-4; /*because on 32 bit system, an int is 4 bytes long*/
        printf("Length of str:%d\n", *length);
        free(str);
}

**Edit: I finally did it. The problem is, it keeps giving 0 as the length instead of the size on my system is because my Ubuntu is 64 bit. I changed str-4 to str-8, and it works now.

If I change the size to 2000, it produces 2017 as the length. However, when I change to 3000, it gives 3009. I am using GCC.

Amumu
  • 17,924
  • 31
  • 84
  • 131
  • 5
    What you describe is completely dependent on the specific malloc implementation storing the information in the way you describe. An implementation of malloc would not necessarily have to track how much memory was allocated (unlikely, but possible given that the API does not *require* this info to be exposed to the caller.) – Joe Mar 27 '11 at 18:01
  • 3
    @Joe: for example you could fairly easily implement a pool-based allocator in which, for example, all allocations of size 1-8 come from one pool, 9-16 from another, 17-32 from another, etc. Then instead of the size being stored before the usable memory, each allocation might instead store a pointer to its pool. Big allocations would have to be treated differently, though. – Steve Jessop Mar 27 '11 at 18:44

7 Answers7

22

You don't have to track it by your self!

size_t malloc_usable_size (void *ptr);

But it returns the real size of the allocated memory block! Not the size you passed to malloc!

Baum mit Augen
  • 49,044
  • 25
  • 144
  • 182
Informate.it
  • 243
  • 3
  • 2
  • 8
    It's important to note that this function is not part of the C standard, and is a gnulib extension – Degustaf Sep 24 '14 at 14:54
  • 1
    [Not only gnulib extension](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/malloc_size.3.html) – CAMOBAP Jan 01 '15 at 21:39
  • @Hydroper Definitely not. That detects GCC, not the GNU C library. Use `__GNU_LIBRARY__` instead. – S.S. Anne Sep 26 '19 at 19:36
17

What you're doing is definitely wrong. While it's almost certain that the word just before the allocated block is related to the size, even so it probably contains some additional flags or information in the unused bits. Depending on the implementation, this data might even be in the high bits, which would cause you to read the entirely wrong length. Also it's possible that small allocations (e.g. 1 to 32 bytes) are packed into special small-block pages with no headers, in which case the word before the allocated block is just part of another block and has no meaning whatsoever in relation to the size of the block you're examining.

Just stop this misguided and dangerous pursuit. If you need to know the size of a block obtained by malloc, you're doing something wrong.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • 3
    I see. Thanks. That's why it has different value for each size. I just try it to see how it works, and I won't do this in real things. However, there's an exercise in a book require to implement malloc() and free() based on its explained implementation, so I just wanna give it a try. – Amumu Mar 27 '11 at 19:22
  • Cool, +1 for doing this as part of experimentation to "figure out how it works and try to implement" rather than for some horrible hackery. – R.. GitHub STOP HELPING ICE Mar 27 '11 at 19:38
  • Do you know the way to filter out all the additional flags it append to the block, so I get the pure memory size? However, by guessing, I think I can't alter that, and have to altered my own way then. Also, I don't know why the sizeof(int) gives 4, while my system is 64 bits? Is there a way to let sizeof know my 64 bits system? – Amumu Mar 27 '11 at 19:51
  • 2
    `sizeof(int)` is always 4 on the vast majority of unix systems, and probably always will be. Use `long` or better yet `size_t` if you want the system word size. – R.. GitHub STOP HELPING ICE Mar 27 '11 at 20:11
  • 1
    @Amumu Nothing is dangerous if you know what you are doing, and the original poster is not misguided. It depends on the implementation. If you look at the implementation in the book of C programming language by Brian Kernighan, there is a general version of `malloc` and `free`. I am not sure it is the same version for GCC (most likely not). You can look into GCC's code, and see where they put the `Header`, and write your own function of getting the header info. It will not be a portable solution, but if you just use it yourself on the same compiler, it would be fine. – SwiftMango Oct 31 '12 at 14:26
  • 1
    @texasbruce: It is inherently dangerous and wrong unless you're programming for a specific version of the specific C implementation (library and compiler) you're going to be using, and nothing else. And even still, making that assumption is dangerous and wrong because it's likely that critical bugs will later be found in the implementation you're using, and that you'll need to upgrade, and if the assumptions you made based on the old version are no longer true with the new version, your program will break catastrophically. – R.. GitHub STOP HELPING ICE Oct 31 '12 at 14:44
  • I bridle when people have the lack of imagination to say there is no possible reason to justify doing something. There's always a plethora of valid reasons. In this case it's even safe on a wide variety of platforms. See malloc_usable_size() on linux and _msize() on windows. – Eloff Jul 17 '13 at 02:23
  • @Eloff: I suppose you never follow the glibc bug tracker. There were critical bugs in `malloc_usable_size` that made it not safe, but dangerous to use, under certain configurations: http://sourceware.org/bugzilla/show_bug.cgi?id=1349 https://bugzilla.redhat.com/show_bug.cgi?id=576431 https://code.google.com/p/skipfish/issues/detail?id=61 ... – R.. GitHub STOP HELPING ICE Jul 17 '13 at 14:06
13

I would suggest you create your own malloc wrapper by compiling and linking a file which defines my_malloc() and then overwiting the default as follows:

// my_malloc.c

#define malloc(sz) my_malloc(sz)

typedef struct {
    size_t size;
} Metadata;

void *my_malloc(size_t sz) {
    size_t size_with_header = sz + sizeof(Metadata);
    void* pointer = malloc(size_with_header);

    // cast the header into a Metadata struct
    Metadata* header = (Metadata*)pointer;
    header->size = sz;    
    // return the address starting after the header 
    // since this is what the user needs
    return pointer + sizeof(Metadata);
}

then you can always retrieve the size allocated by subtracting sizeof(Metadata), casting that pointer to Metadata and doing metadata->size:

Metadata* header = (Metadata*)(ptr - sizeof(Metadata));
printf("Size allocated is:%lu", header->size); // don't quote me on the %lu ;-)
Mike M
  • 4,879
  • 5
  • 38
  • 58
  • This is a good one workaround than looking into the gcc source code. – SwiftMango Oct 31 '12 at 14:27
  • 2
    Worth noting that the memory returned by your wrapper will no longer be aligned on most architectures. I should know, I wrote the same and my code relying on alignment blew up in my face. – Thomas Jun 28 '13 at 07:53
  • That makes sense and this code should only be used for debug purposes under strict control conditions. – Mike M Jul 26 '13 at 09:00
  • FYI pointer arithmetic with void* is not allowed so this code will fail to compile on compliant compiles (for instance, GCC with -pedantic). – Michael Graczyk Jun 03 '14 at 06:40
  • 1
    @Thomas To fix alignment you could use a union with a member of type max_align_t: typedef union { size_t size; max_align_t ma; } Metadata; – Fernando Costa Bertoldi Jun 16 '16 at 22:40
  • Interesting that in my linux (OpenSUSE 64) the code when substracting from int pointer to get the header with `sizeof(Metadata)` always returns me 0 size. But substracting with `- 2` works. – Ilian Zapryanov Jan 25 '17 at 14:03
  • I guess alignment could be solved by having the size field at the end of the block. – exebook Dec 26 '22 at 19:39
6

This is not Standard C. However, it is supposed to work on Windows operatings systems and might to be available on other operating systems such as Linux (msize?) or Mac (alloc_size?), as well.

size_t _msize( void *memblock );

_msize() returns the size of a memory block allocated in the heap.

See this link: http://msdn.microsoft.com/en-us/library/z2s077bc.aspx

Indinfer
  • 915
  • 9
  • 6
6

You're not supposed to do that. If you want to know how much memory you've allocated, you need to keep track of it yourself.

Looking outside the block of memory returned to you (before the pointer returned by malloc, or after that pointer + the number of bytes you asked for) will result in undefined behavior. It might work in practice for a given malloc implementation, but it's not a good idea to depend on that.

metamatt
  • 13,809
  • 7
  • 46
  • 56
3

This is implementation dependent

BlackBear
  • 22,411
  • 10
  • 48
  • 86
0

Every block you're allocating is precedeed by a block descriptor. Problem is, it dependends on system architecture. Try to find the block descriptor size for you own system. Try take a look at you system malloc man page.

Oleiade
  • 6,156
  • 4
  • 30
  • 42