14

I recently saw the following post:

A memory allocator isn't lower level than malloc. (The default allocator typically calls malloc directly or indirectly)

An allocator just allows you to specify different allocation strategies. For example, you might use an allocator which calls malloc once to retrieve a large pool of memory, and then for subsequent allocation requests, it just returns a small chunk of this pool.

Or you may use it as a hook to allow you to perform some additional task every time memory is allocated or freed.

As to your second question, malloc is the lowest you can go without losing portability. malloc is typically implemented using some OS-specific memory allocation function, so that would be lower level still. But that's unrelated to your main question, since C++ allocators are a higher-level abstraction.

from: C++: Memory allocators

My question is- how is malloc implemented in the following Operating systems?

  • for Windows
  • for Linux

what are the OS-specific functions which are called/implementations of malloc()?

Community
  • 1
  • 1
user997112
  • 29,025
  • 43
  • 182
  • 361
  • for Linux, it's `sbrk` – Yu Hao Aug 03 '13 at 13:51
  • For which implementation of the Standard Library? `new` / `malloc` isn't necessarily part of the OS API itself, e.g. on Windows, which only has `HeapAlloc`, `VirtualAlloc` etc. as part of its API. – dyp Aug 03 '13 at 14:07
  • 3
    No, `sbrk` is nearly obsolete on Linux, it is using `mmap` – Basile Starynkevitch Aug 03 '13 at 14:11
  • You can use a fixed-size pool using a static array to provide an implementation of `malloc` or `new`. The answers currently refer to *how it's done for the most popular Standard Library implementations*. I'd say the question is not formulated well. – dyp Aug 03 '13 at 14:22
  • `malloc` *is a memory allocator*. It usually is built above virtual memory space management primitives (`mmap` on many Posix systems). – Basile Starynkevitch Aug 03 '13 at 15:03

5 Answers5

18

In Windows, in recent versions of MSVC, malloc (and C++ new, as it is implemented using the same fundamentals for the actual memory allocation part of new) calls HeapAlloc(). In other versions, such as g++ mingw, the C runtime is an older version, which doesn't call quite as directly to HeapAlloc, but at the base of it, it still goes to HeapAlloc - to find something different, we need to go back to Windows pre-95, which did have a GlobalAlloc and LocalAlloc set of functions - but I don't think people use 16-bit compilers these days - at least not for Windows programming.

In Linux, if you are using glibc, it depends on the size of the allocation whether it calls sbrk or mmap - mmap (with MAP_ANONYMOUS in the flags) is used for larger allocations (over a threshold, which I believe is 2MB in the typical implementation)

Mats Petersson
  • 126,704
  • 14
  • 140
  • 227
  • "In Windows, in recent versions" -> Do all implementations of the Standard Libraries (of C and C++) for Windows use that? (I know the MSVC one does) – dyp Aug 03 '13 at 14:14
  • Since G++ (mingw - not sure how Cygwin works) uses the MSVC (although an older version, which may not map as directly to HeapAlloc), yes. I'm not aware of any C or C++ library that uses anything other than HeapAlloc to actually get a lump of memory from Windows - unless we go back so far that it's using `LocalAlloc` and `GlobalAlloc`, but then we're talking 16-bit windows. – Mats Petersson Aug 03 '13 at 14:16
  • Our project uses ptmalloc() and dlmalloc() with MSVC on Windows which emulate sbrk() using VirtualAlloc(). – brian beuning Aug 03 '13 at 14:17
  • The mmap page seems to refer to mapping devices in to memory more than dynamically allocating memory? – user997112 Aug 03 '13 at 14:22
  • @user997112: Depends on what flags you pass in, and it's more about mapping a file into memory. But with `MAP_ANYNOMOUS` (along with correctly passing other parameters, as described in the link), it just gives the process a lump of memory. – Mats Petersson Aug 03 '13 at 14:24
5

My question is- how is malloc implemented in the following Operating systems?

On Linux there are two famous malloc implementations:

dlmalloc (Doug Lea's malloc)

ptmalloc

On Linux libc like glibc, eglibc or newlib implement ptmalloc or a variant of ptmalloc.

what are the OS-specific functions which are called/implementations of malloc()?

On Unix and Linux systems sbrk and mmap system calls are used. See man 2 sbrk and man 2 mmap for more information.

ouah
  • 142,963
  • 15
  • 272
  • 331
2

Alright, I am not sure about Linux, but when it comes to windows...

Memory can be allocated in two categorized places.

1) Heaps (Process Heap, Custom Created Heaps) see -> http://msdn.microsoft.com/en-us/library/windows/desktop/aa366711(v=vs.85).aspx using functions like HeapAlloc & HeapFree. LocalAlloc and LocalFree can be used as 'shortcuts' to HeapAlloc when you want to allocate in the default process heap.

2) Virtual Memory (usually only process-specific due to access restrictions in global virtual memory for security), using VirtualAlloc, VirtualFree. see -> http://msdn.microsoft.com/en-us/library/windows/desktop/aa366916(v=vs.85).aspx

To my knowledge, malloc will use the heap allocation functions on windows.

I hope this helps.

Dimitris Preketes
  • 83
  • 1
  • 8
  • "The global and local functions are supported for porting from 16-bit code" [from here](http://msdn.microsoft.com/en-us/library/aa366596%28v=vs.85%29.aspx) -> don't use `LocalAlloc` – dyp Aug 03 '13 at 14:09
  • Memory allocated with the `Heap*` functions is also "local" to the process. IMO, your answer suggests that a heap and virtual memory are two independent things - which they are not. A heap uses "virtual memory", i.e. virtual address space, and physical memory. So do the `Virtual*` functions, they don't only deal with virtual address space. – dyp Aug 03 '13 at 14:11
  • @DyP The first comment is right: don't use `LocalAlloc` anymore, that's a vestige of 16-bit Windows. Always use `HeapAlloc`. But the second part isn't *exactly* correct. Confusingly, even in Win32, the local and global heap are two slightly different things. Memory allocated from one cannot be freed on the other. So although you're correct that they are semantically similar and both "local" to the process, they're not identical. – Cody Gray - on strike Aug 03 '13 at 14:22
  • Also, there is little point in calling `VirtualAlloc` directly, as `HeapAlloc` will do it for you whenever necessary. The only time you should call `VirtualAlloc` directly is if you're doing some *really* low-level memory management stuff and need the extra options it provides. – Cody Gray - on strike Aug 03 '13 at 14:23
  • @CodyGray I didn't talk about global and local heaps, what are you referring to? What I meant with "local" is that every memory allocation with `VirtualAlloc` or `HeapAlloc` is process-specific. – dyp Aug 03 '13 at 14:28
2

malloc() and friends are considered part of the runtime system that comes with a compiler. So each compiler can and does use different OS calls to implement malloc.

As others have said, on Linux the options are sbrk() and mmap().

On Windows the options are HeapAlloc() and VirtualAlloc().

brian beuning
  • 2,836
  • 18
  • 22
0

On Windows, malloc implementations will usally call the win32 heap functions like HeapCreate, HeapDestroy, HeapAlloc, HeapFree. Those functions will call the NTDLL usermode heap manager located within ntdll.dll, those functions will have the RtlxxxHeap name, RtlAllocateHeap, RtlCreateHeap, etc...

In the end system calls within the NtxxxVirtualMemory group will be called, NtAllocateVirtualMemory, NtQueryVirtualMemory, NtFreeVirtualMemory.

Ugne
  • 71
  • 1
  • 2