1

From this post I realized that:

the smallest allocation that kmalloc can handle is as big as 32 or 64 bytes

and

The actual memory you get back is dependent on the system's architecture

But also memory page size is mentioned there and on other sites. I can't figure out how the page size is related to smallest kmalloc() allocation? The page size is usually 4096 bytes, but the smallest allocation is 32 or 64 bytes (depending on arch).

So what is the relation between the smallest kmalloc() allocation and page size? And why is the smallest allocation is 32 or 64 bytes but not 16 (e.g.)?

Marco Bonelli
  • 63,369
  • 21
  • 118
  • 128
narotello
  • 419
  • 2
  • 14
  • 1/ On MMU-enabled machines you will get granularity of MMU (defined by hardware). 2/ There is may be a software abstraction (like in Linux kernel) which provides better granularity for some cost (I don't know the area but can speculate that it has page for small allocations where they are managed like a linked list). Summarize: *there are (at least) two levels of granularity that play a role in small memory allocations*. – 0andriy Feb 27 '20 at 16:47
  • IMO you're looking at the wrong answer. @iGRJ has posted the relevant answer to your question: *"It all depends on the allocator used in your kernel. Slab, Slub or Slob"*. Linux uses slab/slub/slob allocator to handle objects smaller than a page. Have you looked into the history and how slab allocation (and its descendants) works? Slab allocation replaced the buddy system in SunOS/Solaris, because the buddy system did not use *processor* cache effectively. See last pages of https://www.cs.columbia.edu/~junfeng/10sp-w4118/lectures/l23-vm-linux.pdf – sawdust Feb 27 '20 at 22:15
  • @0andriy Could you answer my question? Otherwise I will be forced to accept the only answer. If you think it is incorrect, please explain it in detail. – narotello Mar 02 '20 at 08:30
  • The minimum granularity of the `kmalloc()` is `min = PAGE_SIZE / 128`, e.g. for x86 `4096/128 = 32`. The 2nd part of the question is related more to the other aspects, e.g. memory overhead. If we have an array of slabs, each 32 bytes will require at least the number and offset in the slab for it. I can imagine that it is some like size of integer (32-bits) at least, so, you'll have 12.5% overhead for minimal allocation. It's a big waste of memory and become huge for less allocations. Also there is a cache line size involvement if you think about SMP and DMA capable buffers. – 0andriy Mar 02 '20 at 14:15
  • @0andriy Can you give some link about `min = PAGE_SIZE / 128`? Where is it defined? – narotello Mar 03 '20 at 09:50
  • Read this comment (it appears that offset is only one byte in SLAB, but minimum size is 16 bytes, so, it's `PAGE_SIZE / 256`): https://elixir.bootlin.com/linux/latest/source/include/linux/slab.h#L294 – 0andriy Mar 03 '20 at 12:16

1 Answers1

0

So what is the relation between the smallest kmalloc() allocation and page size?

None. They are unrelated.

As you can see from the source code:

#ifdef CONFIG_SLAB
// ...
#ifndef KMALLOC_SHIFT_LOW
#define KMALLOC_SHIFT_LOW   5
#endif

#ifdef CONFIG_SLUB
// ...
#ifndef KMALLOC_SHIFT_LOW
#define KMALLOC_SHIFT_LOW   3
#endif
#endif

#ifdef CONFIG_SLOB
// ...
#ifndef KMALLOC_SHIFT_LOW
#define KMALLOC_SHIFT_LOW   3
#endif
#endif

// ...

#ifndef KMALLOC_MIN_SIZE
#define KMALLOC_MIN_SIZE (1 << KMALLOC_SHIFT_LOW)
#endif

None of the macros that end up defining KMALLOC_MIN_SIZE for different allocators depends on the page size, so there's no relation between page size and kmalloc() minimum allocation size.

On some architectures though, the minimum size can be different if kmalloc() memory is also used for direct memory access. That's why you see the various #ifndef above. It's still not related to page size though. 

/*
 * Some archs want to perform DMA into kmalloc caches and need a guaranteed
 * alignment larger than the alignment of a 64-bit integer.
 * Setting ARCH_KMALLOC_MINALIGN in arch headers allows that.
 */
#if defined(ARCH_DMA_MINALIGN) && ARCH_DMA_MINALIGN > 8
#define ARCH_KMALLOC_MINALIGN ARCH_DMA_MINALIGN
#define KMALLOC_MIN_SIZE ARCH_DMA_MINALIGN
#define KMALLOC_SHIFT_LOW ilog2(ARCH_DMA_MINALIGN)
#else
#define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long)
#endif

The value of ARCH_DMA_MINALIGN is architecture-specific and is usually related to the processor L1 cache size, as you can see for example for ARM:

#define L1_CACHE_SHIFT      CONFIG_ARM_L1_CACHE_SHIFT
#define L1_CACHE_BYTES      (1 << L1_CACHE_SHIFT)

/*
 * Memory returned by kmalloc() may be used for DMA, so we must make
 * sure that all such allocations are cache aligned. Otherwise,
 * unrelated code may cause parts of the buffer to be read into the
 * cache before the transfer is done, causing old data to be seen by
 * the CPU.
 */
#define ARCH_DMA_MINALIGN   L1_CACHE_BYTES
Marco Bonelli
  • 63,369
  • 21
  • 118
  • 128
  • You failed to answer the question. Why there is no relation? (See my comment to OP for the details). – 0andriy Feb 28 '20 at 15:34
  • @0andriy that's simply a design choice. Why should there be any relation anyway. – Marco Bonelli Feb 28 '20 at 15:51
  • You can't override hardware limitations in any case. You may put something on top, but it will bring security / data coherency / consistency issues (and here is another story how to solve or mitigate all of them). – 0andriy Feb 28 '20 at 15:57
  • 4
    @0andriy I'm not sure I follow you here. How does that have anything to do with the fact that kmalloc allocation size and page size are related or not? kmalloc min size is not related to page size, that's it, the code speaks for itself. I don't get what's more to discuss really. – Marco Bonelli Feb 28 '20 at 16:01
  • Via those implications (security, data coherency, etc). Because only page size granularity is guaranteed by hardware (MMU enabled, otherwise you may do whatever you want to with a certain cost of memory lost). Everything else is *software emulation*. – 0andriy Feb 28 '20 at 16:02
  • @0andriy so, first of all, OP is *not* asking "why is there no relation". OP is asking "is there relation?" - and I answered that extensively. Secondly, I don't get where you are aiming when talking about emulation, mmu and all that. If you want to answer, create another answer. If you want to downvote me then that's fine, but I did answer the question, and the answer is correct to the best of my knowledge. – Marco Bonelli Feb 28 '20 at 16:06
  • @0andriy you still are not explaining why you think hardware should be relevant when in the code there is no correlation between hardware and minimum kmalloc size except for what I explained above. – Marco Bonelli Feb 28 '20 at 23:24