2

Architecture : x86-64

Linux Version : 4.11.3

This is in reference to the below Stack Overflow post :-

Allocating more than 4 MB of pinned contiguous memory in the Linux Kernel

I see that the question was asked for a PCI driver, which requested for more than 4 MB of contiguous memory in the kernel. However, my intention was to use another function in place of kzalloc_node function (or modify it!). I want to modify the kernel code (if feasible) so that somehow I can allocate more than 4 MB of contiguous memory, which kzalloc_node does not allow me to do. Of course, it will be difficult to modify MAX_ORDER as it may give rise to compiler errors. Also here the kzalloc_node function is computing the node corresponding to the CPU - so the allocation of memory happens at the node level.

Background

Basically I am trying to increase the size of a sampling buffer so as to reduce the overhead that it incurs when it gets full and interrupts need to be raised to read in the data from the buffer. So I am trying to reduce the number of interrupts and thereby, need to increase the size of the buffer. The kernel code is using kzalloc_node to allocate memory, and hence it cannot get more than 4 MB contiguous memory. I want to know what mechanisms I have to either replace this function/allocate more memory ?

Can I replace this function ? Since I am trying to modify the kernel code, do the same boot-time allocation methods apply here ? I read that this mechanism applies for device drivers, can I also use it ?

Arnabjyoti Kalita
  • 2,325
  • 1
  • 18
  • 31
  • Is it allowed by the hardware interface of PEBS to have large buffer? What is the exact code place where `kzalloc_node` is called in the sampling used by you (is it http://elixir.free-electrons.com/linux/v4.11.5/source/kernel/events/ring_buffer.c#L593 or other)? Basically if you want more physical memory, use right interface https://lwn.net/Articles/486301/ - "*The Contiguous Memory Allocator (or CMA), which LWN looked at back in June 2011, has been developed to allow allocation of big, physically-contiguous memory blocks.*", and just add zeroing if needed – osgx Jun 15 '17 at 23:38
  • Hi @osgx, as far as I know, there is no mention of a restriction on the **PEBS buffer** in the Intel documentation. It is a `kzalloc_node` function as usual, but the correct place is (http://elixir.free-electrons.com/linux/v4.11.5/source/arch/x86/events/intel/ds.c#L281). I am currently reading the buddy allocation and slab allocation mechanism employed by Linux. I want to see if there is some modification possible that would allow me to allocate something in the order of ~Gigabytes. I will read about CMA also. – Arnabjyoti Kalita Jun 16 '17 at 00:02
  • Hi @osgx, I read about CMA. The CMA context is aimed at addresses that the DMA can handle. Moreover, to allocate the CMA memory, one calls functions (**dma_alloc_from_contiguous**) that takes a `device` as a parameter. I am not quite sure if I can use CMA in this case. – Arnabjyoti Kalita Jun 17 '17 at 18:56
  • DMA can access physical memory. Which memory uses (supports) PEBS hw engine: virtual or physical? Why it needs "contiguous memory"? I think it is like DMA and will use (physical) memory allocated for DMA. Where is PEBS programming manual? There is from Nehalem: https://software.intel.com/sites/default/files/m/5/2/c/f/1/30320-Nehalem-PMU-Programming-Guide-Core.pdf#page=35 "PEBS Buffer Base .. Software should allocate this memory from the non-paged pool.". About device parameter: there is lower level CMA API include/linux/cma.h & use kernel command line to preallocate some memory for your PEBS. – osgx Jun 17 '17 at 22:15

0 Answers0