Questions tagged [huge-pages]

Hugepages is a mechanism that allows the Linux kernel to utilize the multiple page size capabilities of modern hardware architectures.

Hugepages is a mechanism that allows the Linux kernel to utilize the multiple page size capabilities of modern hardware architectures. Linux uses pages as the basic unit of memory, where physical memory is partitioned and accessed using the basic page unit. The default page size is 4096 Bytes in the x86 architecture. Hugepages allows large amounts of memory to be utilized with a reduced overhead. Linux uses “Transaction Lookaside Buffers” (TLB) in the CPU architecture. These buffers contain mappings of virtual memory to actual physical memory addresses. So utilizing a huge amount of physical memory with the default page size consumes the TLB and adds processing overhead.

The Linux kernel is able to set aside a portion of physical memory to be able be addressed using a larger page size. Since the page size is higher, there will be less overhead managing the pages with the TLB. In the Linux 2.6 series of kernels, hugepages is enabled using the CONFIG_HUGETLB_PAGE feature when the kernel is built. Systems with large amount of memory can be configured to utilize the memory more efficiently by setting aside a portion dedicated for hugepages. The actual size of the page is dependent on the system architecture.

Source: http://linux.cloudibee.com/2007/09/linux-hugepages/

188 questions
4
votes
1 answer

Create huge page shared memory for IPC in linux

I have the following huge page create source code in linux: int iTotalByte = sizeof(datafeed)* ARRAYSIZE ; conf = (datafeed*) mmap(0, iTotalByte , (PROT_READ | PROT_WRITE), MAP_PRIVATE|MAP_ANONYMOUS|MAP_POPULATE|MAP_HUGETLB , -1 , 0) ; if(conf ==…
barfatchen
  • 1,630
  • 2
  • 24
  • 48
4
votes
0 answers

Linux Huge Pages and child termination by SIGBUS after fork()

In Linux, access to hugepages is provided through a virtual file system hugetlbfs. The libhugetlbfs library interface works with hugetlbfs to provide more convenient specific application-level services. libhugetlbfs can be used to make an existing…
UnSat
  • 1,347
  • 2
  • 14
  • 28
4
votes
2 answers

How to keep transparent hugepages config after reboot?

I used the below way to disable transparent hugepages. But they are restored (enabled again) after reboot. echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag echo no > /sys/kernel/mm/redhat_transparent_hugepage/khugepaged/defrag How to…
Bin
  • 83
  • 2
  • 8
3
votes
1 answer

How to open and use huge page and transparent huge page in code on Ubuntu

I want to use huge page or transparent huge page in my code to optimize the performance of data structure. But when I use the madvise() in my code, it Can allocate memory for me. There is always [madvise] never in…
dy66
  • 51
  • 3
3
votes
2 answers

DPDK sample application aborts after EAL: Couldn't get fd on hugepage file

After cloning the dpdk git repository and building the helloworld application, I get the following error: $ ./examples/helloworld/build/helloworld EAL: Detected 4 lcore(s) EAL: Detected 1 NUMA nodes EAL: Multi-process socket…
Nagev
  • 10,835
  • 4
  • 58
  • 69
3
votes
1 answer

How to release the hugepages allocated by DPDK application?

I am using DPDK-PROX application. Whenever I am closing that application, hugepages allocated by that application are not released. Every time I have to restart the system. Any solution? I looked into below question but my issue is not resolved. How…
Gokul
  • 73
  • 8
3
votes
0 answers

hugepages allocated by mmap is slower than posix_memalign

#include #include #include #include #include #define HUGEPAGE 2048*1024 void *normal_malloc(int len) { void *ptr = malloc(len); bzero(ptr, len); return ptr; } void…
Mr Pang
  • 1,083
  • 1
  • 8
  • 20
3
votes
1 answer

Huge number of "dTLB-load-misses" when DPDK forwarding test

Recently I am trying to do some forwarding test with DPDK "testpmd" application. And I find something interesting. When 512 descriptors are used for TX and RX, the performance is better than using 4096 descriptors. After checking the counters with…
3
votes
5 answers

Disabling THP (Transparent hugepages) with ansible role

I'm trying to disable THP via ansible on vagrant up, because it interferes with Redis (causes latency and memore usage issues with redis if enabled) The command to disable THP is "echo never > /sys/kernel/mm/transparent_hugepage/enabled" but it…
madvic
  • 123
  • 2
  • 13
3
votes
1 answer

What "deleted" means in /proc/$pid/maps?

I downloaded libhugetlbfs.so and have a simple test source : int glbarr[1024*1024]={0} ; int main() { char * ptr ; ptr = (char *) malloc( 1024 * 1024 * 1 ) ; printf(" press any key to go on \n"); getchar() ; for(int…
barfatchen
  • 1,630
  • 2
  • 24
  • 48
3
votes
0 answers

Continuous Full GC - Java 8

Cross Posting from openJDK mailing list. Why was Full GC (Allocation Failure) or for that matter a GC being triggered even when Eden Space was (almost) empty. I can see Metaspace being full, but then a GC due to that should have had Full GC…
rohitmohta
  • 1,001
  • 13
  • 22
3
votes
1 answer

huge page calculation in linux

I have the following config in linux : cat /proc/sys/vm/nr_hugepages 100 cat /proc/meminfo | grep "Page" AnonPages: 149012 kB PageTables: 13800 kB AnonHugePages: 4096 kB HugePages_Total: 100 HugePages_Free: …
barfatchen
  • 1,630
  • 2
  • 24
  • 48
3
votes
1 answer

Can't Write to /sys/kernel/ to disable Transparent Huge Pages (THP) for MongoDB on OVH CentOS 7

My Issue I am having trouble removing MongoDB warnings about Transparent Huge Pages (THP) on an OVH CentOS 7 installation, and the issue appears to be the inability to write to /sys/kernel/mm as root. First, I realize the OVH kernel is customized,…
Benjamin Ray
  • 1,855
  • 1
  • 14
  • 32
3
votes
3 answers

mmap failed when trying to map huge page (1GB)

What I did: Enable huge page with root (my system supports 1MB huge page) $ echo 20 > /proc/sys/vm/nr_hugepages Mount huge page filesystem to /mnt/hugepages $ mount -t hugetlbfs nodev /mnt/hugepages Create a file in huge page filesystem $ touch…
drdot
  • 3,215
  • 9
  • 46
  • 81
3
votes
1 answer

How to enable hugepages on Travis CI build

I tried following config lines and got Permission denied install: - sudo echo 2000 > /proc/sys/vm/nr_hugepages
Andriy Plokhotnyuk
  • 7,883
  • 2
  • 44
  • 68
1 2
3
12 13