hugepages not working with mariadb

Question

We tried to setup the hugepage for mariabdb running with galera cluster. We had allocated hugepages high enough to accommodate the innodb buffer.

server.conf

[mysqld]
large-pages
innodb_buffer_pool_size =4G

/etc/sysctl.conf

# hugepages information
vm.nr_hugepages = 3072

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296

/etc/security/limits.conf

@mysql      soft    memlock     unlimited
@mysql      hard    memlock     unlimited

free -m

             total       used       free     shared    buffers     cached
Mem:         11908      11635        272          0         68       5502
-/+ buffers/cache:       6064       5843
Swap:         2559          0       2559

Errors in mysqld.log

150403  4:46:12 [Note] InnoDB: Initializing buffer pool, size = 4.0G
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
InnoDB: HugeTLB: Warning: Failed to allocate 687865856 bytes. errno 1
InnoDB HugeTLB: Warning: Using conventional memory pool
150403  4:46:13 [Note] InnoDB: Completed initialization of buffer pool

cat /proc/meminfo |grep -i huge

AnonHugePages:         0 kB
HugePages_Total:       3072
HugePages_Free:        3072
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Tried the script at the bottom of this page to calculate your memory requirements? http://dev.mysql.com/doc/refman/5.6/en/large-page-support.html — Norbert, Apr 06 '15 at 02:32
@NorbertvanNobelen When i try the scrip i get (standard_in) 2: syntax error — user1595858, Apr 06 '15 at 19:10

score 1 · Answer 1 · answered Oct 07 '15 at 19:33

There is one step that I can see is missing in your configuration: you need to set the group number (of mysql) which can access the HugeTLB memory.

Check the currently assigned value:

$ sysctl vm.hugetlb_shm_group

This would probably only return 0, so the "root" group. You should either create a hugetlb group and add mysql to this group or use one of the groups mysql belongs too. Whichever group, use its GID (use respectively grep '^hugetlb:' /etc/group | cut -d: -f3 or id -g mysql) and do:

# sysctl -w vm.hugetlb_shm_group=<gid>

(replace by the GID of your group)

Test if it works, it it does add to /etc/sysctl.conf (or in a new file under /etc/sysctl.d/) the following line:

vm.hugetlb_shm_group = <gid>

hugepages not working with mariadb

server.conf

/etc/sysctl.conf

/etc/security/limits.conf

Errors in mysqld.log

1 Answers1