Memory is managed in blocks known as pages. A standard page is 4096 bytes. 1MB of memory is equal to 256 pages; 1GB of memory is equal to 256,000 pages, etc. Those pages can be swapped to disk, etc. CPUs have a built-in memory management unit that contains a list of these pages, with each page referenced through a page table entry. Since the process address space are virtual, the CPU and the operating system have to remember which page belong to which process, and where it is stored. Obviously, the more pages you have, the more time it takes to find where the memory is mapped.
Most current CPU architectures support bigger pages (so the CPU/OS have less entries to look-up), those are named HugePages (on Linux), Super Pages (on BSD) or Large Pages (on Windows), but it all the same thing.
HugePages are a feature integrated into the Linux kernel 2.6. Enabling HugePages makes it possible for the operating system to support memory pages greater than the default (usually 4KB). Using very large page sizes can improve system performance by reducing the amount of system resources required to access page table entries. HugePages are useful for both 32-bit and 64-bit configurations. HugePage sizes vary from 2MB to 256MB, depending on the kernel version and the hardware architecture.
You should take into account that not every application can work with HugePages, and those which can, should be properly configured. In most cases using HugePages does make sense in relation to MySQL, PostgreSQL, Oracle, and JAVA.
root@svelab:~# grep ^Huge /proc/meminfo HugePages_Total: 4096 HugePages_Free: 3079 HugePages_Rsvd: 2822 HugePages_Surp: 0 Hugepagesize: 2048 kB
Size of the pool of huge pages.
Number of huge pages in the pool that are not yet allocated.
Number of huge pages for which a commitment to allocate from the pool has been made, but no allocation has yet been done. Reserved huge pages guarantee that an application will be able to allocate a huge page from the pool of huge pages at fault time.
Number of huge pages in the pool above the value in /proc/sys/vm/nr_hugepages. The maximum number of surplus huge pages is controlled by /proc/sys/vm/nr_overcommit_hugepages.
Size of a single memory page.
HugePages Configurator (hugecfg.sh) is the bash script which allows you to easily configure HugePages and show real-time values as well as settings in a config file. You do not need to memorize and calculate values for the Linux kernel parameters, this script will do all the needful for you.
|Version:||0.2 (check for the latest version)|
|Requirements:||GNU awk, grep|
Configurable user variables
Path to config. All kernel parameters will be saved to this file
Authorized group to use HugePages. Service accounts (for example, mysql, tomcat, etc.) must be members of this group to be allowed for using HugePages
Group ID of ‘hpGroup’. Change it to any value suitable for your OS or keep it as is
This amount of memory (in megabytes) will be added to the final amount of memory you want to allocate for HugePages. This variable is to simplify calculation of needs in the volume of HugePages
|hugecfg.sh help||Show all command line arguments (see below)|
|hugecfg.sh||Show real-time values (RAM, HugePages, Swap)|
|hugecfg.sh -s value||Configure HugePages (where ‘value’ in megabytes):|
|hugecfg.sh -g||Show HugePages config|
|cat /root/hugepages_reminder||Show a reminder (will be created automatically after configuring HugePages)|
The script creates the file /root/hugepages_reminder as a reminder with the following text:
- The memory associated with HugePages cannot be swapped to disk.
- The memory associated with HugePages are marked as used.
- If an application cannot take the needed amount of HugePages it will use standard pages.
- MySQL: add parameter ‘large-pages’ under [mysql] section in /etc/mysql/my.cnf
- MySQL: HugePages must be greater than ‘innodb_buffer_pool_size’ + all buffers
- JAVA: add parameter CATALINA_OPTS=”$CATALINA_OPTS -XX:+UseLargePages”
- JAVA: HugePages must be greater than ‘-Xmx’ + heaps
<a href=”https://svelab.com/2017/09/11/system-monitor/”>System Monitor</a>
Please, feel free to contact us if you have any questions or suggestions. Post a comment below if you want to report a bug.