+++ This bug was initially created as a clone of Bug #2043588 +++ The parent bug is used to discuss the implementation of a feature that would automate the best we can the handling of hugepages in OSP18 This bug will be used to document a procedure to mount the hugepages folders on the compute nodes during firstboot. Nova actually supports the handling of multiple pagesizes. This is how it can be done [1] using firstboot. It's important to note that pre-deployed nodes are probably not running the firstboot templates so it might fall on the operator's shoulder to run something similar during their node provisioning process. It's also important to note that these hugepage mounts need to be passed to the nova_compute and nova_libvirt containers. In this case, we create them under /dev which is automatically bindmounted to these containers, but if operators are choosing another destination, they need to make sure that it's passed to the containers as well. Here's a quick summary of what happens with this template: - We mask the default dev-hugepages.mount systemd unit file as we want to create new mounts using the pagesize. - We make sure that the folders are created beforehand - We create systemd mount units for each pagesizes - After the first loop, we need to systemd daemon-reload to include these newly created units - Finally, we start each one of these mounts [1] ~~~ (undercloud) [stack@undercloud-0 ~]$ cat 16.2_deployment_files/hugepages.yaml heat_template_version: queens description: > Hugepages configuration resources: userdata: type: OS::Heat::MultipartMime properties: parts: - config: {get_resource: hugepages_config} hugepages_config: type: OS::Heat::SoftwareConfig properties: config: | #!/bin/bash systemctl mask dev-hugepages.mount || true for pagesize in 2M 1G;do if ! [ -d "/dev/hugepages${pagesize}" ]; then mkdir -p "/dev/hugepages${pagesize}" cat << EOF > /etc/systemd/system/dev-hugepages${pagesize}.mount [Unit] Description=${pagesize} Huge Pages File System Documentation=https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt Documentation=https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems DefaultDependencies=no Before=sysinit.target ConditionPathExists=/sys/kernel/mm/hugepages ConditionCapability=CAP_SYS_ADMIN ConditionVirtualization=!private-users [Mount] What=hugetlbfs Where=/dev/hugepages${pagesize} Type=hugetlbfs Options=pagesize=${pagesize} EOF fi done systemctl daemon-reload for pagesize in 2M 1G;do systemctl start dev-hugepages${pagesize}.mount done outputs: OS::stack_id: value: {get_resource: userdata} (undercloud) [stack@undercloud-0 ~]$ cat 16.2_deployment_files/firstboot.yaml resource_registry: OS::TripleO::ComputeHugepage::NodeUserData: ./hugepages.yaml ~~~
I'm not sure if the described procedure in the issue description is self-sufficient, or kernelargs should be tweaked also? If so, we either need to clarify that in docs, or consider the RFE backport process (bz 2076498)
I propose to re-test with the following adjustment, assuming the computes hostname detection pattern should be adjusted by a user to match the reality: hugepages_config: type: OS::Heat::SoftwareConfig properties: config: | #!/bin/bash hostname | grep -qiE 'co?mp' || exit 0 systemctl mask dev-hugepages.mount || true for pagesize in 2M 1G;do if ! [ -d "/dev/hugepages${pagesize}" ]; then mkdir -p "/dev/hugepages${pagesize}" cat << EOF > /etc/systemd/system/dev-hugepages${pagesize}.mount [Unit] Description=${pagesize} Huge Pages File System Documentation=https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt Documentation=https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems DefaultDependencies=no Before=sysinit.target ConditionPathExists=/sys/kernel/mm/hugepages ConditionCapability=CAP_SYS_ADMIN ConditionVirtualization=!private-users [Mount] What=hugetlbfs Where=/dev/hugepages${pagesize} Type=hugetlbfs Options=pagesize=${pagesize} [Install] WantedBy = sysinit.target EOF fi done systemctl daemon-reload for pagesize in 2M 1G;do systemctl start dev-hugepages${pagesize}.mount done
Sorry, I forgot to amend the last lines: for pagesize in 2M 1G;do systemctl enable --now dev-hugepages${pagesize}.mount done
Updated content available on the Customer Portal: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/configuring_the_compute_service_for_instance_creation/assembly_configuring-compute-nodes-for-performance_compute-performance#proc_mounting-multiple-huge-page-folders-during-first-boot_compute-performance https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/configuring_the_compute_service_for_instance_creation/assembly_configuring-compute-nodes-for-performance_compute-performance#proc_mounting-multiple-huge-page-folders-during-first-boot_compute-performance