RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1945002 - Enable CMA on x86-64 as tech preview
Summary: Enable CMA on x86-64 as tech preview
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: kernel
Version: 9.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: beta
: ---
Assignee: David Hildenbrand
QA Contact: Ping Fang
URL:
Whiteboard:
Depends On:
Blocks: 1950885
TreeView+ depends on / blocked
 
Reported: 2021-03-31 07:51 UTC by David Hildenbrand
Modified: 2023-08-08 03:04 UTC (History)
10 users (show)

Fixed In Version: kernel-5.13.0-0.rc3.25.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2043141 (view as bug list)
Environment:
Last Closed: 2021-12-07 21:55:02 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description David Hildenbrand 2021-03-31 07:51:46 UTC 
We want to enable/support CMA in RHEL9 on x86-64 (and eventually aarch64). Enabling CMA mostly involves enabling CONFIG_CMA and adding RHEL-specific warnings that CMA areas were defined / CMA allocations happened.

As one example CMA will be required to eventually support RDMA<->GPU p2pdma via dma-buf. As another example, CMA will be useful useful for more reliable runtime allocation of gigantic pages.

CMA is already enabled in ARK for s390x and ppc64le. For aarch64, 64k base page size currently implies a minimum CMA area size of 512 MiB (due to large pageblock order), which could be problematic on smaller machines.
Comment 4 David Hildenbrand 2021-04-14 10:01:22 UTC 
Related MR

https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1023
Comment 6 David Hildenbrand 2021-05-25 17:37:31 UTC 
Change has been merged into kernel-ark:

https://gitlab.com/cki-project/kernel-ark/-/commit/4c454e8be716ad8b7b529afd221f380447ea0f9a

Closing as RAWHIDE as the change is available in upstream kernel-ark.
Comment 7 David Hildenbrand 2021-05-25 19:46:55 UTC 
@Ping Fang I remember closing as RAWHIDE this is the right procedure for kernel-ark, but I wonder how to to handle QE for such things. Do you know what the right target state is once on upstream kernel-ark?
Comment 9 David Hildenbrand 2021-06-14 09:25:29 UTC 
Okay, thanks. Reopening this one as "TestOnly" for simplicity.
Comment 14 David Hildenbrand 2021-06-15 15:28:00 UTC 
@Ping Fang who'd be the right QE person to tackle testing this feature (hoping that there is capacity :) )? I can give some guidance regarding what/how to test.
Comment 18 David Hildenbrand 2021-06-17 08:08:33 UTC 
We can also test with gigantic page allocation via CMA. I assume a system with 64G memory,

1. Define "hugetlb_cma=16G" on the kernel cmdline

2. Boot the system and observe how much free memory there is (free -g), like 62G

3. Run some workload (e.g., memtester 62G) that consumes most of the free memory in the system to try shuffling free page lists.

4. Stop the workload.

5. Run a workload that leaves roughly 4G in the system free (e.g., memtester 58G)

6. While the workload is running, try allocating some gigantic pages.

echo 4 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

7. Observe if allocation succeeded

cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages


Might have to retry 6/7 a couple of times if such an extreme workload (memtester) is running concurrently. memtester might be an extreme workload as in mlocks all memory. I'll try out myself later if that workload is suitable for testing CMA here or if we need another one (like memhog).
Comment 19 David Hildenbrand 2021-06-17 08:09:46 UTC 
(In reply to David Hildenbrand from comment #18)
> We can also test with gigantic page allocation via CMA. I assume a system
> with 64G memory,
> 
> 1. Define "hugetlb_cma=16G" on the kernel cmdline
> 
> 2. Boot the system and observe how much free memory there is (free -g), like
> 62G
> 
> 3. Run some workload (e.g., memtester 62G) that consumes most of the free
> memory in the system to try shuffling free page lists.
> 
> 4. Stop the workload.
> 
> 5. Run a workload that leaves roughly 4G in the system free (e.g., memtester
> 58G)
> 
> 6. While the workload is running, try allocating some gigantic pages.
> 
> echo 4 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
> 
> 7. Observe if allocation succeeded
> 
> cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages


Sorry, both paths should target

/sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
Note You need to log in before you can comment on or make changes to this bug.