Bug 1320890

Summary: The cpu-pinning with numa documentation should be more comprehensive while specifying which cpus to use for pinning and which isolate
Product: Red Hat OpenStack Reporter: Anand Nande <anande>
Component: documentationAssignee: Radek Bíba <rbiba>
Status: CLOSED CURRENTRELEASE QA Contact: Deepti Navale <dnavale>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0 (Kilo)CC: anande, dnavale, rbiba, srevivo
Target Milestone: ---Keywords: Documentation
Target Release: 8.0 (Liberty)   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-10 08:02:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Anand Nande 2016-03-24 09:39:14 UTC
Regarding the isolation issue:  documentation at
https://access.redhat.com/documentation/en/red-hat-enterprise-linux-openstack-platform/version-7/red-hat-enterprise-linux-openstack-platform-7-instances-and-images-guide/chapter-
7-configure-cpu-pinning-with-numa

explains this as follows:

It gives an example of host processes using cores 0, 1, 4, 5, and guest processes using cores 2, 3, 6, 7.  Then it says:

"Set the vcpu_pin_set option in the /etc/nova/nova.conf file to the list of CPU cores reserved for guest processes. Using the example above, you would set:
vcpu_pin_set=2,3,6,7"

A little later it says this:
"Restrict host processes to the selected CPU cores by adding the isolcpus argument followed by the list of CPUs to the boot options. Using the example above, you would run:
grubby --update-kernel=ALL --args="isolcpus=2,3,6,7"

If you read only this last sentence, you might think that this make the host processes use cores 2,3,6,7.  But we know from the example that it is doing the opposite;
it is making the host processes NOT use 2,3,6,7.  When it says "the selected CPU cores", it means the ones that the user has selected to in the example (0,1,4,5).
It doesn't mean the cores actually listed in this "grubby" command.

And customer finds that it works the way they are saying.  They think we should change the sentence in the documentation, to make it clear.

And then, as for the restriction they found: "If the target host has an isolation list, that list must include all the vCPUs that the migrating host is using on the source host."

Either 
A) fix migration to remove this restriction 
   (by recalcuating the vCPUs to be used [1]) or 
B) document this restriction on migration?

Note: when customer tested with isolate, they did this:

New values:

# 9-13^29-33 (10 vcpus) left for host:
vcpu_pin_set = 0,1,2,3,4,5,6,7,8,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,34,35,36,37,38,39

New values (match nova.conf):
grubby --update-kernel=ALL --args="isolcpus=0,1,2,3,4,5,6,7,8,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,34,35,36,37,38,39"

grub2-install /dev/sda

---

Is it possible to make the document more comprehensive? 


[1] https://bugs.launchpad.net/nova/+bug/1417667

Comment 2 Radek Bíba 2016-03-24 14:02:15 UTC
I'll modify the description of the grubby command and add a note about the restriction. I can't tell if the restriction can actually be removed because that's an engineering problem and that would require a separate engineering bug.

Comment 3 Radek Bíba 2016-03-24 14:59:06 UTC
How about this:

4. Ensure that host processes do not run on the CPU cores reserved for guest processes by adding the isolcpus argument to the system’s boot configuration. Use the list of CPU cores reserved for guest processes as a parameter of this argument. Using the topology from the example above, you would run the following command:

grubby --update-kernel=ALL --args="isolcpus=2,3,6,7"

Comment 9 Radek Bíba 2016-06-10 08:02:14 UTC
https://access.redhat.com/documentation/en/red-hat-enterprise-linux-openstack-platform/version-7/instances-and-images-guide/#ch-cpu_pinning has been updated.

The same changes have been made to the OSP 8 version of the doc. Also, the same warning now appears in the Migrating Instances guide.