Bug 1570605

(In reply to Kashyap Chamarthy from comment #0)

[...]

> 
> A potential example of specifying multiple CPU feature flags
> ------------------------------------------------------------
> 
> If you specify:
> 
>     [libvirt]
>     cpu_mode=custom
>     cpu_model=IvyBridge
>     cpu_model_extra_flags="+pcid,-mtrr,pdpe1gb"
> 
> Then, Nova should generate the below XML:
> 
>     <cpu match='exact'> 
>       <model fallback='forbid'>IvyBridge</model>
>       <vendor>Intel</vendor>
>       <feature policy='require' name='pcid'/>
>       <feature policy='require' name='pdpe1gb'/>
>     </cpu>
> 
> The +/- for individual flags in nova.conf will be optional.  If nothing
> is specified, assume '+'.
> 
> You might ask: "Why would you want to remove a CPU flag though?"  One
> scenario for that is: An Operator wants to generate a baseline CPU
> config.  And a certain CPU flag is causing performance issue or other
> nuisance, and if the Operator isolated the problem to _that_ specific
> CPU flag, then she may want to remove the flag.

The existing patch only allows *adding* of CPU flags:

    https://review.openstack.org/#/c/563926/ -- libvirt: Lift the 
    restriction of choices for `cpu_model_extra_flags`

(Removal of CPU flags is to be added in a future patch.)

The patch is now merged in upstream "Rocky":

commit cc27a2007f314df18812d54a087f6192d6aed3fb
Author: Kashyap Chamarthy <kchamart>
Date:   Sat Apr 28 20:04:36 2018 +0200

    libvirt: Lift the restriction of choices for `cpu_model_extra_flags`
    
    Commit 6b601b7 (libvirt: Allow to specify granular CPU feature flags)
    added support for allowing to specify individual CPU feature flags, but
    restricted the options only to "PCID" (refer to its commit message for
    why).
    
    In this change we lift the restriction of choices, and allow to specify
    multiple CPU feature flags for all three CPU modes for the libvirt
    driver: 'custom', 'host-model', and 'host-passthrough'.
    
    For example:
    
         [libvirt]
         cpu_mode = custom
         cpu_model = IvyBridge
         cpu_model_extra_flags = pcid, vmx, pdpe1gb
    
    This will allow additional use cases such as:
    
      - Ability to use 1GB huge pages with models that don't provide it
        (such as Intel "Haswell" variants) as one use case for extra flags:
    
            cpu_mode = custom
            cpu_model = Haswell-noTSX-IBRS
            cpu_model_extra_flags = pdpe1gb
    
      - Nested Virtualization -- an operator can specify the Intel 'vmx' (or
        AMD 'svm') flags for the level-1 Nova guest CPU models.  (Assuming
        the 'nested' flag is enabled on the level-0 / bare-metal kernel.)
    
    (A future Nova patch will also allow ability to remove CPU flags.)
    
    Change-Id: I9a862619f379057bb48cb85a84dfc50d763030a6
    Signed-off-by: Kashyap Chamarthy <kchamart>
    BluePrint: libvirt-cpu-model-extra-flags

Test Evidence.

Environment: I tested this on Fedora in a DevStack environment with
nested KVM.  My Compute node is a level-1 guest, with
'host-passthrough' as its CPU mode.  And the level-2 guest, the Nova
instance has a custom model with specific CPU flags enabled.

Test
----

On the Compute node, in /etc/nova/nova.conf, configure a custom CPU
model:

    [libvirt]
    cpu_mode = custom
    cpu_model = Haswell-noTSX
    cpu_model_extra_flags = vmx, pdpe1gb
    virt_type = kvm
    [...]

Then start a CirrOS guest:

    [ComputeNode]$ openstack server create testvm1 \
        --image cirros-0.3.5-x86_64-disk --flavor 2

Ensure the guest is running

    [ComputeNode]$ openstack server list -f json
    [
      {
        "Status": "ACTIVE", 
        "Name": "testvm1", 
        "Image": "cirros-0.3.5-x86_64-disk", 
        "ID": "eb1ba291-93eb-4cde-b5e9-3386bfe5fa3e", 
        "Flavor": "m1.small", 
        "Networks": "private=fda0:77af:6ab0:0:f816:3eff:fe50:f6a5, 10.0.0.7"
      }

Then, ensure that the 'vmx' and 'pdpe1gb' CPU features are available in
the CPU configuration of the just-booted Nova instance
("instance-00000001"):

    $ sudo virsh dumpxml instance-00000001 | grep "cpu mode" -A12
      <cpu mode='custom' match='exact' check='full'>
        <model fallback='forbid'>Haswell-noTSX</model>
        <topology sockets='1' cores='1' threads='1'/>
        <feature policy='require' name='pdpe1gb'/>
        <feature policy='disable' name='vmx'/>
        [...]
      </cpu>

As a final confirmation, ensure that the 'vmx' and 'pdpe1gb' CPU
features show up as "-cpu Haswell-noTSX,pdpe1gb=on,vmx=on" as part of
the Nova instance's QEMU command-line:

[ComputeNode]$ ps -ef | grep qemu
qemu     17738     1 11 17:06 ?        00:00:03 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name guest=instance-00000001,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-instance-00000001/master-key.aes -machine pc-i440fx-2.9,accel=kvm,usb=off,dump-guest-core=off -cpu Haswell-noTSX,pdpe1gb=on,vmx=on -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid eb1ba291-93eb-4cde-b5e9-3386bfe5fa3e -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=18.0.0,serial=5cfd0699-ad8c-be09-a3a8-a4446a0aff89,uuid=eb1ba291-93eb-4cde-b5e9-3386bfe5fa3e,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-instance-00000001/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/stack/src/cloud/data/nova/instances/eb1ba291-93eb-4cde-b5e9-3386bfe5fa3e/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=31,id=hostnet0,vhost=on,vhostfd=33 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:50:f6:a5,bus=pci.0,addr=0x3 -add-fd set=2,fd=35 -chardev pty,id=charserial0,logfile=/dev/fdset/2,logappend=on -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045