Bug 1577088

Summary: [RHOS-13] Lift the restriction on choices for `cpu_model_extra_flags` config attribute
Product: Red Hat OpenStack Reporter: Kashyap Chamarthy <kchamart>
Component: openstack-novaAssignee: Kashyap Chamarthy <kchamart>
Status: CLOSED ERRATA QA Contact: nova-maint
Severity: medium Docs Contact:
Priority: medium    
Version: 13.0 (Queens)CC: berrange, dasmith, eglynn, jhakimra, kchamart, lyarwood, mbooth, nova-maint, sbauza, sferdjao, sgordon, srevivo, vromanso
Target Milestone: rcKeywords: Triaged
Target Release: 13.0 (Queens)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-nova-17.0.3-0.20180420001139.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1570605 Environment:
Last Closed: 2018-06-27 13:55:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1570605    
Bug Blocks: 1577093, 1591296    
Attachments:
Description Flags
nova logs
none
libvirt logs
none
instance logs none

Comment 6 Joe H. Rahme 2018-05-16 15:39:27 UTC

1. On the compute node, add the following to the [libvirt] section of
   nova.conf:

    [libvirt]
    cpu_mode = custom
    cpu_model = Haswell-noTSX
    cpu_model_extra_flags = vmx, pdpe1gb
    virt_type = kvm

2. Start an instance on this compute node.

3. Dump the instance XML, we can see the vmx and pdpe1gb policies in
   the cpu definition.


  $ virsh dumpxml instance-00000001
  ...
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Haswell-noTSX</model>
    <topology sockets='1' cores='1' threads='1'/>
    <feature policy='disable' name='pdpe1gb'/>
    <feature policy='disable' name='vmx'/>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='arat'/>
    <feature policy='disable' name='invpcid'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='abm'/>
  </cpu>
  ...

Comment 7 Joe H. Rahme 2018-05-17 13:13:46 UTC
Created attachment 1437979 [details]
nova logs

Comment 8 Joe H. Rahme 2018-05-17 13:26:32 UTC
Created attachment 1437982 [details]
libvirt logs

Comment 9 Joe H. Rahme 2018-05-17 13:28:45 UTC
Checking the command line arguments of the qemu process we can see vmx=on and pdpe1gb=on too

[heat-admin@compute-1 ~]$ ps -ef | grep qemu
root         665       1  0 May16 ?        00:00:00 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open,guest-file-close,guest-file-read,guest-file-write,guest-file-seek,guest-file-flush,guest-exec,guest-exec-status -F/etc/qemu-ga/fsfreeze-hook
qemu      299412  293722  4 11:24 ?        00:00:16 /usr/libexec/qemu-kvm -name guest=instance-00000001,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-instance-00000001/master-key.aes -machine pc-i440fx-rhel7.5.0,accel=kvm,usb=off,dump-guest-core=off -cpu Haswell-noTSX,pdpe1gb=on,vmx=on -m 512 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 27fa6edb-67c3-4ec8-8558-1e0ece59e9e9 -smbios type=1,manufacturer=Red Hat,product=OpenStack Compute,version=17.0.3-0.20180420001139.el7ost,serial=a81fb8af-efb7-466e-a161-0dd3582aa3f7,uuid=27fa6edb-67c3-4ec8-8558-1e0ece59e9e9,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-instance-00000001/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/instances/27fa6edb-67c3-4ec8-8558-1e0ece59e9e9/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=36,id=hostnet0,vhost=on,vhostfd=38 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:41:67:49,bus=pci.0,addr=0x3 -add-fd set=2,fd=40 -chardev pty,id=charserial0,logfile=/dev/fdset/2,logappend=on -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 172.17.1.15:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
heat-ad+  300357  299395  0 11:29 pts/0    00:00:00 grep --color=auto qemu

Comment 10 Joe H. Rahme 2018-05-17 13:38:45 UTC
Created attachment 1437984 [details]
instance logs

Comment 11 Kashyap Chamarthy 2018-05-17 13:55:05 UTC
(In reply to Joe H. Rahme from comment #6)
> 
> 1. On the compute node, add the following to the [libvirt] section of
>    nova.conf:
> 
>     [libvirt]
>     cpu_mode = custom
>     cpu_model = Haswell-noTSX
>     cpu_model_extra_flags = vmx, pdpe1gb
>     virt_type = kvm
> 
> 2. Start an instance on this compute node.
> 
> 3. Dump the instance XML, we can see the vmx and pdpe1gb policies in
>    the cpu definition.
> 
> 
>   $ virsh dumpxml instance-00000001
>   ...
>   <cpu mode='custom' match='exact' check='full'>
>     <model fallback='forbid'>Haswell-noTSX</model>
>     <topology sockets='1' cores='1' threads='1'/>
>     <feature policy='disable' name='pdpe1gb'/>
>     <feature policy='disable' name='vmx'/>

As DanPB also said on IRC, what you are seeing above seems to be an artifact of using nested virt in your environment.


From your Nova Compute log, Nova is actually doing the Right Thing and ensures the 'policy' attribute is "require":

  <cpu mode='custom' match='exact' check='partial'>
    <model fallback='allow'>Haswell-noTSX</model>
    <topology sockets='1' cores='1' threads='1'/>
    [...]
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='vmx'/>
  </cpu>

And your QEMU command-line from comment#9 shows, most importantly that the CPU flags are added:

    "-cpu Haswell-noTSX,pdpe1gb=on,vmx=on"

[...]

Comment 14 errata-xmlrpc 2018-06-27 13:55:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086