1. On the compute node, add the following to the [libvirt] section of
   nova.conf:

    [libvirt]
    cpu_mode = custom
    cpu_model = Haswell-noTSX
    cpu_model_extra_flags = vmx, pdpe1gb
    virt_type = kvm

2. Start an instance on this compute node.

3. Dump the instance XML, we can see the vmx and pdpe1gb policies in
   the cpu definition.


  $ virsh dumpxml instance-00000001
  ...
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Haswell-noTSX</model>
    <topology sockets='1' cores='1' threads='1'/>
    <feature policy='disable' name='pdpe1gb'/>
    <feature policy='disable' name='vmx'/>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='arat'/>
    <feature policy='disable' name='invpcid'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='abm'/>
  </cpu>
  ...

Created attachment 1437979 [details]
nova logs

Created attachment 1437982 [details]
libvirt logs

Checking the command line arguments of the qemu process we can see vmx=on and pdpe1gb=on too

[heat-admin@compute-1 ~]$ ps -ef | grep qemu
root         665       1  0 May16 ?        00:00:00 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open,guest-file-close,guest-file-read,guest-file-write,guest-file-seek,guest-file-flush,guest-exec,guest-exec-status -F/etc/qemu-ga/fsfreeze-hook
qemu      299412  293722  4 11:24 ?        00:00:16 /usr/libexec/qemu-kvm -name guest=instance-00000001,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-instance-00000001/master-key.aes -machine pc-i440fx-rhel7.5.0,accel=kvm,usb=off,dump-guest-core=off -cpu Haswell-noTSX,pdpe1gb=on,vmx=on -m 512 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 27fa6edb-67c3-4ec8-8558-1e0ece59e9e9 -smbios type=1,manufacturer=Red Hat,product=OpenStack Compute,version=17.0.3-0.20180420001139.el7ost,serial=a81fb8af-efb7-466e-a161-0dd3582aa3f7,uuid=27fa6edb-67c3-4ec8-8558-1e0ece59e9e9,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-instance-00000001/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/instances/27fa6edb-67c3-4ec8-8558-1e0ece59e9e9/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=36,id=hostnet0,vhost=on,vhostfd=38 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:41:67:49,bus=pci.0,addr=0x3 -add-fd set=2,fd=40 -chardev pty,id=charserial0,logfile=/dev/fdset/2,logappend=on -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 172.17.1.15:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
heat-ad+  300357  299395  0 11:29 pts/0    00:00:00 grep --color=auto qemu

Created attachment 1437984 [details]
instance logs

(In reply to Joe H. Rahme from comment #6)
> 
> 1. On the compute node, add the following to the [libvirt] section of
>    nova.conf:
> 
>     [libvirt]
>     cpu_mode = custom
>     cpu_model = Haswell-noTSX
>     cpu_model_extra_flags = vmx, pdpe1gb
>     virt_type = kvm
> 
> 2. Start an instance on this compute node.
> 
> 3. Dump the instance XML, we can see the vmx and pdpe1gb policies in
>    the cpu definition.
> 
> 
>   $ virsh dumpxml instance-00000001
>   ...
>   <cpu mode='custom' match='exact' check='full'>
>     <model fallback='forbid'>Haswell-noTSX</model>
>     <topology sockets='1' cores='1' threads='1'/>
>     <feature policy='disable' name='pdpe1gb'/>
>     <feature policy='disable' name='vmx'/>

As DanPB also said on IRC, what you are seeing above seems to be an artifact of using nested virt in your environment.


From your Nova Compute log, Nova is actually doing the Right Thing and ensures the 'policy' attribute is "require":

  <cpu mode='custom' match='exact' check='partial'>
    <model fallback='allow'>Haswell-noTSX</model>
    <topology sockets='1' cores='1' threads='1'/>
    [...]
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='vmx'/>
  </cpu>

And your QEMU command-line from comment#9 shows, most importantly that the CPU flags are added:

    "-cpu Haswell-noTSX,pdpe1gb=on,vmx=on"

[...]

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086