Bug 1852324

Summary: Failed to create VM with video by setting image metadata
Product: Red Hat OpenStack Reporter: chhu
Component: openstack-novaAssignee: Stephen Finucane <stephenfin>
Status: CLOSED ERRATA QA Contact: OSP DFG:Compute <osp-dfg-compute>
Severity: medium Docs Contact:
Priority: medium    
Version: 16.1 (Train)CC: dasmith, eglynn, jhakimra, kchamart, lyarwood, sbauza, sgordon, stephenfin, vromanso
Target Milestone: z6Keywords: Regression, Triaged
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: x86_64   
OS: Linux   
Whiteboard: libvirt_OSP_INT
Fixed In Version: openstack-nova-20.4.1-1.20201114041748.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-26 13:49:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
nova-conductor.log none

Description chhu 2020-06-30 06:52:52 UTC
Description of problem:
Failed to create VM with video by setting image metadata

Version-Release number of selected component (if applicable):
openstack-nova-compute-20.2.1-0.20200528080027.1e95025.el8ost.noarch
libvirt-daemon-kvm-6.0.0-23.module+el8.2.1+6955+1e1fca42.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a qcow2 image and update the image metadata with hw_video_ram='8'
(overcloud) [stack@dell-per730-** ~]$ openstack image show r8
------------------------------------------------------------------
| properties       | direct_url='file:///var/lib/glance/images/181ea5e5-c345-4200-b205-8a6dd8d66d91', hw_video_ram='8', os_hash_algo='sha512', os_hash_value='527d0a141dc1968b39d5528cf10223bb7849c77d0bc3cc67ed8024789b4c61aec6feaa444cd3ad0a305d80fd4ed1ebdc8f164527dc3d1437d3c04321fd76a78d', os_hidden='False', stores='default_backend' |
------------------------------------------------------------------

2. Create a flavor m2 with metadata hw_video:ram_max_mb='16384'
(overcloud) [stack@dell-per730-** ~]$ openstack flavor show m2
-------------------------------------------------------------------
| properties                 | hw_video:ram_max_mb='16384'      |
-------------------------------------------------------------------

3. Try to start VM from the image without creating volume with flavor m2, hit error in nova-conductor.log :
---------------------------------------------------------------------
File "/usr/lib64/python3.6/site-packages/libvirt.py", line 4047, in defineXML\n    if ret is None:raise libvirtError(\'virDomainDefineXML() failed\', conn=self)\n', "libvirt.libvirtError: XML error: cannot parse video vram '8192.0'\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', '  File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2161, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', '  File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2537, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', "nova.exception.RescheduledException: Build of instance ff326add-9c76-4ad7-9dd0-a48a36f654c6 was re-scheduled: XML error: cannot parse video vram '8192.0'\n"]
---------------------------------------------------------------------

Error in libvirtd.log:
----------------------------------------------------------------------
2020-06-30 06:35:53.913+0000: 4516: error : virDomainVideoDefParseXML:15903 : XML error: cannot parse video vram '8192.0'
----------------------------------------------------------------------

3. Change the image metadata hw_video_ram='16', try to start VM from the image without creating volume with flavor m2, hit similar error
------------------------------------------------------------------
File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2537, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', "nova.exception.RescheduledException: Build of instance 5a57dba5-f6dc-4090-a56a-13b16d91c35d was re-scheduled: XML error: cannot parse video vram '16384.0'\n"
---------------------------------------------------------------------

Error in libvirtd.log:
------------------------------------------------------------------------
2020-06-30 06:38:08.493+0000: 4519: error : virDomainVideoDefParseXML:15903 : XML error: cannot parse video vram '16384.0'
------------------------------------------------------------------------

4. Remove the image metadata hw_video_ram, start VM from the image without creating volume with flavor m2 successfully, related xml is as below:
--------------------------------------------------------------------
    <video>
      <model type='cirrus' vram='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
---------------------------------------------------------------------

Actual results:
In step2,3, failed to create the VM

Expected results:
In step2,3, create the VM successfully

Additional info:
- nova-conductor.log

Comment 1 chhu 2020-06-30 06:55:28 UTC
Created attachment 1699246 [details]
nova-conductor.log

Comment 2 smooney 2020-07-03 09:59:27 UTC
i think this is a actully a libvirt regression but its trival to work around in nova.
when we convert the vram amount to KB https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5367
we are doing floating point math so the convertion to string https://github.com/openstack/nova/blob/43802e5170595ce20cee73517e1e4d7c81fef079/nova/virt/libvirt/config.py#L1965
reults in a .0 being added which libvirt is not happy with.

the format code has not been altered in 7 years since it was added and the convertion code dates form 2014-10-29
https://github.com/openstack/nova/commit/f84fe4b067b12ff0b56cc788057f1726df0feace
so this issue is not cause by a change in nova but a change in behavior in libvirt.

it may never have been valid to pass a floating point value but it was previously accpeted which is why we are now seeing this after so long.

Comment 3 Stephen Finucane 2020-09-21 15:19:50 UTC
(In reply to smooney from comment #2)
> i think this is a actully a libvirt regression but its trival to work around
> in nova.

I think this is actually a Python 3 issue. The division operator returns floats in Python 3.

Comment 16 errata-xmlrpc 2021-05-26 13:49:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.6 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2097