RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1275039 - internal error: Invalid or not yet handled value 'emptyBackingString' for VMX entry 'ide1:0.fileName' for device type 'cdrom-image'
Summary: internal error: Invalid or not yet handled value 'emptyBackingString' for VMX...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: Virtualization Bugs
URL:
Whiteboard: virt
Depends On:
Blocks: 1266088 1301892
TreeView+ depends on / blocked
 
Reported: 2015-10-25 09:01 UTC by Shahar Havivi
Modified: 2016-11-03 18:28 UTC (History)
20 users (show)

Fixed In Version: libvirt-1.3.2-1.el7
Doc Type: Bug Fix
Doc Text:
Previously, when using the libvirt library with the VMWare ESXi hypervisor, loading a guest virtual machine that contained empty CD-ROM images failed with an "internal error" message. With this update, libvirt is able to successfully parse ESXi configuration values for empty CD-ROM images, and the described failure no longer occurs.
Clone Of: 1266088
: 1301892 (view as bug list)
Environment:
Last Closed: 2016-11-03 18:28:32 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:2577 0 normal SHIPPED_LIVE Moderate: libvirt security, bug fix, and enhancement update 2016-11-03 12:07:06 UTC
oVirt gerrit 46634 0 None MERGED v2v: Skip vm if XMLDesc() fails 2020-04-15 14:15:19 UTC
oVirt gerrit 47363 0 None MERGED v2v: Skip vm if XMLDesc() fails 2020-04-15 14:15:19 UTC

Description Shahar Havivi 2015-10-25 09:01:49 UTC 
+++ This bug was initially created as a clone of Bug #1266088 +++

Description of problem:
oVirt fails to load the list of VMs when querying a vCenter installation.

Version-Release number of selected component (if applicable):
3.6 rc5

How reproducible:
Seems only when certain VMs are present on the vCenter host queried.

Steps to Reproduce:
1. Log into administration portal, System->Virtual Machines. 
2. Click Import. Select VMware for source  and complete details for your vCenter environment or select a valid external provider (previously set).
3. Click the "Load" button.

Actual results:
The "Import Virtual Machine(s)" popup window in webui hangs forever. "internal error: Invalid or not yet handled value 'emptyBackingString' for VMX entry 'ide1:0.fileName' for device type 'cdrom-image'" appears in vdsm.log
Expected results:
list of VM's available for migration is show.


Additional info:

--- Additional comment from  on 2015-09-24 08:51 EDT ---



--- Additional comment from Nir Soffer on 2015-09-24 10:20:51 EDT ---

This fails when vdsm try to get domain xml for a remote vm:

      File "/usr/share/vdsm/v2v.py", line 152, in get_external_vms
        root = ET.fromstring(vm.XMLDesc(0))
      File "/usr/lib64/python2.7/site-packages/libvirt.py", line 478, in XMLDesc
        if ret is None: raise libvirtError ('virDomainGetXMLDesc() failed', dom=self)
    libvirtError: internal error: Invalid or not yet handled value
    'emptyBackingString' for VMX entry 'ide1:0.fileName' for device type
    'cdrom-image'

Vdsm enumerate all the vms on the remote server; if it fail to get the
xml for one of them, the entire request is aborted, preventing import
of a vm which may be fine.

--- Additional comment from Nir Soffer on 2015-09-24 10:27:21 EDT ---

Eric, who can help with the underlying libvirt error?

--- Additional comment from Red Hat Bugzilla Rules Engine on 2015-10-19 07:03:02 EDT ---

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

--- Additional comment from Nir Soffer on 2015-10-19 10:28:09 EDT ---

This is an issue with libvirt, we cannot do anything with this expect
filing a libvirt bug about this and block on it.

--- Additional comment from Yaniv Dary on 2015-10-20 10:55:10 EDT ---

Can you clone this bug to libvirt?
Comment 4 tingting zheng 2015-12-22 10:24:53 UTC 
Tested with:
rhevm-3.6.1.1-0.1.el6.noarch
libvirt-1.2.17-13.el7.x86_64

I can not reproduce the bug on rhev 3.6,after connect esx 6.0,all guests can be loaded successfully without the above error.

According to the bug description,it's related to guest with iso attached,in my environment,there are several guests which has iso attached in cd.

Can you confirm whether you can use the command "virsh -c vpx://$vcenter/$datacenter/$esxi/?no_verify=1 list --all" to list all guests on your esx server?thanks.
Comment 5 Shahar Havivi 2015-12-22 13:22:20 UTC 
sure I can list the vms via virsh
Comment 6 tingting zheng 2015-12-23 09:04:21 UTC 
Refer to the error info,there is an item in $guest.vmx which libvirt can not recognize,I tried to test as below:
1.In esx 6.0,find a guest,modify it's vmx file as below:
From:
ide1:0.deviceType = "cdrom-image"
ide1:0.fileName =  "/vmfs/volumes/f0a07907-3759bde7/en_win_srv_2003_r2_enterprise_with_sp2_cd1_X13-05460.iso"                        ide1:0.present = "TRUE"

to:
ide1:0.deviceType = "cdrom-image"
ide1:0.fileName = "emptyBackingString"    ----> modify this value                                                     
ide1:0.present = "TRUE"

2.Save the above change,from vsphere client,I can see in "CD/DVD driver 1" shows iso File path as "[]",but I can not set the value as '[]' from vsphere client.

3.Use virsh to connect the esx server and dumpxml the xml of the above guest.
# virsh -c esx://$esx/?no_verify=1
Enter username for $esx [root]: 
Enter root's password for $esx: 

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 1     win2012R2-vcenter              running
 33    iscsi-test                     running
 43    VMware-vcenter-server-app      running
 -     app_vcenter600                 shut off
 -     Auto-esx6.0-win7-efi           shut off
 -     bug1260689-win2008R2-GPO-Macfee shut off
 -     esx6.0-rhel5.11-i386           shut off
 -     esx6.0-rhel5.11-x86_64         shut off
 -     esx6.0-rhel6.7-i386            shut off
 -     esx6.0-rhel6.7-x86_64          shut off
 -     esx6.0-rhel7.2-x86_64          shut off
 -     esx6.0-win2003-i386            shut off
 -     esx6.0-win2003-x86_64          shut off
 -     esx6.0-win2008-i386            shut off
 -     esx6.0-win2008-x86_64          shut off


virsh # dumpxml $guest
error: internal error: Invalid or not yet handled value 'emptyBackingString' for VMX entry 'ide1:0.fileName' for device type 'cdrom-image'

4.Tried on rhev 3.6 GUI,I can load guests from esx server as steps from descrption,but the above guest is not in the guest list.

My question is since users can not set the iso file as "[]" which related to 
ide1:0.fileName = "emptyBackingString" in vmx file from vsphere client,do you have updated the vmx file for some guest on your esx server.

You can use virsh dumpxml $guest to find the guest with special configuration.
Comment 7 Shahar Havivi 2015-12-23 09:20:24 UTC 
we didn't modify the guest.
I dont have that environment anymore and in ovirt we block guest that libvirt cannot parse its xml.
Libvirt should be return a legal xml for any VM it represent.
Comment 9 Michal Privoznik 2016-01-11 07:21:55 UTC 
Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2016-January/msg00120.html
Comment 10 Michal Privoznik 2016-01-20 16:11:43 UTC 
v2:

https://www.redhat.com/archives/libvir-list/2016-January/msg00875.html
Comment 11 Michal Privoznik 2016-01-25 07:42:54 UTC 
Moving to POST:

commit 35c3aab44d20daa2623fcf2191fc2e3afc12b9f0
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Jan 7 09:51:55 2016 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Mon Jan 25 08:34:23 2016 +0100

    vmx: Adapt to emptyBackingString for cdrom-image
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1266088
    
    We are missing this value for cdrom-image device. It seems like
    there's no added value to extend this to other types of disk
    devices [1].
    
    1: https://www.redhat.com/archives/libvir-list/2016-January/msg01038.html
    
    Signed-off-by: Michal Privoznik <mprivozn>
Comment 15 Shahar Havivi 2016-03-02 13:19:07 UTC 
on libvirt version: 
===================
libvirt-daemon-1.2.17-13.el7_2.3.x86_64

its faling the following if statement:
======================================

for vm in conn.listAllDomains():
    params['vmName'] = vm.name()
    if vm.state()[0] == libvirt.VIR_DOMAIN_SHUTOFF:
        params['status'] = "Down"
    else:
        params['status'] = "Up"

with the exception:
===================
Traceback (most recent call last):
  File "/usr/share/vdsm/v2v.py", line 156, in get_external_vms
    _add_vm_info(vm, params)
  File "/usr/share/vdsm/v2v.py", line 662, in _add_vm_info
    if vm.state()[0] == libvirt.VIR_DOMAIN_SHUTOFF:
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2661, in state
    if ret is None: raise libvirtError ('virDomainGetState() failed', dom=self)
libvirtError: Domain not found: Could not find domain with UUID '4216df1f-d02a-adc9-d0c8-e2c9e9ab2ef2'


when I comment the if with statement I get error on the following code:
======================================================================
xml = vm.XMLDesc(0)
Comment 16 tingting zheng 2016-03-03 03:06:32 UTC 
(In reply to Shahar Havivi from comment #15)
> on libvirt version: 
> ===================
> libvirt-daemon-1.2.17-13.el7_2.3.x86_64
> 
> its faling the following if statement:
> ======================================
> 
> for vm in conn.listAllDomains():
>     params['vmName'] = vm.name()
>     if vm.state()[0] == libvirt.VIR_DOMAIN_SHUTOFF:
>         params['status'] = "Down"
>     else:
>         params['status'] = "Up"
> 
> with the exception:
> ===================
> Traceback (most recent call last):
>   File "/usr/share/vdsm/v2v.py", line 156, in get_external_vms
>     _add_vm_info(vm, params)
>   File "/usr/share/vdsm/v2v.py", line 662, in _add_vm_info
>     if vm.state()[0] == libvirt.VIR_DOMAIN_SHUTOFF:
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2661, in state
>     if ret is None: raise libvirtError ('virDomainGetState() failed',
> dom=self)
> libvirtError: Domain not found: Could not find domain with UUID
> '4216df1f-d02a-adc9-d0c8-e2c9e9ab2ef2'
> 
> 
> when I comment the if with statement I get error on the following code:
> ======================================================================
> xml = vm.XMLDesc(0)


Is it the same issue with bug 1287681?
Comment 17 Michal Privoznik 2016-03-03 06:20:48 UTC 
(In reply to tingting zheng from comment #16)
> (In reply to Shahar Havivi from comment #15)
> > 
> > when I comment the if with statement I get error on the following code:
> > ======================================================================
> > xml = vm.XMLDesc(0)
> 
> 
> Is it the same issue with bug 1287681?

I'm afraid so. if you try another domain does it work?
Comment 18 Shahar Havivi 2016-03-03 06:39:58 UTC 
It fail for some of the domains and some works fine
Comment 19 tingting zheng 2016-03-03 06:49:24 UTC 
(In reply to Michal Privoznik from comment #17)
> (In reply to tingting zheng from comment #16)
> > (In reply to Shahar Havivi from comment #15)
> > > 
> > > when I comment the if with statement I get error on the following code:
> > > ======================================================================
> > > xml = vm.XMLDesc(0)
> > 
> > 
> > Is it the same issue with bug 1287681?
> 
> I'm afraid so. if you try another domain does it work?

check bug 1287681 comment 13, # virsh list --all works some times  and fails a few seconds later.
Comment 20 Shahar Havivi 2016-03-03 07:05:02 UTC 
We have no problem via virsh lists --all, its working good.
The problem is the python API as described in comment #15
Comment 21 Moran Goldboim 2016-03-03 12:59:57 UTC 
based on comment 20, would you like us to reopen this bug or open a new bug?
Comment 22 Shahar Havivi 2016-03-03 13:06:37 UTC 
I don't know if that is a new bug...
I do think its the same - Nisim said that he can share his environment if its needed.
Comment 24 Michal Privoznik 2016-03-04 15:31:56 UTC 
(In reply to Nisim Simsolo from comment #23)
> It looks like vSphere 5.5 bug:
> I've succeeded to create similar VM with empty datastore ISO file on TLV
> VMware environment (Version 6.0.0 Build 2594327), and when "loading" VMs
> using webadmin import dialog, VMware VMs are listed except the problematic
> VM (FAD).
> 
> bug: https://bugzilla.redhat.com/show_bug.cgi?id=1287681 is mentioning this
> issue. 
> 
> If required, I can open a bug to VMware related to this issue.

That would be great! I would have done it myself but as I said in the linked bug I have no idea where to report that.
Comment 25 keanli 2016-04-16 00:41:47 UTC 
I can reproduce it with:

libvirt-1.3.1-1.el7.x86_64

1,Modify vmx file on the esx server
From:
ide1:0.deviceType = "cdrom-image"
ide1:0.fileName = "/vmfs/volumes/f0a07907-3759bde7/en_win_srv_2003_r2_enterprise_with_sp2_cd1_X13-05460.iso" ide1:0.present = "TRUE"

to:
ide1:0.deviceType = "cdrom-image"
ide1:0.fileName = "emptyBackingString" ----> modify this value
ide1:0.present = "TRUE"

2, Use virsh dumpxml the xml of the above guest

    virsh -c vpx://root.145.47/data/10.66.144.40/?no_verify=1 dumpxml empty_cdrom
    Enter root's password for 10.66.145.47:
    error: internal error: Invalid or not yet handled value 'emptyBackingString' for VMX entry 'ide0:0.fileName' for device type 'cdrom-image'

3,Go to rhevm GUI , the above guest can not be loaded

Test it with

Libvirt-1.3.3-2.el7.x86_64
virt-v2v-1.32.3-1.el7.x86_64


1,After step 2:

    virsh -c vpx://root.145.47/data/10.66.144.40/?no_verify=1 dumpxml empty_cdrom
    Enter root's password for 10.66.145.47:
    ...
    <devices>
    <disk type='file' device='disk'>
    <source file='[datastore1] empty_cdrom/empty_cdrom.vmdk'/>
    <target dev='sda' bus='scsi'/>
    <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
    <target dev='hda' bus='ide'/>
    <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    ...
    2,Go to rhevm ,the above guest can be loaded successfully
    3,Using the latest version virt-v2v convert it to kvm,it's can be boot normally 
    As the above comments ,the bug has already been fixed .So move it to VERIFIED
Comment 27 errata-xmlrpc 2016-11-03 18:28:32 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html
Note You need to log in before you can comment on or make changes to this bug.