Bug 1066564 - file volumes with control codes in their names generate invalid XML
Summary: file volumes with control codes in their names generate invalid XML
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt
Version: 21
Hardware: x86_64
OS: Linux
Target Milestone: ---
Assignee: Ján Tomko
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2014-02-18 16:13 UTC by Jens Neu
Modified: 2016-04-26 21:55 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2015-06-05 23:01:14 UTC
Type: Bug

Attachments (Terms of Use)
virt-manager --debug whilst repoducing the bug (74.36 KB, text/x-log)
2014-02-18 16:42 UTC, Jens Neu
no flags Details
virt-manager --debug while trying to edit storage volume on existing vm (24.67 KB, text/x-log)
2014-02-18 17:13 UTC, Jens Neu
no flags Details
debug patch (606 bytes, text/plain)
2014-02-18 17:45 UTC, Cole Robinson
no flags Details
debug run with patch (900.32 KB, text/plain)
2014-02-20 09:13 UTC, Jens Neu
no flags Details

System ID Private Priority Status Summary Last Updated
Debian BTS 659770 0 None None None Never

Description Jens Neu 2014-02-18 16:13:39 UTC
Description of problem:
Can not create virtual machine

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Create a new virtual machine
2. Network Boot (PXE)
3. Forward
4. select OS Type (Linux)
5. Version ( e.g. Redhat 6)
6. Forward
7. leave defaults, Forward
8. Select managed or existing storage, Browse (select image)
9. Forward

Actual results:
Uncaught error validating install parameters: xmlParseDoc() failed

Uncaught error validating install parameters: xmlParseDoc() failed

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/create.py", line 1521, in validate
    return self.validate_storage_page()
  File "/usr/share/virt-manager/virtManager/create.py", line 1809, in validate_storage_page
    names = disk.is_conflict_disk(self.guest.conn)
  File "/usr/share/virt-manager/virtinst/devicedisk.py", line 803, in is_conflict_disk
  File "/usr/share/virt-manager/virtinst/devicedisk.py", line 349, in path_in_use_by
    for vol in conn.fetch_all_vols():
  File "/usr/share/virt-manager/virtinst/connection.py", line 231, in fetch_all_vols
    return self.cb_fetch_all_vols()  # pylint: disable=E1102
  File "/usr/share/virt-manager/virtManager/connection.py", line 146, in <lambda>
    for obj in pool.get_volumes(refresh=False).values()])
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 132, in get_xmlobj
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 197, in _reparse_xml
    self._xmlobj = self._build_xmlobj(self._get_raw_xml())
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 200, in _build_xmlobj
    return self._parseclass(self.conn.get_backend(), parsexml=xml)
  File "/usr/share/virt-manager/virtinst/storage.py", line 479, in __init__
    _StorageObject.__init__(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtinst/xmlbuilder.py", line 772, in __init__
    parent_xpath, relative_object_xpath)
  File "/usr/share/virt-manager/virtinst/xmlbuilder.py", line 677, in __init__
    self._parse(parsexml, parsexmlnode)
  File "/usr/share/virt-manager/virtinst/xmlbuilder.py", line 688, in _parse
    doc = libxml2.parseDoc(xml)
  File "/usr/lib64/python2.7/site-packages/libxml2.py", line 1325, in parseDoc
    if ret is None:raise parserError('xmlParseDoc() failed')
parserError: xmlParseDoc() failed

Expected results:
creates maschine.

Additional info:

Comment 1 Jens Neu 2014-02-18 16:42:44 UTC
Created attachment 864644 [details]
virt-manager --debug whilst repoducing the bug

Comment 2 Cole Robinson 2014-02-18 16:48:00 UTC
Can you try virt-manager 1.0.0 in updates testing? sudo yum  --enablerepo=updates-testing update virt-manager

If that still fails, please provide

sudo virsh pool-list --all
sudo virsh vol-list $POOL  (for every pool)
sudo virsh pool-dumpxml $POOL
sudo virsh vol-dumpxml --pool $POOL $VOLNAME

Comment 3 Jens Neu 2014-02-18 17:08:12 UTC
# rpm -qa virt-manager

-> behaviour unchanged.

[18:04:14 root@eluveitie work]# virsh pool-list --all
Name                 State      Autostart 
default              active     yes       
kvm                  active     yes       
kvm2                 active     yes    

-> behaviour the same on all pools

[18:04:14 root@eluveitie work]# virsh vol-list kvm
Name                 Path                                    
.                 /media/kvm/.                         
export               /media/kvm/export                       
fedora18-jens.img    /media/kvm/fedora18-jens.img            
freenas-data0.img    /media/kvm/freenas-data0.img            
freenas-data1.img    /media/kvm/freenas-data1.img            
freenas-data2.img    /media/kvm/freenas-data2.img            
ipfire.img           /media/kvm/ipfire.img                   
iso                  /media/kvm/iso                          
jens                 /media/kvm/jens                         
karelia-root.img     /media/kvm/karelia-root.img             
kvm                  /media/kvm/kvm                          
logstash-dev.img     /media/kvm/logstash-dev.img             
logstash.img         /media/kvm/logstash.img                 
lost+found           /media/kvm/lost+found                   
repo-src             /media/kvm/repo-src                     
sechs-client.img     /media/kvm/sechs-client.img             
sechs-router.img     /media/kvm/sechs-router.img             
win-jens             /media/kvm/win-jens                     

[18:05:07 root@eluveitie work]# virsh vol-list default
Name                 Path                                    
foreman-unattended.img /var/lib/libvirt/images/foreman-unattended.img
freenas.img          /var/lib/libvirt/images/freenas.img     
logstash-dev.img     /var/lib/libvirt/images/logstash-dev.img

[18:05:13 root@eluveitie work]# virsh vol-list kvm
Name                 Path                                    
.                 /media/kvm/.                         
export               /media/kvm/export                       
fedora18-jens.img    /media/kvm/fedora18-jens.img            
freenas-data0.img    /media/kvm/freenas-data0.img            
freenas-data1.img    /media/kvm/freenas-data1.img            
freenas-data2.img    /media/kvm/freenas-data2.img            
ipfire.img           /media/kvm/ipfire.img                   
iso                  /media/kvm/iso                          
jens                 /media/kvm/jens                         
karelia-root.img     /media/kvm/karelia-root.img             
kvm                  /media/kvm/kvm                          
logstash-dev.img     /media/kvm/logstash-dev.img             
logstash.img         /media/kvm/logstash.img                 
lost+found           /media/kvm/lost+found                   
repo-src             /media/kvm/repo-src                     
sechs-client.img     /media/kvm/sechs-client.img             
sechs-router.img     /media/kvm/sechs-router.img             
win-jens             /media/kvm/win-jens                     

[18:05:15 root@eluveitie work]# virsh vol-list kvm2
Name                 Path                                    
[18:05:19 root@eluveitie work]# virsh pool-dumpxml default
<pool type='dir'>
  <capacity unit='bytes'>52710469632</capacity>
  <allocation unit='bytes'>29199175680</allocation>
  <available unit='bytes'>23511293952</available>

[18:06:16 root@eluveitie work]# virsh pool-dumpxml kvm
<pool type='dir'>
  <capacity unit='bytes'>316934193152</capacity>
  <allocation unit='bytes'>156490461184</allocation>
  <available unit='bytes'>160443731968</available>

[18:06:21 root@eluveitie work]# virsh pool-dumpxml kvm2
<pool type='dir'>
  <capacity unit='bytes'>667305000960</capacity>
  <allocation unit='bytes'>56128987136</allocation>
  <available unit='bytes'>611176013824</available>

-> on of the volume(s) in question, none of them works in neither pool

[18:06:57 root@eluveitie work]# virsh vol-dumpxml --pool kvm logstash-dev.img
  <capacity unit='bytes'>85899345920</capacity>
  <allocation unit='bytes'>13254656</allocation>
    <format type='qcow2'/>

Comment 4 Jens Neu 2014-02-18 17:13:06 UTC
Created attachment 864650 [details]
virt-manager --debug while trying to edit storage volume on existing vm

Just noticed, when I try to edit the disk on a existing VM I get similar weird stuff (see log).

Comment 5 Cole Robinson 2014-02-18 17:45:07 UTC
Created attachment 864656 [details]
debug patch

Hmm, strange. Seems like libvirt is generating some XML that we fail to parse, just have to figure out what the XML is.

Please do this:

git clone git://git.fedorahosted.org/virt-manager.git
cd virt-manager
patch -p1 < $ATTACHED_PATCH
./virt-manager --debug

Reproduce the issue, and post the debugging output here

Comment 6 Jens Neu 2014-02-19 07:54:27 UTC
the patched virt-manager with debug only gives this, then prompt comes back regardless what I do or click:

[08:50:08 root@eluveitie virt-manager]# ./virt-manager --debug
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (cli:187) Launched with command line: ./virt-manager --debug
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (virt-manager:150) virt-manager version: 1.0.0
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (virt-manager:151) virtManager import: <module 'virtManager' from '/media/work/virt-manager/virt-manager/virtManager/__init__.py'>
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (virt-manager:209) GTK version: 3.10.6
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] ERROR (importer:51) Could not find any typelib for AppIndicator3
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (engine:475) libguestfs inspection support: False
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (systray:163) Showing systray: False
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (engine:231) About to connect to uris ['qemu:///system']
[08:50:15 root@eluveitie virt-manager]#

Comment 7 Cole Robinson 2014-02-19 14:36:27 UTC
Usually means you have another instance of virt-manager already running, make sure there's no virt-manager output in ps axwwww

Comment 8 Jens Neu 2014-02-20 09:13:47 UTC
Created attachment 865396 [details]
debug run with patch

re- run with debug patch, creation of new virtual machine with try to choose existing storage

Comment 9 Cole Robinson 2014-02-25 17:41:38 UTC
Okay, the problem in your setup is that you have a file in /media/kvm that has a control code in its name:

  <capacity unit='bytes'>0</capacity>
  <allocation unit='bytes'>0</allocation>
    <format type='dir'/>

Delete that file and things should work. However, libvirt should never generate invalid XML, so there's a fix needed on our side.

Comment 10 Jens Neu 2014-02-28 09:32:38 UTC
thank you very much for your help, with this information I investigated a little further. I found the file with the control characters in, turned out it was a directory with a qcow2 disk in it.
After I deleted the directory, virt-manager works again.
I have the suspicion that the control files appeared there due to a filesystem  corruption (most likey) or hardware error.
What I can't figure out is how this ended up in the xml. Does libvirt maybe re-generate the xml(s) upon restart and therefore reads the garbage from the filesystem? I recall the machine with the corrupted qcow2 image path working flawlessly, but I haven't bootet it in months.

Comment 11 Eric Blake 2014-02-28 13:03:08 UTC
Looks like this upstream patch, first released in 1.2.0, needs to be backported to F20:

commit 6cc4d6a3fe82653c607c4f159901790298e80e1f
Author: Eric Blake <eblake@redhat.com>
Date:   Wed Nov 20 17:04:05 2013 -0700

    storage: use valid XML for awkward volume names
    $ touch /var/lib/libvirt/images/'a<b>c'
    $ virsh pool-refresh default
    $ virsh vol-dumpxml 'a<b>c' default | head -n2
    Oops.  That's not valid XML.  And when we fix the XML
    generation, it fails RelaxNG validation.
    I'm also tired of seeing <key>(null)</key> in the example
    output for volume xml; while we used NULLSTR() to avoid
    a NULL deref rather than relying on glibc's printf
    extension behavior, it's even better if we avoid the issue
    in the first place.  But this requires being careful that
    we don't invalidate any storage backends that were relying
    on key being unassigned during virStoragVolCreateXML[From].
    I would have split this into two patches (one for escaping,
    one for avoiding <key>(null)</key>), but since they both
    end up touching a lot of the same test files, I ended up
    merging it into one.
    Note that this patch allows pretty much any volume name
    that can appear in a directory (excluding . and .. because
    those are special), but does nothing to change the current
    (unenforced) RelaxNG claim that pool names will consist
    only of letters, numbers, _, -, and +.  Tightening the C
    code to match RelaxNG patterns and/or relaxing the grammar
    to match the C code for pool names is a task for another
    day (but remember, we DID recently tighten C code for
    domain names to exclude a leading '.').
    * src/conf/storage_conf.c (virStoragePoolSourceFormat)
    (virStoragePoolDefFormat, virStorageVolTargetDefFormat)
    (virStorageVolDefFormat): Escape user-controlled strings.
    (virStorageVolDefParseXML): Parse key, for use in unit tests.
    * src/storage/storage_driver.c (storageVolCreateXML)
    (storageVolCreateXMLFrom): Ensure parsed key doesn't confuse
    volume creation.
    * docs/schemas/basictypes.rng (volName): Relax definition.
    * tests/storagepoolxml2xmltest.c (mymain): Test it.
    * tests/storagevolxml2xmltest.c (mymain): Likewise.
    * tests/storagepoolxml2xmlin/pool-dir-naming.xml: New file.
    * tests/storagepoolxml2xmlout/pool-dir-naming.xml: Likewise.
    * tests/storagevolxml2xmlin/vol-file-naming.xml: Likewise.
    * tests/storagevolxml2xmlout/vol-file-naming.xml: Likewise.
    * tests/storagevolxml2xmlout/vol-*.xml: Fix fallout.
    Signed-off-by: Eric Blake <eblake@redhat.com>

Comment 12 Cole Robinson 2014-02-28 16:00:37 UTC
When i tested this issue it was with virt-preview libvirt which should have that patch.

virBufferEscapeString doesn't do anything with ascii control codes, some of which seem to be valid in filenames. So git is still affected.

Comment 13 Cole Robinson 2014-03-10 13:22:02 UTC
I filed https://bugzilla.redhat.com/show_bug.cgi?id=1074528 to track backporting the patch eric mentioned above, since we need it anyways

Comment 14 Ján Tomko 2015-03-30 11:08:08 UTC
I have sent a patch to strip control codes when generating XML:

Comment 16 Ján Tomko 2015-04-15 17:47:38 UTC
Fixed upstream by
commit aeb5262e4397528d582682471cb8075141189465
    Strip control codes in virBufferEscapeString
commit 60db2bc80fb5048b227c77c5138fe0e2c97e9c14
    Ignore storage volumes with control codes in their names
commit 557107500b22d4a5ba7d1b09f5f516512dfca67b
    Strip control characters from sysfs attributes
commit 2a530a3e50d9314950cff0a5790c81910b0750a9
    Add functions dealing with control characters in strings
git describe: v1.2.14-194-gaeb5262

Backported to v1.2.9-maint as of: v1.2.9.2-10-g7eb973e

Comment 17 Fedora Update System 2015-04-28 16:54:45 UTC
libvirt- has been submitted as an update for Fedora 21.

Comment 18 Fedora Update System 2015-04-29 13:05:52 UTC
Package libvirt-
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing libvirt-'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).

Note You need to log in before you can comment on or make changes to this bug.