Bug 1066564

Created attachment 864644 [details]
virt-manager --debug whilst repoducing the bug

Can you try virt-manager 1.0.0 in updates testing? sudo yum  --enablerepo=updates-testing update virt-manager

If that still fails, please provide

sudo virsh pool-list --all
sudo virsh vol-list $POOL  (for every pool)
sudo virsh pool-dumpxml $POOL
sudo virsh vol-dumpxml --pool $POOL $VOLNAME

# rpm -qa virt-manager
virt-manager-1.0.0-1.fc20.noarch

-> behaviour unchanged.


[18:04:14 root@eluveitie work]# virsh pool-list --all
Name                 State      Autostart 
-----------------------------------------
default              active     yes       
kvm                  active     yes       
kvm2                 active     yes    

-> behaviour the same on all pools

[18:04:14 root@eluveitie work]# virsh vol-list kvm
Name                 Path                                    
-----------------------------------------
.                 /media/kvm/.                         
export               /media/kvm/export                       
fedora18-jens.img    /media/kvm/fedora18-jens.img            
freenas-data0.img    /media/kvm/freenas-data0.img            
freenas-data1.img    /media/kvm/freenas-data1.img            
freenas-data2.img    /media/kvm/freenas-data2.img            
ipfire.img           /media/kvm/ipfire.img                   
iso                  /media/kvm/iso                          
jens                 /media/kvm/jens                         
karelia-root.img     /media/kvm/karelia-root.img             
kvm                  /media/kvm/kvm                          
logstash-dev.img     /media/kvm/logstash-dev.img             
logstash.img         /media/kvm/logstash.img                 
lost+found           /media/kvm/lost+found                   
repo-src             /media/kvm/repo-src                     
sechs-client.img     /media/kvm/sechs-client.img             
sechs-router.img     /media/kvm/sechs-router.img             
win-jens             /media/kvm/win-jens                     

[18:05:07 root@eluveitie work]# virsh vol-list default
Name                 Path                                    
-----------------------------------------
foreman-unattended.img /var/lib/libvirt/images/foreman-unattended.img
freenas.img          /var/lib/libvirt/images/freenas.img     
logstash-dev.img     /var/lib/libvirt/images/logstash-dev.img

[18:05:13 root@eluveitie work]# virsh vol-list kvm
Name                 Path                                    
-----------------------------------------
.                 /media/kvm/.                         
export               /media/kvm/export                       
fedora18-jens.img    /media/kvm/fedora18-jens.img            
freenas-data0.img    /media/kvm/freenas-data0.img            
freenas-data1.img    /media/kvm/freenas-data1.img            
freenas-data2.img    /media/kvm/freenas-data2.img            
ipfire.img           /media/kvm/ipfire.img                   
iso                  /media/kvm/iso                          
jens                 /media/kvm/jens                         
karelia-root.img     /media/kvm/karelia-root.img             
kvm                  /media/kvm/kvm                          
logstash-dev.img     /media/kvm/logstash-dev.img             
logstash.img         /media/kvm/logstash.img                 
lost+found           /media/kvm/lost+found                   
repo-src             /media/kvm/repo-src                     
sechs-client.img     /media/kvm/sechs-client.img             
sechs-router.img     /media/kvm/sechs-router.img             
win-jens             /media/kvm/win-jens                     

[18:05:15 root@eluveitie work]# virsh vol-list kvm2
Name                 Path                                    
-----------------------------------------
[18:05:19 root@eluveitie work]# virsh pool-dumpxml default
<pool type='dir'>
  <name>default</name>
  <uuid>328f8385-3954-6948-bce7-fd7c37e71831</uuid>
  <capacity unit='bytes'>52710469632</capacity>
  <allocation unit='bytes'>29199175680</allocation>
  <available unit='bytes'>23511293952</available>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>

[18:06:16 root@eluveitie work]# virsh pool-dumpxml kvm
<pool type='dir'>
  <name>kvm</name>
  <uuid>eb258558-7edf-8f76-9d35-9e3d098c567c</uuid>
  <capacity unit='bytes'>316934193152</capacity>
  <allocation unit='bytes'>156490461184</allocation>
  <available unit='bytes'>160443731968</available>
  <source>
  </source>
  <target>
    <path>/media/kvm</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>

[18:06:21 root@eluveitie work]# virsh pool-dumpxml kvm2
<pool type='dir'>
  <name>kvm2</name>
  <uuid>c5e4519c-83dc-4415-b886-e245082ffe06</uuid>
  <capacity unit='bytes'>667305000960</capacity>
  <allocation unit='bytes'>56128987136</allocation>
  <available unit='bytes'>611176013824</available>
  <source>
  </source>
  <target>
    <path>/media/work/kvm</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>



-> on of the volume(s) in question, none of them works in neither pool

[18:06:57 root@eluveitie work]# virsh vol-dumpxml --pool kvm logstash-dev.img
<volume>
  <name>logstash-dev.img</name>
  <key>/media/kvm/logstash-dev.img</key>
  <source>
  </source>
  <capacity unit='bytes'>85899345920</capacity>
  <allocation unit='bytes'>13254656</allocation>
  <target>
    <path>/media/kvm/logstash-dev.img</path>
    <format type='qcow2'/>
    <permissions>
      <mode>0600</mode>
      <owner>107</owner>
      <group>107</group>
    </permissions>
    <timestamps>
      <atime>1392740969.004293229</atime>
      <mtime>1392740176.256680370</mtime>
      <ctime>1392740239.600162688</ctime>
    </timestamps>
  </target>
</volume>

Created attachment 864650 [details]
virt-manager --debug while trying to edit storage volume on existing vm

Just noticed, when I try to edit the disk on a existing VM I get similar weird stuff (see log).

Created attachment 864656 [details]
debug patch

Hmm, strange. Seems like libvirt is generating some XML that we fail to parse, just have to figure out what the XML is.

Please do this:

git clone git://git.fedorahosted.org/virt-manager.git
cd virt-manager
patch -p1 < $ATTACHED_PATCH
./virt-manager --debug

Reproduce the issue, and post the debugging output here

the patched virt-manager with debug only gives this, then prompt comes back regardless what I do or click:

[08:50:08 root@eluveitie virt-manager]# ./virt-manager --debug
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (cli:187) Launched with command line: ./virt-manager --debug
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (virt-manager:150) virt-manager version: 1.0.0
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (virt-manager:151) virtManager import: <module 'virtManager' from '/media/work/virt-manager/virt-manager/virtManager/__init__.py'>
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (virt-manager:209) GTK version: 3.10.6
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] ERROR (importer:51) Could not find any typelib for AppIndicator3
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (engine:475) libguestfs inspection support: False
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (systray:163) Showing systray: False
[Wed, 19 Feb 2014 08:50:15 virt-manager 17263] DEBUG (engine:231) About to connect to uris ['qemu:///system']
[08:50:15 root@eluveitie virt-manager]#

Usually means you have another instance of virt-manager already running, make sure there's no virt-manager output in ps axwwww

Created attachment 865396 [details]
debug run with patch

re- run with debug patch, creation of new virtual machine with try to choose existing storage

Okay, the problem in your setup is that you have a file in /media/kvm that has a control code in its name:

<volume>
  <name>.[B</name>
  <key>/media/kvm/.[B</key>
  <source>
  </source>
  <capacity unit='bytes'>0</capacity>
  <allocation unit='bytes'>0</allocation>
  <target>
    <path>/media/kvm/.[B</path>
    <format type='dir'/>
    <permissions>
      <mode>0700</mode>
      <owner>1000</owner>
      <group>1000</group>
    </permissions>
    <timestamps>
      <atime>1391679752.934444408</atime>
      <mtime>1353075987.292170225</mtime>
      <ctime>1353075987.292170225</ctime>
    </timestamps>
  </target>
</volume>

Delete that file and things should work. However, libvirt should never generate invalid XML, so there's a fix needed on our side.

thank you very much for your help, with this information I investigated a little further. I found the file with the control characters in, turned out it was a directory with a qcow2 disk in it.
After I deleted the directory, virt-manager works again.
I have the suspicion that the control files appeared there due to a filesystem  corruption (most likey) or hardware error.
What I can't figure out is how this ended up in the xml. Does libvirt maybe re-generate the xml(s) upon restart and therefore reads the garbage from the filesystem? I recall the machine with the corrupted qcow2 image path working flawlessly, but I haven't bootet it in months.

Looks like this upstream patch, first released in 1.2.0, needs to be backported to F20:

commit 6cc4d6a3fe82653c607c4f159901790298e80e1f
Author: Eric Blake <eblake>
Date:   Wed Nov 20 17:04:05 2013 -0700

    storage: use valid XML for awkward volume names
    
    $ touch /var/lib/libvirt/images/'a<b>c'
    $ virsh pool-refresh default
    $ virsh vol-dumpxml 'a<b>c' default | head -n2
    <volume>
      <name>a<b>c</name>
    
    Oops.  That's not valid XML.  And when we fix the XML
    generation, it fails RelaxNG validation.
    
    I'm also tired of seeing <key>(null)</key> in the example
    output for volume xml; while we used NULLSTR() to avoid
    a NULL deref rather than relying on glibc's printf
    extension behavior, it's even better if we avoid the issue
    in the first place.  But this requires being careful that
    we don't invalidate any storage backends that were relying
    on key being unassigned during virStoragVolCreateXML[From].
    
    I would have split this into two patches (one for escaping,
    one for avoiding <key>(null)</key>), but since they both
    end up touching a lot of the same test files, I ended up
    merging it into one.
    
    Note that this patch allows pretty much any volume name
    that can appear in a directory (excluding . and .. because
    those are special), but does nothing to change the current
    (unenforced) RelaxNG claim that pool names will consist
    only of letters, numbers, _, -, and +.  Tightening the C
    code to match RelaxNG patterns and/or relaxing the grammar
    to match the C code for pool names is a task for another
    day (but remember, we DID recently tighten C code for
    domain names to exclude a leading '.').
    
    * src/conf/storage_conf.c (virStoragePoolSourceFormat)
    (virStoragePoolDefFormat, virStorageVolTargetDefFormat)
    (virStorageVolDefFormat): Escape user-controlled strings.
    (virStorageVolDefParseXML): Parse key, for use in unit tests.
    * src/storage/storage_driver.c (storageVolCreateXML)
    (storageVolCreateXMLFrom): Ensure parsed key doesn't confuse
    volume creation.
    * docs/schemas/basictypes.rng (volName): Relax definition.
    * tests/storagepoolxml2xmltest.c (mymain): Test it.
    * tests/storagevolxml2xmltest.c (mymain): Likewise.
    * tests/storagepoolxml2xmlin/pool-dir-naming.xml: New file.
    * tests/storagepoolxml2xmlout/pool-dir-naming.xml: Likewise.
    * tests/storagevolxml2xmlin/vol-file-naming.xml: Likewise.
    * tests/storagevolxml2xmlout/vol-file-naming.xml: Likewise.
    * tests/storagevolxml2xmlout/vol-*.xml: Fix fallout.
    
    Signed-off-by: Eric Blake <eblake>

When i tested this issue it was with virt-preview libvirt which should have that patch.

virBufferEscapeString doesn't do anything with ascii control codes, some of which seem to be valid in filenames. So git is still affected.

I filed https://bugzilla.redhat.com/show_bug.cgi?id=1074528 to track backporting the patch eric mentioned above, since we need it anyways

I have sent a patch to strip control codes when generating XML:
https://www.redhat.com/archives/libvir-list/2015-March/msg01529.html

v2: https://www.redhat.com/archives/libvir-list/2015-April/msg00551.html

Fixed upstream by
commit aeb5262e4397528d582682471cb8075141189465
    Strip control codes in virBufferEscapeString
commit 60db2bc80fb5048b227c77c5138fe0e2c97e9c14
    Ignore storage volumes with control codes in their names
commit 557107500b22d4a5ba7d1b09f5f516512dfca67b
    Strip control characters from sysfs attributes
commit 2a530a3e50d9314950cff0a5790c81910b0750a9
    Add functions dealing with control characters in strings
git describe: v1.2.14-194-gaeb5262

Backported to v1.2.9-maint as of: v1.2.9.2-10-g7eb973e

libvirt-1.2.9.3-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/libvirt-1.2.9.3-1.fc21

Package libvirt-1.2.9.3-1.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing libvirt-1.2.9.3-1.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-7150/libvirt-1.2.9.3-1.fc21
then log in and leave karma (feedback).