Bug 601067 - Using a self recursive snapshot backing store *takes out* libvirtd
Using a self recursive snapshot backing store *takes out* libvirtd
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.0
All Linux
low Severity high
: rc
: ---
Assigned To: Daniel Berrange
Virtualization Bugs
:
Depends On:
Blocks: Rhel6.0LibvirtTier2
  Show dependency treegraph
 
Reported: 2010-06-07 03:24 EDT by Justin Clift
Modified: 2010-11-11 09:48 EST (History)
8 users (show)

See Also:
Fixed In Version: libvirt-0_8_1-14_el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-11-11 09:48:33 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Justin Clift 2010-06-07 03:24:10 EDT
Description of problem:

On RHEL 6 beta, when using a qcow2 snapshot volume that references itself as a backing store, libvirtd goes into a continuous loop when the VM is started.  It's effectively hard-locked, and need to be kill -9'd.

For example:

  # qemu-img create -f qcow2 -o backing_file=/home/images/sdbimagesnap2.qcow2,backing_fmt=host_device /home/images/sdbimagesnap2.qcow2 20G
  Formatting '/home/images/sdbimagesnap2.qcow2', fmt=qcow2 size=21474836480 backing_file='/home/images/sdbimagesnap2.qcow2' backing_fmt='host_device' encryption=off cluster_size=0 
  #

With a VM that uses it like:

  <disk type='block' device='disk'>
    <driver name='qemu' type='qcow2'/>
    <source dev='/home/images/sdbimagesnap2.qcow2'/>
    <target dev='hda' bus='ide'/>
    <address type='drive' controller='0' bus='0' unit='0'/>
  </disk>

When the VM is started, libvirtd effectively hard locks.  Logs from the libvirtd daemon show:

  17:08:49.926: info : qemuSecurityDACSetOwnership:40 : Setting DAC context on '/home/images/sdbimagesnap2.qcow2' to '0:0'
  17:08:49.926: info : qemuSecurityDACSetOwnership:40 : Setting DAC context on '/home/images/sdbimagesnap2.qcow2' to '0:0'
  17:08:49.926: info : qemuSecurityDACSetOwnership:40 : Setting DAC context on '/home/images/sdbimagesnap2.qcow2' to '0:0'

At a logging rate of 10,000 - 40,000+ new log lines per second until it's kill -9'd.

It does this even on a server which has had SELinux disabled:

  # getenforce 0
  Disabled
  #

Version-Release number of selected component (if applicable):

  libvirt-0.7.6-2.el6.x86_64


How reproducible:

Every time.


Steps to Reproduce:
1. Manually create a self-referencing snapshot with qemu-img.
2. Define a virtual machine using this snapshot volume.
3. Start the virtual machine.  libvirtd will go into an endless loop now.
  
Actual results:

libvirtd goes into an endless loop.  The guest virtual machine never gets to start.


Expected results:

Failure to boot in the guest VM.


Additional info:
Comment 2 Daniel Berrange 2010-06-07 05:47:10 EDT
Opps, that's a very nasty bug, even if this is a crazy config :-) Clearly we need to figure out how to fix this. Should also check that qemu handles this scenario sensibly.
Comment 3 RHEL Product and Program Management 2010-06-07 13:03:35 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.
Comment 5 Dave Allan 2010-07-14 02:15:29 EDT
libvirt-0_8_1-14_el6 has been built in RHEL-6-candidate with the fix.

Dave
Comment 7 weizhang 2010-07-27 07:48:34 EDT
Verify steps:
1.create qcow2 snapshot volume 
#qemu-img create -f qcow2 -o backing_file=/home/images/sdbimagesnap2.qcow2,backing_fmt=host_device /home/images/sdbimagesnap2.qcow2 20G

Formatting '/home/images/sdbimagesnap2.qcow2', fmt=qcow2 size=21474836480
backing_file='/home/images/sdbimagesnap2.qcow2' backing_fmt='host_device'
encryption=off cluster_size=0 

2. define qcow2-img.xml
#cat qcow2-img.xml

<domain type='kvm'>
  <name>qcow2-img</name>
  <uuid>c77b3a27-1bfb-65bd-6212-c72127d63af3</uuid>
  <memory>2097152</memory>
  <currentMemory>2097152</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.0.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
        <disk type='block' device='disk'>
            <driver name='qemu' type='qcow2'/>
            <source dev='/var/lib/libvirt/images/sdbimagesnap2.qcow2'/>
            <target dev='hda' bus='ide'/>
            <address type='drive' controller='0' bus='0' unit='0'/>
        </disk> 
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:dd:50:97'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
  </devices>
</domain>

#virsh define qcow2-img.xml

3. start vm
#virsh start qcow2-img
error: Failed to start domain qcow2-img
error: internal error backing store for /var/lib/libvirt/images/sdbimagesnap2.qcow2 is self-referential

Boot guest VM failed. What about the error information on the second line? Is that right? Does it mean the bug is fixed? 

version:
libvirt-0.8.1-18.el6.x86_64
kernel 2.6.32-52.el6.x86_64
Comment 8 Dave Allan 2010-07-28 14:38:07 EDT
Yes, that error message is correct.  If libvirt is still responsive after that error message, the bug is fixed.
Comment 9 weizhang 2010-07-28 21:21:51 EDT
According to the comment 7 and comment 8, the bug is verified
Comment 10 xhu 2010-09-07 23:27:19 EDT
Verified this bug with RHEL6 RC build and it passed:
libvirt-0.8.1-27.el6.x86_64
qemu-kvm-0.12.1.2-2.113.el6.x86_64
kernel-2.6.32-71.el6.x86_64
Comment 11 releng-rhel@redhat.com 2010-11-11 09:48:33 EST
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.

Note You need to log in before you can comment on or make changes to this bug.