Bug 1509110
| Summary: | backingStore element disappear after restart libvirtd | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Junxiang Li <junli> |
| Component: | libvirt | Assignee: | Peter Krempa <pkrempa> |
| Status: | CLOSED ERRATA | QA Contact: | Meina Li <meili> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 7.5 | CC: | dyuan, lmen, mtessun, pkrempa, rbalakri, xuzhang, yafu |
| Target Milestone: | rc | Keywords: | Automation, Regression |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-3.9.0-5.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-04-10 10:59:09 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
So the very important but missing fact is that the termination backing store element (the empty one) vanishes:
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/base.qcow2'/>
<backingStore/>
<target dev='vdb' bus='virtio'/>
<alias name='virtio-disk1'/>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</disk>
This is true for single layer and for nested images.
Fixed upstream:
commit 19448a2561699807c075e071a45df292efd01f6b
Author: Peter Krempa <pkrempa>
Date: Wed Nov 8 15:42:51 2017 +0100
conf: Properly parse <backingStore/>
The terminator would not be parsed properly since the XPath selector was
looking for an populated element, and also the code did not bother
assigning the terminating virStorageSourcePtr to the backingStore
property of the parent.
Some tests would catch it if there wasn't bigger fallout from the change
to backing store termination in a693fdba0111. Fix them properly now.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1509110
Hi,Peter
I verified it on libvirt-3.9.0-2.el7.x86_64/qemu-kvm-rhev-2.10.0-6.el7.x86_64,the result seems not expected on raw source file. Could you help to check it again? Thanks.
Steps:
1. Prepare vm with the following xml and start vm:
…
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/lmn.qcow2'/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/demo.img'/>
<target dev='sda' bus='scsi'/>
<alias name='scsi0-0-0-1'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
...
# virsh start lmn
Domain lmn started
2. Check the backingStore element in dumpxml:
# virsh dumpxml lmn
...
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/lmn.qcow2'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/demo.img'/>
<target dev='sda' bus='scsi'/>
<alias name='scsi0-0-0-1'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
...
Hmm right. The difference is that now that we require an explicit terminator in the data structures to format the element in the XML. This configuration does not add the terminating data structure since raw images don't go through the detection code which would add it normally. Hi,Peter
About your response, I still have two questions confused:
(1) Is it different with rhel7.4 for raw image source file in design? I can test the case with raw source file in rhel7.4, and I think it should be the same result now. Or it is a another bug in this version.
(2) In this bug, if raw images don't go through the detection code, why modify the test xml which include raw source file simultaneously? For example:
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/dev/null'/>
<target dev='sdg' bus='scsi'/>
<readonly/>
<shareable/>
<wwn>0x5000c50015ea71aa</wwn>
<alias name='scsi0-0-0-6'/>
<address type='drive' controller='0' bus='0' target='0' unit='6'/>
</disk>
Can you help me to analyze if my understanding is wrong, thank you very much.
Verified successfully on qcow2 image source file:
Test env components:
libvirt-3.9.0-2.el7.x86_64
Test steps:
Scenario 1: start vm with file/block disk and check backingStore element.
1. Prepare vm with the following xml and start vm:
…
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/lmn.qcow2'/>
<target dev='sda' bus='ide'/>
<wwn>0x5000c50015ea71aa</wwn>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/test.img'/>
<target dev='sdb' bus='scsi'/>
<readonly/>
<shareable/>
<wwn>0x5000c50015ea71aa</wwn>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='qcow2'/>
<source dev='/dev/Host/test.img'/>
<target dev='vde' bus='virtio'/>
<alias name='virtio-disk4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</disk>
...
# virsh start lmn
Domain lmn started
2. Check the backingStore element in dumpxml:
# virsh dumpxml lmn
...
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/lmn.qcow2'/>
<backingStore/>
<target dev='sda' bus='ide'/>
<wwn>0x5000c50015ea71aa</wwn>
...
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/test.img'/>
<backingStore/>
<target dev='sdb' bus='scsi'/>
...
<disk type='block' device='disk'>
<driver name='qemu' type='qcow2'/>
<source dev='/dev/Host/test.img'/>
<backingStore/>
<target dev='vde' bus='virtio'/>
...
3. Restart libvirtd and check vm's dumpxml, it is same with step2.
Scenario 2: check backingStore element after blockcopy.
1. Directly use the vm started in scenario 1.
2. Do blockcopy and check vm's dumpxml.
# virsh blockcopy lmn vda /tmp/test1/lmn1.py --blockdev --transient-job
Block Copy started
# virsh blockcopy lmn vde --blockdev /dev/HostVG/vol1 --transient-job
Block Copy started
# virsh dumpxml lmn
...
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/lmn.qcow2'/>
<backingStore/>
<mirror type='file' file='/tmp/test1/lmn.py' format='qcow2' job='copy' ready='yes'>
<format type='qcow2'/>
<source file='/tmp/test1/lmn.py'/>
</mirror>
...
<disk type='block' device='disk'>
<driver name='qemu' type='qcow2'/>
<source dev='/dev/Host/test.img'/>
<backingStore/>
<mirror type='block' job='copy' ready='yes'>
<format type='qcow2'/>
<source dev='/dev/HostVG/vol1'/>
</mirror>
...
3. Restart libvirtd and check vm's dumpxml, it same with step2.
Scenario 3: check backingStore element after snapshot.
1. Directly use the vm started in scenario 1.
2. Do snapshot and check vm's dumpxml.
# virsh snapshot-create-as lmn vol1 --disk-only
Domain snapshot vol1 created
# virsh snapshot-create-as lmn vol2 --disk-only
Domain snapshot vol2 created
# virsh dumpxml lmn
…
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/lmn.vol2'/>
<backingStore type='file' index='1'>
<format type='qcow2'/>
<source file='/var/lib/libvirt/images/lmn.vol1'/>
<backingStore type='file' index='2'>
<format type='qcow2'/>
<source file='/var/lib/libvirt/images/lmn.qcow2'/>
<backingStore/>
</backingStore>
</backingStore>
<target dev='sda' bus='ide'/>
…
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/dev/Host/test.vol2'/>
<backingStore type='file' index='1'>
<format type='qcow2'/>
<source file='/dev/Host/test.vol1'/>
<backingStore type='block' index='2'>
<format type='qcow2'/>
<source dev='/dev/Host/test.img'/>
<backingStore/>
</backingStore>
</backingStore>
...
3. Restart libvirtd and check vm's dumpxml, it same with step2.
(In reply to Meina Li from comment #9) > Hi,Peter > About your response, I still have two questions confused: > (1) Is it different with rhel7.4 for raw image source file in design? I can > test the case with raw source file in rhel7.4, and I think it should be the > same result now. Or it is a another bug in this version. No, it was a bug in the refactoring I've done. I've posted patches that fix it: https://www.redhat.com/archives/libvir-list/2017-November/msg01001.html > (2) In this bug, if raw images don't go through the detection code, why > modify the test xml which include raw source file simultaneously? For That was a mistake. The code which formats <backingStore/> was relying on default value of some variables to format it. Thus by adding an explicit terminator element into the chain we stopped formatting it. It was missed during review. > example: > <disk type='file' device='disk'> > <driver name='qemu' type='raw' cache='none'/> > <source file='/dev/null'/> > <target dev='sdg' bus='scsi'/> > <readonly/> > <shareable/> > <wwn>0x5000c50015ea71aa</wwn> > <alias name='scsi0-0-0-6'/> > <address type='drive' controller='0' bus='0' target='0' unit='6'/> > </disk> > Can you help me to analyze if my understanding is wrong, thank you very much. I've fixed it upstream by:
commit adcc31bb89e47ef642cbcfcff372131db7cd8d8b
Author: Peter Krempa <pkrempa>
Date: Fri Nov 24 12:59:59 2017 +0100
qemu: domain: Fix backing store terminator for non-backing local files
Raw local files do not pass through the backing store detector and thus
the code did not allocate the required backing store terminator for
them. Previously the terminating element would be formatted into the XML
since the default values used for the metadata allowed that. This is a
regression since a693fdba0111ff which was not detected in the review.
This patch also reverts all the changes in the test files.
commit 24e47ee2b93312742f73a785b9ca788df4677749
Author: Peter Krempa <pkrempa>
Date: Fri Nov 24 12:04:14 2017 +0100
qemu: process: Move handling of non-backing files into qemuDomainDetermineDiskChain
Until now we would skip loading of the backing chain for files which
don't support backing chains only when starting up the VM. Move the
check from qemuProcessPrepareHostStorage with some adaptations so that's
always applied.
commit b19710b9b6039728620a8d557a322ef139529226
Author: Peter Krempa <pkrempa>
Date: Fri Nov 24 12:09:53 2017 +0100
qemu: domain: Refactor control flow in qemuDomainDetermineDiskChain
Split out clearing of the backing chain prior to other code since it
will be required later and optimize few layers of nested conditions and
loops.
Verified successfully on raw image source file
Test env components:
libvirt-3.9.0-6.el7.x86_64
Test steps:
Prepare a guest with the following xml:
... <disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/rhel7.qcow2'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw'/>
<source dev='/dev/HostVG/vol1'/>
<target dev='sda' bus='scsi'/>
<wwn>0x5000c50015ea71aa</wwn>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<source protocol='gluster' name='gv0/vol1'>
<host name='10.66.4.164' port='24007'/>
</source>
<target dev='sdb' bus='ide'/>
<wwn>0x5000c50015ea71aa</wwn>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>...
1. Start guest with file/block/network disk and check backingStore element.
# virsh dumpxml rhel7 | grep "backingStore" -a1
<source file='/var/lib/libvirt/images/rhel7.qcow2'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
--
<source dev='/dev/HostVG/vol1'/>
<backingStore/>
<target dev='sda' bus='scsi'/>
--
</source>
<backingStore/>
<target dev='sdb' bus='ide'/>
# systemctrl restart libvirtd
# virsh dumpxml rhel7 | grep "backingStore" -a1
---->The same as above.
2. Do blockcopy for guest and check backingStore element.
# virsh blockcopy rhel7 sda /var/lib/libvirt/images/test.img --wait --verbose --transient-job
Block Copy: [100 %]
Now in mirroring phase
# virsh blockcopy rhel7 sdb /var/lib/libvirt/images/test1.img --wait --verbose --transient-job
Block Copy: [100 %]
Now in mirroring phase
# virsh dumpxml rhel7 | grep "backingStore" -a1
<source file='/var/lib/libvirt/images/rhel7.qcow2'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
--
<source dev='/dev/HostVG/vol1'/>
<backingStore/>
<mirror type='file' file='/var/lib/libvirt/images/test.img' format='raw' job='copy' ready='yes'>
--
</source>
<backingStore/>
<mirror type='file' file='/var/lib/libvirt/images/test1.img' format='raw' job='copy' ready='yes'>
# systemctrl restart libvirtd
# virsh dumpxml rhel7 | grep "backingStore" -a1
---->The same as above.
3. Hotplug disk device and check backingStore element.
# cat test.xml
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/test.img'/>
<target dev='vdb' bus='virtio'/>
</disk>
# virsh attach-device rhel7 test.xml
Device attached successfully
# virsh dumpxml rhel7 | grep "backingStore" -a1
<source file='/var/lib/libvirt/images/rhel7.qcow2'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
--
<source file='/var/lib/libvirt/images/test.img'/>
<backingStore/>
<target dev='vdb' bus='virtio'/>
--
<source dev='/dev/HostVG/vol1'/>
<backingStore/>
<target dev='sda' bus='scsi'/>
--
</source>
<backingStore/>
<target dev='sdb' bus='ide'/>
# systemctrl restart libvirtd
# virsh dumpxml rhel7 | grep "backingStore" -a1
---->The same as above.
Move it to Verified.
Hi,Peter
For <backingStore> element, it can output correctly after guest start for network disk.
For rhel7.5, as above test for gluster, it can output after guest start; but for the other network disk type(iscsi and rbd), it can't output <backingStore> element.
Version: libvirt-3.9.0-7.el7.x86_64
Result after guest start:
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none' error_policy='stop' io='threads'/>
<source file='/var/lib/libvirt/images/lmn.qcow2'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<boot order='1'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
<source protocol='rbd' name='lmn/vol1'>
<host name='**IP**'/>
</source> ---------><backingStore> not show up
<target dev='vdb' bus='virtio'/>
<alias name='virtio-disk1'/>
<address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
<source protocol='gluster' name='gv0/test1.img'>
<host name='**IP**' port='24007'/>
</source>
<backingStore/>
<target dev='vdc' bus='virtio'/>
<alias name='virtio-disk2'/>
<address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
</disk>
So, this is a regression question, please help to check it again,thank you very much.
That certainly is not a regression. Libvirt wasn't ever able to traverse backing chains to iSCSI or RBD disks. We report the rest of the chain as unknown by not putting the </backingStore> terminator in the chain for the disk. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:0704 |
Description of problem: backingStore element disappear after restart libvirtd Version-Release number of selected component (if applicable): Both on x86_64 and ppc64le, and the example is on ppc64le # rpm -q kernel qemu-kvm-rhev libvirt kernel-3.10.0-734.el7.ppc64le qemu-kvm-rhev-2.10.0-3.el7.ppc64le libvirt-3.9.0-1.el7.ppc64le How reproducible: 100% Steps to Reproduce: 1.prepare a domain xml: <domain type='kvm'> <name>avocado-vt-vm1</name> <uuid>d3370c29-64e5-4104-8131-3342ea422374</uuid> <memory unit='KiB'>1048576</memory> <currentMemory unit='KiB'>1048576</currentMemory> <vcpu placement='static'>2</vcpu> <os> <type arch='ppc64le' machine='pseries-rhel7.5.0'>hvm</type> <boot dev='hd'/> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/avocado/data/avocado-vt/images/jeos-25-64.qcow2'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='pci' index='0' model='pci-root'> <model name='spapr-pci-host-bridge'/> <target index='0'/> </controller> <controller type='usb' index='0' model='qemu-xhci'> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:e5:30:c9'/> <source bridge='virbr0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> <address type='spapr-vio' reg='0x30000000'/> </serial> <console type='pty'> <target type='serial' port='0'/> <address type='spapr-vio' reg='0x30000000'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='keyboard' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='usb'> <address type='usb' bus='0' port='2'/> </input> <graphics type='vnc' port='-1' autoport='yes'> <listen type='address'/> </graphics> <video> <model type='vga' vram='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> <panic model='pseries'/> </devices> </domain> 2. Define and start the guest 3. Dump the xml Run 'virsh dumpxml avocado-vt-vm1 > a1.xml' 4. Restart libvirtd Run 'systemctl restart libvirtd' 5. Dump the xml again Run 'virsh dumpxml avocado-vt-vm1 > a2.xml' Actual results: The backingStore element disappear after restart libvirtd Expected results: The backingStore element still exits. Additional info: