RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 856247 - full RHEL 6.4 block-copy support
Summary: full RHEL 6.4 block-copy support
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.3
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: ---
Assignee: Eric Blake
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 772088 837787
Blocks: 860720 871055 888426
TreeView+ depends on / blocked
 
Reported: 2012-09-11 14:53 UTC by Federico Simoncelli
Modified: 2013-02-21 07:23 UTC (History)
13 users (show)

Fixed In Version: libvirt-0.10.2-5.el6
Doc Type: Bug Fix
Doc Text:
Cause: RHEL 6.3 picked up an early version of block copy, prior to upstream acceptance of the feature. In the time since then, several improvements were made to the upstream feature. Consequence: Older libvirt was unable to fully manage the block copy features present in this release of qemu. Fix: The block copy code was finally approved upstream in both qemu and libvirt, and then libvirt was improved to work around the differences in the early implementation so that it can now work with all of RHEL 6.3, RHEL 6.4, and upstream qemu implementations. Result: Block copy as a means of backing up the disk images of a running domain is more powerful, and has been tested to have no regressions over the support available in RHEL 6.3.
Clone Of:
: 871055 888426 (view as bug list)
Environment:
Last Closed: 2013-02-21 07:23:25 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2013:0276 0 normal SHIPPED_LIVE Moderate: libvirt security, bug fix, and enhancement update 2013-02-20 21:18:26 UTC

Description Federico Simoncelli 2012-09-11 14:53:19 UTC 
Description of problem:
At the end of a mirroring block job (with the VIR_DOMAIN_BLOCK_REBASE_REUSE_EXT flag) the destination image chain is not relabeled for selinux and it's not included in the appropriate cgroup.

Version-Release number of selected component (if applicable):
libvirt-0.9.10-21.el6_3.4

How reproducible:
100%

Steps to Reproduce:
1. create an image chain using block devices
2. start a VM using the just created image and issue a blockRebase (using VIR_DOMAIN_BLOCK_REBASE_COPY|VIR_DOMAIN_BLOCK_REBASE_REUSE_EXT|VIR_DOMAIN_BLOCK_REBASE_SHALLOW)
3. complete the operation with blockJobAbort (VIR_DOMAIN_BLOCK_JOB_ABORT_PIVOT)

Actual results:
The operation fails with:

2012-09-10 12:01:54.144+0000: 2522: error : qemuMonitorJSONCheckError:338 : internal error unable to execute QEMU command '__com.redhat_drive-reopen': Could not open '/rhev/data-center/f3064767-68eb-4cac-bf87-2050539fa4a2/3d9c48da-ec64-45bb-940e-b065a4e8c150/images/9f2f5133-b44a-4375-8ecb-1e48c3dee1ee/03afd2bd-428a-457a-bee6-3978e9fa2940'

Expected results:
The operation should succeed.
Comment 2 zhe peng 2012-09-14 05:17:48 UTC 
Hi Federico:

    Would you help tell me whick qemu-kvm build you used? I want to reproduce this issue on rhel6.3.z, i try qemu-kvm-rhev-0.12.1.2-2.295.el6_3.2 and qemu-kvm-0.12.1.2-2.295.el6_3.2, they all not support block jobs, thanks in advance.
Comment 3 Federico Simoncelli 2012-09-14 07:46:30 UTC 
(In reply to comment #2)
> Hi Federico:
> 
>     Would you help tell me whick qemu-kvm build you used? I want to
> reproduce this issue on rhel6.3.z, i try
> qemu-kvm-rhev-0.12.1.2-2.295.el6_3.2 and qemu-kvm-0.12.1.2-2.295.el6_3.2,
> they all not support block jobs, thanks in advance.

qemu-kvm-rhev-0.12.1.2-2.312.el6.x86_64
Comment 4 zhe peng 2012-09-14 10:20:56 UTC 
Hi Fedrico,

    Thanks you for your quickly response, but i still have a question about this issue, as i know, issue a blockRebase need use virsh command "blockcopy",but now,this cmd can't be used in libvirt,refer this bug: https://bugzilla.redhat.com/show_bug.cgi?id=837583, so i want to know how can reproduce this ,please give me some advice for it ,very thanks. If the feature add-in build again, i will use virsh command to try this again.
Comment 5 Federico Simoncelli 2012-09-14 11:26:03 UTC 
# vgcreate vgtest1 /dev/mapper/1IET_00010001
# lvcreate -n lvbaseimg1 -L 1G vgtest1
# lvcreate -n lvbaseimg2 -L 1G vgtest1
# lvcreate -n lvleafimg1 -L 1G vgtest1
# lvcreate -n lvleafimg2 -L 1G vgtest1
# qemu-img create -f qcow2 -F raw -b /dev/vgtest1/lvbaseimg1 /dev/vgtest1/lvleafimg1
# qemu-img create -f qcow2 -F raw -b /dev/vgtest1/lvbaseimg2 /dev/vgtest1/lvleafimg2
# cat > vm1.xml <<EOF
<domain type="kvm">
  <name>vm1</name>
  <memory>131072</memory>
  <devices>
    <disk device="disk" type="block">
      <source dev="/dev/vgtest1/lvleafimg1"/>
      <target bus="ide" dev="hda"/>
      <driver name="qemu" type="qcow2"/>
    </disk>
  </devices>
  <os>
    <type arch="x86_64" machine="pc">hvm</type>
  </os>
</domain>
EOF

# for i in /dev/vgtest1/lv*; do chgrp -v qemu $(readlink -f $i); done

# virsh blockcopy --shallow --reuse-external --wait vm1 /dev/vgtest1/lvleafimg1 /dev/vgtest1/lvleafimg2
# virsh blockjob --pivot vm1 /dev/vgtest1/lvleafimg1
error: internal error unable to execute QEMU command '__com.redhat_drive-reopen': Could not open '/dev/vgtest1/lvleafimg2'
Comment 7 zhe peng 2012-09-17 10:22:31 UTC 
Hi Fedrico,

     very thanks, i can reproduce this now by your steps.
build: 
  libvirt-0.9.10-21.el6_3.4.x86_64
  qemu-kvm-rhev-0.12.1.2-2.312.el6.x86_64
Comment 12 Eric Blake 2012-09-18 12:30:55 UTC 
> Sadly the relabeling happens at blockcopy time instead of blockjob-pivot
> time and if the lv gets deactivated before the pivoting (there's actually a
> flow in rhev where this happens) then the switch is still failing:

Yep, I realized that overnight as well - I have to label at two different points for this to work correctly - flat label at copy start, and recursive label at pivot time.  Let me work up a v2 patch.
Comment 13 Eric Blake 2012-09-20 17:29:00 UTC 
For 6.3.z:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-September/msg00561.html

For 6.4, either this series will be picked up in time for 0.10.2 (in which case the rebase will move it into POST):
https://www.redhat.com/archives/libvir-list/2012-September/msg01122.html
or I will have to supply that series as a RHEL-only patch.
Comment 18 Dave Allan 2012-09-25 14:37:17 UTC 
I talked to Eric Blake, and his changes for bug 856247 cover the libvirt side.
Comment 19 Dave Allan 2012-09-25 14:37:53 UTC 
(In reply to comment #18)
> I talked to Eric Blake, and his changes for bug 856247 cover the libvirt
> side.

Urgh, wrong BZ, sorry for the noise.
Comment 22 Eric Blake 2012-09-25 19:10:21 UTC 
6.3.z will be in POST with:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-September/msg00674.html
Comment 23 Eric Blake 2012-09-26 11:49:34 UTC 
6.4 in POST:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-September/msg00701.html
Comment 25 Eric Blake 2012-10-15 20:30:47 UTC 
The version in comment 23 needs a respin, moving back to assigned until I post the next version.
Comment 27 Eric Blake 2012-10-20 22:25:01 UTC 
In POST:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-October/msg01033.html
Comment 28 Eric Blake 2012-10-23 03:49:51 UTC 
v3 of POST:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-October/msg01115.html
Comment 36 Eric Blake 2012-12-18 16:31:36 UTC 
I got another report from Federico today about a failure message of 'requires an active copy job', so there probably is a real data race that needs to be solved before the RHEL 6.4 release.  I'm looking into it further today.
Comment 37 Eric Blake 2012-12-18 16:45:13 UTC 
I'll go ahead and clone a new bug tracking the 'requires an active copy job' issue, to avoid holding up validation of the rest of the block-copy feature.
Comment 38 zhe peng 2013-01-04 05:36:03 UTC 
verify with build:
libvirt-0.10.2-14.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.348.el6.x86_64

step:
# vgcreate vgtest1 /dev/mapper/1IET_00010001
# lvcreate -n lvbaseimg1 -L 1G vgtest1
# lvcreate -n lvbaseimg2 -L 1G vgtest1
# lvcreate -n lvleafimg1 -L 1G vgtest1
# lvcreate -n lvleafimg2 -L 1G vgtest1
# qemu-img create -f qcow2 -F raw -b /dev/vg_virt/lvbaseimg1 /dev/vg_virt/lvleafimg1
# qemu-img create -f qcow2 -F raw -b /dev/vg_virt/lvbaseimg2 /dev/vg_virt/lvleafimg2
# cat > vm1.xml <<EOF
<domain type="kvm">
  <name>vm1</name>
  <memory>131072</memory>
  <devices>
    <disk device="disk" type="block">
      <source dev="/dev/vg_virt/lvleafimg1"/>
      <target bus="ide" dev="hda"/>
      <driver name="qemu" type="qcow2"/>
    </disk>
  </devices>
  <os>
    <type arch="x86_64" machine="pc">hvm</type>
  </os>
</domain>
EOF

# for i in /dev/vg_virt/lv*; do chgrp -v qemu $(readlink -f $i); done

S1: 
#virsh create vm1.xml

# virsh blockcopy --shallow --reuse-external --wait vm1 /dev/vg_virt/lvleafimg1 /dev/vg_virt/lvleafimg2

Now in mirroring phase

#for i in /dev/vg_virt/lv*; do ls -Z $(readlink -f $i); done
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-0
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-1
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c256,c359 /dev/dm-2
brw-rw----. root root system_u:object_r:fixed_disk_device_t:s0 /dev/dm-3


make sure blockjob in phase two
#virsh blockjob --info vm1 /dev/vg_virt/lvleafimg1
Block Copy: [100 %]

# virsh blockjob --pivot vm1 /dev/vg_virt/lvleafimg1

no error display
# for i in /dev/vg_virt/lv*; do ls -Z $(readlink -f $i); done
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-0
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-1
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c538,c961 /dev/dm-2
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c538,c961 /dev/dm-3

#virsh dumpxml vm1
.....
<disk type='block' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source dev='/dev/vg_virt/lvleafimg2'/>
      <target dev='hda' bus='ide'/>
.....
worked as expect

S2:
#virsh create vm1.xml

# virsh blockcopy --shallow --reuse-external --wait vm1 /dev/vg_virt/lvleafimg1 /dev/vg_virt/lvleafimg2

Now in mirroring phase

#for i in /dev/vg_virt/lv*; do ls -Z $(readlink -f $i); done
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-0
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-1
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c256,c359 /dev/dm-2
brw-rw----. root root system_u:object_r:fixed_disk_device_t:s0 /dev/dm-3

# lvchange -an /dev/vg_virt/lvbaseimg2
# lvchange -ay /dev/vg_virt/lvbaseimg2

# for i in /dev/vg_virt/lv*; do ls -Z $(readlink -f $i); done
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-0
brw-rw----. root disk system_u:object_r:fixed_disk_device_t:s0 /dev/dm-1
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c256,c359 /dev/dm-2
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c256,c359 /dev/dm-3
# virsh blockjob --pivot vm1 /dev/vg_virt/lvleafimg1

no error display
# for i in /dev/vg_virt/lv*; do ls -Z $(readlink -f $i); done
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-0
brw-rw----. qemu qemu system_u:object_r:virt_content_t:s0 /dev/dm-1
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c256,c359 /dev/dm-2
brw-rw----. qemu qemu unconfined_u:object_r:svirt_image_t:s0:c256,c359 /dev/dm-3

#virsh dumpxml vm1
.....
<disk type='block' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source dev='/dev/vg_virt/lvleafimg2'/>
      <target dev='hda' bus='ide'/>
.....

verification passed.
Comment 39 errata-xmlrpc 2013-02-21 07:23:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html
Note You need to log in before you can comment on or make changes to this bug.