Bug 1302168 - blockjob --pivot failed with virtlockd on after blockcopy for os images
blockjob --pivot failed with virtlockd on after blockcopy for os images
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt (Show other bugs)
7.2
Unspecified Linux
unspecified Severity medium
: rc
: ---
Assigned To: Peter Krempa
yisun
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-26 20:29 EST by Han Han
Modified: 2017-08-01 19:51 EDT (History)
5 users (show)

See Also:
Fixed In Version: libvirt-3.0.0-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-08-01 13:09:12 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
libvirtd.log (1.01 MB, text/plain)
2016-01-26 20:31 EST, Han Han
no flags Details

  None (edit)
Description Han Han 2016-01-26 20:29:21 EST
Description of problem:
After blockcopy, do blockjob --pivot failed on guest os image due to copy file locked when virtlockd is on.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.3.0-31.el7_2.7.x86_64
libvirt-1.2.17-13.el7_2.2.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Preparing virlockd environment
# cat /etc/libvirt/qemu-lockd.conf
auto_disk_leases = 1
require_lease_for_disks = 1
file_lockspace_dir = "/var/lib/libvirt/lockd/scsivolumes"

# cat /etc/libvirt/qemu.conf
lock_manager = "lockd"

# systemctl restart virtlockd && systemctl restart libvirtd

2. Do blockcopy&blockjob --pivot for a running guest.
# virsh blockcopy n1 vda /var/lib/libvirt/images/n1.cpy --wait --verbose
Block Copy: [100 %]
Now in mirroring phase
# virsh blockjob n1 vda --pivot
error: resource busy: Lockspace resource '873758f7fe507f7b66e4f6653fdf965b9c61ab99fa941840b22ecbc2940d7bf8' is locked

Check log and find the lockspace file belongs to n1.cpy:
2016-01-26 11:05:00.667+0000: 19216: debug : virDomainLockManagerAddImage:90 : Add disk /var/lib/libvirt/images/n1.cpy
2016-01-26 11:05:00.667+0000: 19216: debug : virLockManagerAddResource:332 : lock=0x7fa5c800ca10 type=0 name=/var/lib/libvirt/images/n1.cpy nparams=0 params=(nil) flags=0
2016-01-26 11:05:00.667+0000: 19216: debug : virLockManagerLockDaemonAddResource:569 : Using indirect lease 873758f7fe507f7b66e4f6653fdf965b9c61ab99fa941840b22ecbc2940d7bf8 for /var/lib/libvirt/images/n1.cpy
2016-01-26 11:05:00.667+0000: 19216: debug : virLockManagerAcquire:350 : lock=0x7fa5c800ca10 state='<null>' flags=0 action=0 fd=(nil)

Actual results:
As step2

Expected results:
blockjob --pivot success.

Additional info:
blockcopy&blockjob --pivot for guest image not installed os is ok.
Comment 1 Han Han 2016-01-26 20:31 EST
Created attachment 1118646 [details]
libvirtd.log
Comment 3 Peter Krempa 2017-01-10 13:13:25 EST
commit b4698538123d57808e1b904b3d06d1adaf7f3624
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Fri Dec 16 16:06:57 2016 +0100

    qemu: blockjob: Fix locking of block copy/active block commit
    
    For the blockjobs, where libvirt is able to track the state internally
    we can fix locking of images we can remove the appropriate locks.
    
    Also when doing a pivoting operation we should not acquire the lock on
    any of those images since both are actually locked already.
Comment 5 yisun 2017-03-02 04:30:03 EST
Verified on:
======
libvirt-3.0.0-2.el7.x86_64
qemu-kvm-rhev-2.8.0-4.el7.x86_64


PASSED


Steps:
======
1. Preparing virlockd environment
# cat /etc/libvirt/qemu-lockd.conf
auto_disk_leases = 1
require_lease_for_disks = 1
file_lockspace_dir = "/var/lib/libvirt/lockd/scsivolumes"

# cat /etc/libvirt/qemu.conf
lock_manager = "lockd"

# systemctl restart virtlockd && systemctl restart libvirtd

2. prepare a transient vm
## virsh start vm1
lDomain vm1 started

## virsh list
 Id    Name                           State
----------------------------------------------------
 11    vm1                            running


## virsh undefine vm1
Domain vm1 has been undefined

3. check the lock file
## ll /var/lib/libvirt/lockd/scsivolumes
total 0
-rw-------. 1 root root 0 Mar  2 16:45 4a871df074d9339f152adf33b7dd7b81c21d427e88009d3606aa78617a422c98

4. Do the blockcopy and check a new lock file generated for the copied file
 ## virsh blockcopy vm1 vda /var/lib/libvirt/images/vm1.cpy --wait --verbose
Block Copy: [100 %]
Now in mirroring phase

 ## virsh dumpxml vm1 | grep cpy -a10
  ...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/RHEL-7.3-latest.img'/>
      <backingStore/>
      <mirror type='file' file='/var/lib/libvirt/images/vm1.cpy' format='qcow2' job='copy' ready='yes'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vm1.cpy'/>
      </mirror>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
...

## ll /var/lib/libvirt/lockd/scsivolumes
total 0
-rw-------. 1 root root 0 Mar  2 16:56 2bc4f0846679db12a738f2019cb3014439047f4357cccdf106e602f2855718f9
-rw-------. 1 root root 0 Mar  2 16:53 4a871df074d9339f152adf33b7dd7b81c21d427e88009d3606aa78617a422c98
5. do the blockjob pivot
## virsh blockjob vm1 vda --pivot
6. check one lock file for the original vda gone
## ll /var/lib/libvirt/lockd/scsivolumes
total 0
-rw-------. 1 root root 0 Mar  2 16:56 2bc4f0846679db12a738f2019cb3014439047f4357cccdf106e602f2855718f9
## virsh dumpxml vm1 | grep cpy -a10
  ...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vm1.cpy'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
...

7. login the vm and check the os works well
## virsh console vm1
Connected to domain vm1
Escape character is ^]

Red Hat Enterprise Linux Server 7.3 Beta (Maipo)
Kernel 3.10.0-461.el7.x86_64 on an x86_64

yisun_vm1 login: root
Password:
Last login: Wed Mar  1 18:16:39 on ttyS0
[root@yisun_vm1 ~]# uname -r
3.10.0-461.el7.x86_64
Comment 6 errata-xmlrpc 2017-08-01 13:09:12 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Comment 7 errata-xmlrpc 2017-08-01 19:51:16 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Note You need to log in before you can comment on or make changes to this bug.