Bug 215977

Summary: a forced pvremove of pvs with pvmoves inprogress will hang
Product: Red Hat Enterprise Linux 4 Reporter: Corey Marthaler <cmarthal>
Component: lvm2Assignee: Alasdair Kergon <agk>
Status: CLOSED ERRATA QA Contact: Corey Marthaler <cmarthal>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.4CC: agk, dwysocha, jbrassow, mbroz, prockai
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: RHBA-2007-0287 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-05-08 00:13:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Corey Marthaler 2006-11-16 17:41:45 UTC
Description of problem:
[root@link-08 bin]# pvmove /dev/sda1
  /dev/sda1: Moved: 2.9%
  /dev/sda1: Moved: 5.7%
  /dev/sda1: Moved: 8.5%
  /dev/sda1: Moved: 11.5%
  /dev/sda1: Moved: 14.4%
  /dev/sda1: Moved: 17.2%
  /dev/sda1: Moved: 20.2%
  /dev/sda1: Moved: 23.2%
  /dev/sda1: Moved: 26.0%

^C

[root@link-08 bin]# lvscan
  ACTIVE            '/dev/mirror_sanity/corey0' [9.98 GB] inherit
  ACTIVE            '/dev/mirror_sanity/corey1' [9.98 GB] inherit
[root@link-08 bin]# lvremove -f mirror_sanity
  Can't remove locked LV corey0
  Logical volume "corey1" successfully removed
  Can't remove locked LV pvmove0
[root@link-08 bin]# pvscan
  PV /dev/sda1   VG mirror_sanity   lvm2 [135.66 GB / 125.68 GB free]
  PV /dev/sdb1   VG mirror_sanity   lvm2 [135.66 GB / 135.66 GB free]
  PV /dev/sdc1   VG mirror_sanity   lvm2 [135.66 GB / 125.68 GB free]
  PV /dev/sdd1   VG mirror_sanity   lvm2 [135.66 GB / 135.66 GB free]
  PV /dev/sde1   VG mirror_sanity   lvm2 [135.66 GB / 135.66 GB free]
  PV /dev/sdf1   VG mirror_sanity   lvm2 [135.66 GB / 135.66 GB free]
  PV /dev/sdg1   VG mirror_sanity   lvm2 [135.66 GB / 135.66 GB free]
  Total: 7 [949.59 GB] / in use: 7 [949.59 GB] / in no VG: 0 [0   ]
[root@link-08 bin]# pvremove -ff /dev/sd[abcdefg]1
Really WIPE LABELS from physical volume "/dev/sda1" of volume group
"mirror_sanity" [y/n]? y
  WARNING: Wiping physical volume label from /dev/sda1 of volume group
"mirror_sanity"
  Can't open /dev/sda1 exclusively.  Mounted filesystem?


  CTRL-c detected: giving up waiting for lock
  /var/lock/lvm/P_orphans: flock failed: Interrupted system call
  Can't get lock for orphan PVs

This forced pvremove should either do the remove since it was called with the
force flag or report an error about not being able to get the lock and then
exit. I would argue that it should just do the removal of the pv.

Version-Release number of selected component (if applicable):
lvm2-2.02.14-1

Comment 1 Alasdair Kergon 2006-11-16 17:47:59 UTC
Can you attach a full -vvvv from the hanging pvremove command?

And there are *no* other lvm processes running at the time (as you killed the
pvmove process)?


Comment 2 Corey Marthaler 2006-11-16 20:11:41 UTC
[root@link-08 ~]# pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sda1  vg   lvm2 a-   135.66G 125.68G
  /dev/sdb1  vg   lvm2 a-   135.66G 125.68G
  /dev/sdc1  vg   lvm2 a-   135.66G 135.66G
  /dev/sdd1  vg   lvm2 a-   135.66G 135.66G
  /dev/sde1  vg   lvm2 a-   135.66G 135.66G
  /dev/sdf1  vg   lvm2 a-   135.66G 135.66G
  /dev/sdg1  vg   lvm2 a-   135.66G 135.66G
[root@link-08 ~]# lvs -a -o +devices
  LV       VG   Attr   LSize Origin Snap%  Move Log Copy%  Devices
  segment0 vg   -wi-a- 9.98G                               /dev/sda1(0)
  segment1 vg   -wi-a- 9.98G                               /dev/sdb1(0)
[root@link-08 ~]# pvmove /dev/sda1
  /dev/sda1: Moved: 2.8%
  /dev/sda1: Moved: 5.8%
  /dev/sda1: Moved: 8.8%
[^C]
[root@link-08 ~]# ps -ef | grep pvmove
root      4084  3944  0 09:16 pts/0    00:00:00 grep pvmove

[root@link-08 ~]# pvremove -vvvv -ff /dev/sd[abcdefg]1
[...]
#metadata/pv_manip.c:241         /dev/sda1 0:      0   2555: pvmove0(0:0)
#metadata/pv_manip.c:241         /dev/sda1 1:   2555  32173: NULL(0:0)
#metadata/pv_manip.c:241         /dev/sdb1 0:      0   2555: segment1(0:0)
#metadata/pv_manip.c:241         /dev/sdb1 1:   2555  32173: NULL(0:0)
#metadata/pv_manip.c:241         /dev/sdc1 0:      0   2555: pvmove0(0:1)
#metadata/pv_manip.c:241         /dev/sdc1 1:   2555  32173: NULL(0:0)
#metadata/pv_manip.c:241         /dev/sdd1 0:      0  34728: NULL(0:0)
#metadata/pv_manip.c:241         /dev/sde1 0:      0  34728: NULL(0:0)
#metadata/pv_manip.c:241         /dev/sdf1 0:      0  34728: NULL(0:0)
#metadata/pv_manip.c:241         /dev/sdg1 0:      0  34728: NULL(0:0)
Really WIPE LABELS from physical volume "/dev/sda1" of volume group "vg" [y/n]? y
  WARNING: Wiping physical volume label from /dev/sda1 of volume group "vg"
#device/dev-io.c:393         /dev/sda1: open failed: Device or resource busy
#pvremove.c:96   Can't open /dev/sda1 exclusively.  Mounted filesystem?
#locking/file_locking.c:164       Locking /var/lock/lvm/P_orphans WB


#locking/file_locking.c:111   CTRL-c detected: giving up waiting for lock
#locking/file_locking.c:187   /var/lock/lvm/P_orphans: flock failed: Interrupted
system call
#pvremove.c:81   Can't get lock for orphan PVs
#locking/file_locking.c:164       Locking /var/lock/lvm/P_orphans WB

#locking/file_locking.c:111   CTRL-c detected: giving up waiting for lock
#locking/file_locking.c:187   /var/lock/lvm/P_orphans: flock failed: Interrupted
system call
#pvremove.c:81   Can't get lock for orphan PVs
#locking/file_locking.c:164       Locking /var/lock/lvm/P_orphans WB

                                                                               
                      [HANG]

It will hang unless I hit ^C 7 times (I assume because that's the number of pvs
that I'm attempting to remove).

Comment 3 Alasdair Kergon 2006-11-17 21:33:28 UTC
fix applied upstream for 2.02.15

Comment 4 RHEL Program Management 2006-11-17 21:47:14 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 5 Alasdair Kergon 2006-11-21 22:13:05 UTC
2.02.15-3

Comment 6 Corey Marthaler 2006-11-21 22:18:58 UTC
fix verified in lvm2-2.02.15-3

Comment 10 Red Hat Bugzilla 2007-05-08 00:13:15 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2007-0287.html