Bug 1511106
| Summary: | RFE: provide a reason why vdo can't be removed | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Corey Marthaler <cmarthal> |
| Component: | vdo | Assignee: | Joseph Chapman <jochapma> |
| Status: | CLOSED ERRATA | QA Contact: | Jakub Krysl <jkrysl> |
| Severity: | low | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.5 | CC: | awalsh, bgurney, jkrysl, jochapma, limershe, raeburn |
| Target Milestone: | rc | Keywords: | FutureFeature |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | 6.1.1.114 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-10-30 09:38:49 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Corey Marthaler
2017-11-08 17:20:32 UTC
The remove is done via "dmsetup remove" so there isn't a general way to pass back a detailed error message. The VDO manager does already check to see whether anything is mounted on the VDO volume before removing it. Looking at adding a check for LVM on top of the device. Comment from corwin on the associated Permabit Jira ticket at 2017-11-14 15:33:00: It is not at all clear that this is possible. *** Bug 1527921 has been marked as a duplicate of this bug. *** Some more info here. vdo remove tries "dmsetup remove" few times and fails after because it cannot remove it:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
--snip--
sdc 8:32 0 5.5T 0 disk
└─md127 9:127 0 10.9T 0 raid0
└─vdo_test 253:2 0 10.9T 0 vdo
└─vg_vdo_test-lv_test 253:3 0 10.9T 0 lvm
sdd 8:48 0 5.5T 0 disk
└─md127 9:127 0 10.9T 0 raid0
└─vdo_test 253:2 0 10.9T 0 vdo
└─vg_vdo_test-lv_test 253:3 0 10.9T 0 lvm
# vdo remove --all --verbose
Removing VDO vdo_test
Stopping VDO vdo_test
dmsetup status vdo_test
mount
udevadm settle
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup remove vdo_test
dmsetup status vdo_test
vdo: ERROR - cannot stop VDO service vdo_test
This is what LVM does in this situation:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
--snip--
sdc 8:32 0 5.5T 0 disk
└─md127 9:127 0 10.9T 0 raid0
└─vdo_test 253:2 0 10.9T 0 vdo
└─vg_vdo_test-lv_test 253:3 0 10.9T 0 lvm
└─test-test 253:4 0 100G 0 lvm
sdd 8:48 0 5.5T 0 disk
└─md127 9:127 0 10.9T 0 raid0
└─vdo_test 253:2 0 10.9T 0 vdo
└─vg_vdo_test-lv_test 253:3 0 10.9T 0 lvm
└─test-test 253:4 0 100G 0 lvm
# vgremove vg_vdo_test
Do you really want to remove volume group "vg_vdo_test" containing 1 logical volumes? [y/n]: y
Logical volume vg_vdo_test/lv_test is used by another device.
# lvremove vg_vdo_test/lv_test
Logical volume vg_vdo_test/lv_test is used by another device.
Maybe we can reproduce similar check to what LVM does in VDO...
"lsblk" can display information about the device dependencies. What LVM is doing is checking /sys/dev/block/<major>:<minor>/holders/. If that directory is non-empty, it generates the "device in use" message. (It then checks for a mounted file system, but we already do that check.) I will implement the same check. Tested on:
RHEL-7.6-20180626.0
kernel-3.10.0-915.el7
kmod-vdo-6.1.1.99-1.el7
vdo-6.1.1.99-2.el7
Now removing VDO under active VG+LV results in 'in use':
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 2T 0 disk
└─vdo 253:3 0 2T 0 vdo
└─vg-lv 253:4 0 2T 0 lvm
# vdo remove --name vdo
Removing VDO vdo
Stopping VDO vdo
vdo: ERROR - cannot stop VDO volume vdo: in use
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 2T 0 disk
└─vg-lv 253:3 0 2T 0 lvm
└─vg2-lv2 253:4 0 2T 0 lvm
# lvremove vg/lv
Logical volume vg/lv is used by another device.
There is a difference when creating target from the device using targetcli:
# targetcli
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/backstores/block> create test /dev/mapper/vdo
Created block storage object test using /dev/mapper/vdo.
/backstores/block> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json
# vdo remove --all --verbose
Removing VDO vdo
Stopping VDO vdo
dmsetup status vdo
mount
udevadm settle
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup remove vdo
dmsetup status vdo
vdo: ERROR - cannot stop VDO service vdo
# targetcli
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/backstores/block> create test /dev/mapper/vg-lv
Created block storage object test using /dev/mapper/vg-lv.
/backstores/block> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json
[root@storageqe-74 ~]# vgremove vg -ff
Logical volume vg/lv in use.
It appears LVM is checking not only sys holders, but also 'open count' on dmsetup (as not all open devices show here):
# dmsetup info -c /dev/mapper/vg-lv
Name Maj Min Stat Open Targ Event UUID
vg-lv 253 3 L--w 1 1 0 LVM-7aGzcEZ288BrHgHW0eQduEZAsDe5s3Xup2kvbsf6eOyuljOGU6n2Ec8I1bHIcj8O
Please add this check too to be safe.
Here, for reference, is what LVM does:
If there's a sysfs directory set (I think this is configuration):
If /sys/dev/block/<major>:<minor>/holders. exists and is not empty:
-> error "Logical volume %s is used by another device."
If a file system is mounted on the device:
-> error "Logical volume %s contains a filesystem in use."
Repeat 25 times:
check dm open count (which I assume vdo can do with "dmsetup info")
if the open count > 0 and thiat was the last retry:
-> error "Logical volume %s in use."
sleep 0.2 sec
No, the second fix (after the ticket was reopened) did not make it in before 6.1.1.111. The second fix simply changes the error message if the remove fails due to the device being open. Tested with vdo-6.1.1.120-3.el7: Now the message provides reason "vdo: ERROR - cannot stop VDO service vdo: device in use" Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:3094 |