Bug 753914

Summary: Improve handling of devices that get resized after pvcreate, detect smaller size
Product: Red Hat Enterprise Linux 6 Reporter: Dave Wysochanski <dwysocha>
Component: lvm2Assignee: Peter Rajnoha <prajnoha>
lvm2 sub component: Devices, Filtering and Stacking (RHEL6) QA Contact: cluster-qe <cluster-qe>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: high CC: agk, cmarthal, dwysocha, heinzm, jbrassow, kkolakow, nperic, prajnoha, prockai, rbednar, thornber, zkabelac
Version: 6.1Keywords: Patch
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: lvm2-2.02.141-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-11 01:19:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 756082, 1172231, 1268411    
Attachments:
Description Flags
test results none

Description Dave Wysochanski 2011-11-14 20:48:18 UTC
Description of problem:
Today, LVM does not detect if a device gets resized after a pvcreate has been done.  This request is to catch the "resized smaller" scenario.  Specifically, detect when the following sequence of events occurs:
1. pvcreate is run on the device
2. device gets resized smaller than the size in #1
3. Some other LVM command is run

This scenario has been seen in customer environments and usually results in a failure at lvcreate time of "device-mapper: reload ioctl failed: Invalid argument".  Detecting this condition in code is not too hard.

Version-Release number of selected component (if applicable):
latest / upstream

How reproducible:
Every time.

Steps to Reproduce:
I'll add a nightly test to reproduce it, but basically.
1. pvcreate on a device, add it to a VG
2. size the device smaller
3. try lvcreate on the VG, needing the full original size.

Actual results:
lvcreate requiring real space will fail with cryptic "device-mapper: reload ioctl failed: Invalid argument"
 
Expected results:
lvcreate should fail with a more descriptive error message.
LVM should detect when any PV is now a smaller size than at pvcreate time.

Additional info:

The first step is to detect it and flag a warning or error.

I have started a patchset against upstream and will post to lvm-devel.

A followup patchset could be done to repair a VG/PV (maybe add in vgck/pvck ?) with a "pvresize --setphysicalsize" equivalent, provided no LVs are involved.
The manual pvresize is what customers have to do to recover from this.
It would be good to have this in a repair command so they don't have
to figure it out, but detection is the primary goal of this bz.

Comment 1 Dave Wysochanski 2011-11-14 21:16:50 UTC
Initial patchset here: https://www.redhat.com/archives/lvm-devel/2011-November/msg00074.html

Comment 4 Alasdair Kergon 2012-03-13 14:08:18 UTC
An updated proposal is needed, taking into account comments in the email thread.

Comment 12 Peter Rajnoha 2015-10-15 12:07:42 UTC
Looking at the original patchset and Alasdairs comments, we could even control the checks per PV with the infrastructure we have now - the PV header extensions and the flags there. As for caching the actual dev size, I don't see a problem here either. We could do this in this release window finally I suppose.

Comment 17 Peter Rajnoha 2016-01-22 13:31:22 UTC
Patches are upstream now:

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=d090d6574e455625ad4e2a1a6f3d93f5c663b097

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=1f5dfb7369600c169ecd2c78e0cd079ad1442548

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=c0912af3104cb72ea275d90b8b1d68a25a9ca48a

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=136fd8f2f642c82b132ded4b9a95b05e822e9117

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=bc8f8ac0facc610c3252eb9af592b9c63a935ce2

In summary:
  - device sizes are cached now for reuse to save resources a bit (there are filters which already get the device size so most of the time the size is cached at that moment)
  - lifetime of cached device sizes are limited to a point when all the VG locks are dropped (we already do dev_close_all at this moment too)
  - there's a warning message issued if device size is found to be less than PV size stored in metadata (when reading VGs and when adding PVs to a VG)
  - there's new metadata/check_pv_device_sizes lvm.conf option to enable or disable these checks and warnings (they're enabled by default)

For example:

# pvcreate --setphysicalvolumesize 200m /dev/sda
  WARNING: /dev/sda: Overriding real size. You could lose data.
  Physical volume "/dev/sda" successfully created

# pvs -o pv_name,dev_size,pv_size /dev/sda
  PV         DevSize PSize  
  /dev/sda   128.00m 200.00m

# vgcreate vg /dev/sda
  Device /dev/sda has size of 262144 sectors which is smaller than corresponding PV size of 409600 sectors. Was device resized?
  Volume group "vg" successfully created

# pvs/vgs/lvs
  Device /dev/sda has size of 262144 sectors which is smaller than corresponding PV size of 409600 sectors. Was device resized?
  One or more devices used as PVs in VG vg have changed sizes.
  ...

Comment 19 Roman Bednář 2016-02-11 13:54:57 UTC
Created attachment 1123149 [details]
test results

Comment 20 Roman Bednář 2016-02-11 13:55:43 UTC
Verified. Test results in attachment.

2.6.32-614.el6.x86_64

lvm2-2.02.141-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016
lvm2-libs-2.02.141-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016
lvm2-cluster-2.02.141-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016
udev-147-2.71.el6    BUILT: Wed Feb 10 14:07:17 CET 2016
device-mapper-1.02.115-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016
device-mapper-libs-1.02.115-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016
device-mapper-event-1.02.115-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016
device-mapper-event-libs-1.02.115-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016
device-mapper-persistent-data-0.6.2-0.1.rc1.el6    BUILT: Wed Feb 10 16:52:15 CET 2016
cmirror-2.02.141-2.el6    BUILT: Wed Feb 10 14:49:03 CET 2016

Comment 22 errata-xmlrpc 2016-05-11 01:19:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0964.html