RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1878893 - [RFE] lvreduce: Check for XFS signature and disallow if present
Summary: [RFE] lvreduce: Check for XFS signature and disallow if present
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: lvm2
Version: 9.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: David Teigland
QA Contact: cluster-qe@redhat.com
Kristina Slaveykova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-14 19:44 UTC by Dave Wysochanski
Modified: 2023-05-14 13:24 UTC (History)
16 users (show)

Fixed In Version: lvm2-2.03.17-2.el9
Doc Type: Enhancement
Doc Text:
.Improved functionality of the `lvreduce` command With this enhancement, when the logical volume (LV) is active, the `lvreduce` command checks if reducing the LV size would damage any file system present on it. If a file system on the LV requires reduction, and the `lvreduce resizefs` option has not been enabled, then the LV will not be reduced. Additionally, new options are now available to control the handling of file systems while reducing an LV. These options provide users with greater flexibility and control when using the `lvreduce` command.
Clone Of:
Environment:
Last Closed: 2023-05-09 08:23:40 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker CLUSTERQE-6154 0 None None None 2022-11-11 20:36:56 UTC
Red Hat Product Errata RHBA-2023:2544 0 None None None 2023-05-09 08:23:57 UTC

Description Dave Wysochanski 2020-09-14 19:44:21 UTC
Description of problem:
If an XFS filesystem is detected on a logical volume, prevent (reject) lvreduce

Background
A total of 90 cases attached to the main “XFS shrink” kbase https://access.redhat.com/solutions/540013 were recently reviewed.  Most of these cases were RHEL7 but some were RHEL8.  All cases were opened between March 2018 and Aug 2020.

Of the 90 cases, a surprising number 22/90 (24%) cases showed an “improper lvreduce” was done by the customer, which was almost always discovered by a Red Hat engineer looking in the lvm archive and finding an lvreduce prior to case open.  These cases almost always started with either a system that would not boot (if root was the filesystem) and/or filesystem or IO errors, and/or unable to mount the filesystem.  In at least one case, permanent data loss occurred.

As far as severity at case closure, the cases were as follows, with 68% (15/22) either Urgent (1) or High (2) severity:
Urgent (1): 6
High (2): 9
Normal (3): 4
Low (4): 3

NOTE: There was at least one bug opened previously from a customer case.  In the bug was discussed having an additional warning for XFS using lvreduce (see https://bugzilla.redhat.com/show_bug.cgi?id=1385443), but this ended up closing as NOTABUG.  I’m not sure though at the time we had this much evidence of this many customers having this problem and opening SEV1/SEV2 cases, and I don’t see any discussion about the fact there is no warning when the LV is not active, and/or the fact that lvreduce should _NEVER_ be appropriate for an LV that has an XFS filesystem on it.  For these reasons, I am opening this new bug with this new evidence, to reconsider allowing lvreduce with XFS.


Version-Release number of selected component (if applicable):
lvm2-2.03.08-3.el8.x86_64 (probably needs upstream patch though)

How reproducible:
Every time

Steps to Reproduce:
There's a few different cases:
1. Inactive LV 
* No warning / check is done today so this silently can corrupt XFS filesystems.  If the storage is then repurposed elsehwere, it may be impossible to "undo" the lvreduce.
2. Active LV
* There's a warning but customers can still proceed.  There should be no reason to proceed if an XFS filesystem is detected, since there's no way to shrink XFS filesystems and this can only lead to corruption.  If a customer wants to destroy the LV / XFS filesystem, they can do so explicitly with lvremove.


Actual results:
Active LV: Prints warning but says "MAY DESTROY", and allows user to continue:
[root@rhel8u2-node1 ~]# lvreduce -L 500M /dev/mapper/testvg-testlv
  WARNING: Reducing active and open logical volume to 500.00 MiB.
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce testvg/testlv? [y/n]: 

Inactive LV: No warning printed
[root@rhel8u2-node1 ~]# lvchange -an /dev/mapper/testvg-testlv
[root@rhel8u2-node1 ~]# lvreduce -L 500M /dev/mapper/testvg-testlv
  Size of logical volume testvg/testlv changed from 1.49 GiB (382 extents) to 500.00 MiB (125 extents).
  Logical volume testvg/testlv successfully resized.
[root@rhel8u2-node1 ~]# 


Expected results:
Reject lvreduce on XFS filesystems, whether active or inactive, because it will definitely corrupt XFS / make it unusable

Additional info:
Other than device signatures, I'm not sure LVM has any any signature detection, for example, for LVs.  So I'm not sure how easy this is to implement.  There's also possible side-effects of requiring activation before lvreduce on non-XFS filesystems - maybe we need some sort of override to skip the active signature detection.

Note that I think in most instances, customers are just using linear LVs (think default installs, etc) and then they run out of space somewhere and try to reclaim / repurpose the space.  So some simple XFS signature detection would probably go a long way towards preventing XFS filesystem corruptions.

Comment 2 Eric Sandeen 2020-10-08 15:50:51 UTC
A few thoughts on this:

* libblkid makes format signature detection easy
* if the user requests -r on a device with a format it doesn't know how to shrink, I'd error out immediately
  - today it seems fsadm runs xfs_repair first, then fails to shrink, and exits
  - running repair is pointless and possibly time-consuming if you'll exit anyway
* consider flatly refusing to shrink a device without "-r" if a filesystem signature is present
  - wipefs -a could be used (and suggested) to remove filesystem signatures, to allow lvreduce to proceed

Comment 3 Zdenek Kabelac 2020-10-08 16:21:33 UTC
Some remarks:

lvreduce can be purely 'metadata' operation - so when  LV is not active - the whole 'reduction' for LV types like 'linear' happens in lvm2 metadata when used extents are returned to free set in VG.

So adding detection of filesystem - may introduce noticeable difference to a user - since we will need to 'active' such LV  - which may in case of being this a snaphost of some other actively running filesystem cause further troubles on the road.

So while detecting signature for already active LV may not be causing such noticeable side-effect - the problem with inactive LV would require more thinking -  it might actually force us to reject command and ask user to active LV before reducing it -
which can be find problematic for some users.

For active LVs - we already print warning about losing 'data' yet - this doesn't make users aware.

When user finds filesystem unusable after lvreduce - he can usually restore previous lvm2 metadata and get the volume size back
(unless he sets issue_discards=1 and uses ssd/nvme class of storage - in this case data are lost...)

One idea could be that maybe we can get some support for volume types as some LV metadata hints (aka  LV_REDUCE_DISALLOWED) - so we know LV has been formatted with XFS and solve this at lvm2 metadata level...
Hint would be dropped with i.e. wipefs as requested by comment 2.

Comment 4 David Teigland 2020-10-12 15:28:19 UTC
udev knows what fs is on the lv, even if the lv is inactive, so we can simply ask udev.  For active LVs there's also /etc/mtab.  These checks may not work perfectly for every case, but they should catch a majority of cases I expect.

Comment 5 Zdenek Kabelac 2020-10-12 16:50:21 UTC
For inactive LV udev can't knbow anything about the LV - since it's simply not physically present on a system.

We would need to update lvm2 metadata and add flag to our metadata when we know during i.e. pvscan
what type of filesystem is there.

Comment 6 Zdenek Kabelac 2020-10-12 16:54:56 UTC
Other idea could be -  we can introduce new  'lvchange' command   --reducable y|n

Then mkfs.xfs  would simply call lvm2 command for LV device - other some other logistic around that could be orchestrated.
(thought this looks like not too hard extension)

Comment 7 David Teigland 2020-10-12 17:19:53 UTC
Don't know what I was thinking about udev db.  lvchange --reducible sounds good.

Comment 8 Eric Sandeen 2021-01-06 18:33:22 UTC
Now I'm wondering if the customer problems are actually unique to XFS, or if they are simply more prevalent on XFS due to our choice of default filesystem in RHEL.

I'm curious about a couple of things:

1) If you run lvreduce without "-r" on an LV containing an ext4 filesystem, will it reduce the LV without shrinking the filesystem first, and corrupt it?
2) If you run lvreduce with "-r" on an LV containing an XFS filesystem, does it properly fail to reduce the LV and keep the filesystem intact?

In other words, are the corrupt-xfs-filesystem cases from "lvreduce" due to administrator actions that would have caused the same problems on ext4?

Comment 9 David Teigland 2021-01-06 18:49:23 UTC
Let's seriously consider changing the lvreduce default behavior in RHEL9.  The new default would activate the LV and check the fs prior to reducing the LV.  A new "--checkfs n" option would let users perform a metadata-only lvreduce without activating or checking the LV.

What are the possible problems with this change?

Comment 10 Zdenek Kabelac 2021-01-06 22:54:51 UTC
Unsure if I remember correctly, but ext4 filesystem checks the size of device - and if it doesn't match the size the filesystem thinks it should have, it stops  (possibly based on  remount-ro  logic) 

Then plain 'revert' of LV size avoids damages to the filesystem itself and restores back fully functional fs.

Comment 11 Zdenek Kabelac 2021-01-06 22:57:36 UTC
IMHO '--checkfs n' is mostly equivalent to make a switch '-r' option a default.

i.e. lvextend -r    (configurable then with  lvm.conf setting)

Comment 12 David Teigland 2021-01-06 23:27:39 UTC
yes I think that's about the same thing.  We could have:

lvm.conf lvreduce_resizefs = y|n

lvreduce --resizefs|--noresizefs on the command line would override the lvm.conf setting.

Possible combinations:

lvreduce_resizefs = y, lvreduce (no option): run resizefs

lvreduce_resizefs = n, lvreduce (no option): do not run resizefs

lvreduce_resizefs = y|n, lvreduce --resizefs: run resizefs

lvreduce_resizefs = y|n, lvreduce --noresizefs: do not run resizefs


In RHEL9 it would make sense to have default lvreduce_resizefs = y, but it's possible we could also use this default in RHEL8 with sufficient coordination and notice.

Comment 13 Zdenek Kabelac 2021-01-06 23:34:32 UTC
I'd not limit the usage only for 'reduce' path - it would work with all variants via 'lvextend/lvreduce/lvresize'.

So the lvm.conf option would be something like i.e. 'lvresize_uses_resizefs = 1/0'

and we would add '--noresizefs' option to match '--resizefs'

If the majority of users are using LVs with  filesystems - then it's likely matching the logic with favouring more used case for users.

so users no longer needs to specify  '-r'  with lvreduce/lvextend - the get it by default.

Comment 14 David Teigland 2021-01-07 00:00:36 UTC
If we want to address both, then I think we'd want separate settings, and probably different defaults, for each.  lvextend is safe and probably common without using resizefs, and changing the default to always run resizefs could be a real problem for existing users (e.g. RHV uses lvextend on LVs which don't have file systems.)

Comment 15 Zdenek Kabelac 2021-01-07 10:37:09 UTC
So there are 2 things to look at:

Existing behavior of 'fsadm' does work with a known filesystem or DM crypto or it reports an error - we can add a new logic -

- if it knows a filesystem/crypto - proceed with a standard resize fs.
- it it doesn't know it - just ignore it and continue only with lvm2 metadata resize operatoin.

Thus we could maybe add a new option for this  --resizeknown  (whatever name fits)

Then it seems more API cleaner to directly enable this new functionality by default - so users will get always extension/reduction of LV.  Such change only for 'lvreduce' doesn't free user from a lot of work, and then the previous --nocheckfs  (and crypto) can be maybe seen slight better.

But in a similar way we may introduce the 'REDUCIBLE' flag with an LV from comment 6.
So users who want to be able to 'reduce' their LVs  can switch it on.
(as by default all LVs would be non-reducible - based on an lvm.conf setting - so user could have LVs reducible if needed)

By default existing (old) metadata for new version of lvm2 would be handled as LVs that CANNOT be reduced unless lvchanged.

We could use a backward compatible flags=[] so if the 'older' lvm2 command would have seen 'REDUCIBLE' flag it would just drop it
and would still work with such metadata. Then again with new lvm2 user needs to set again 'lvchange --reducible y' to be able to reduce the LV.

It seems the 'REDUCIBLE' flag is possibly universally usable - as this gives admins better control whether they want to allow
particular LVs to be ever reduced in their size independently on what we think there is a filesystem or something we know how to resize as for lvm2 is actually hard to decide what could be reduced - we know only about extX - and everything else by default fails.

To handle single command line bypass, we may have 'lvreduce --ignorereducible'  or  let it proceed with '--force'...

Comment 16 David Teigland 2021-01-07 17:00:54 UTC
A reducible LV flag may be useful for other things, but I don't think it will help this bug request.  First, it's unlikely that mkfs.xfs will run an lvchange command to set it.  Second, it doesn't give us a clean and simple way to deal with all of the existing file systems.  Third, xfs will soon have a limited ability to shrink, so we want to be able to shrink xfs.  

My suggestion remains:

- leave lvextend as it is, where by default it just extends the LV, and with --resizefs it will try to resize the fs.

- change lvreduce to run resizefs by default, but configurable via lvm.conf lvreduce_lvresize, --resizefs|--noresizefs.

Comment 17 Zdenek Kabelac 2021-01-08 11:49:28 UTC
The difference from origin comment 6 is - by default we would be creating  'non reducible'  LVs
(so no change on external commands like mkfs is needed)

When the user wants to be able to reduce LV - there needs to be an 'active' action taken on his side
and it's then his full responsibility over what is put there:

# lvcreate --reducible y|n
# lvchange --reducible y|n

Clearly this action will affect also extX reducibility - but I think when we will properly handle warning messages 
- users can accept the fact they will need to actively mark LVs they want to have then reducible.
(several similar changes happened in past)

Also IMHO reducibility is very edge use-case so quite low number of users will hit this issue.

Once the user sets LV to be REDUCIBLE - there will be no further limitation on lvreduce usage.
For the old behavior its then easy to provide lvm.conf switch  (i.e.: limit_reducibility = 0|1)

Users are given a chance to stay protected from accidental reductions of LV - and we do not need to care what's on a device.
When user wants reducibility - it's then his choice - so a new feature like.


As for comment 16:

IMHO it's going to be a bit puzzling when lvresize command will work differently for different direction aka
why 'reduction' goes with filesystem automagically, while 'extension'  keeps  filesystem at the size it had.
There are then tricky cases i.e. user may have already 'reduced' filesystem to 'their' limit - and just wanted
to reduce LV to some other boundary size.  So here we may 'silently' behind the scene change behavior without noticing.

So this may need some new enhancements on 'fsadm' size:

What should we do when 'fsadm' see an unknown  device type (fs...) - currently stops:
maybe prompt if user wants to continue on given 'fs-type' ?

What 'fsadm' should do when it sees fs size < new reduced LV size  - currently we try to calc size and match new LV size
likely some prompting should be added ?

Comment 18 Zdenek Kabelac 2021-01-08 14:01:51 UTC
So from discussion with Lukas - with new REDUCIBLE attr seemed maybe a bit more complicated.

We both agree the existing big bold letter WARNING should be mostly sufficient.

But he seemed to look as better choice for the comment 9  '--checkfs n' -
where we would extend  'fsadm' with new functionality to report current FS size for
some commonly known/used FS seemed might looks more simple to users.

So this would require to 'always' active reduced LV and check for a content there,
which is worrying part - it's unclear if this would be implicit responsibility of '--checkfs y'
or for inactive LV we say use to activate reduced LV first - so user solves associated issues
with i.e. duplicated FS UUID being present in kernel aside first.
(and this is where REDUCIBLE attr solution shines)

After this check when it's known the size of FS fits - we proceed - 
so cases where user reduces FS in front of lvreduce would work normally.

Questionable could be the same usage on 'lvextend' part -  where we could validate
if the  bigger size is OK for 'fs'  as well  - this can catch some 'corner' cases
where users previously forcibly 'reduced' their size and now they try to cover
whole 'fs' back

Comment 19 David Teigland 2021-01-08 16:05:46 UTC
> So this would require to 'always' active reduced LV and check for a content
> there, which is worrying part - it's unclear if this would be implicit
> responsibility of '--checkfs y' or for inactive LV we say use to activate 
> reduced LV first

I wondered the same, and I think either approach is probably ok.

> Questionable could be the same usage on 'lvextend' part

Since there's no problem with lvextend, and nobody is asking for changes, I'd hesitate to change it.

Comment 20 Dave Wysochanski 2021-01-15 20:50:37 UTC
Old comment, but FWIW.

(In reply to Eric Sandeen from comment #8)
> Now I'm wondering if the customer problems are actually unique to XFS, or if
> they are simply more prevalent on XFS due to our choice of default
> filesystem in RHEL.
> 
> I'm curious about a couple of things:
> 
> 1) If you run lvreduce without "-r" on an LV containing an ext4 filesystem,
> will it reduce the LV without shrinking the filesystem first, and corrupt it?
Yes

> 2) If you run lvreduce with "-r" on an LV containing an XFS filesystem, does
> it properly fail to reduce the LV and keep the filesystem intact?
> 
Yes

> In other words, are the corrupt-xfs-filesystem cases from "lvreduce" due to
> administrator actions that would have caused the same problems on ext4?

Yes it is true that we also get cases with ext4, though of course not as many.  The study started from a comment on the 'xfs shrink' kbase, and I was surprised at what I found.  I did not expand it further to look at other filesystems / kbases.  FWIW, here is the one for ext4 and there are a decent number of cases on it: https://access.redhat.com/solutions/2132941

Comment 21 David Teigland 2021-04-28 22:15:20 UTC
Initial implementation:
https://sourceware.org/git/?p=lvm2.git;a=commit;h=ae51c9bff429774f57fc26f136221902272e7e71

lvreduce: check for file system that cannot be reduced

New command line option --checkfs y|n

New config setting global/lvreduce_checkfs=1|0

The default lvm.conf setting is 1 (enabled).

The command line option overrides lvm.conf.

If the new lvm.conf setting is not changed, and the
new command line option is not used, then lvreduce
will check the LV and look a file system type that
it knows can be reduced (ext4).  It will activate the
LV if it's not active already to do this check.
The LV will not be reduced unless a file system type
is found that lvreduce knows is reducible (only ext4
at this point.)

if lvm.conf lvreduce_checkfs=1 and --checkfs is not used,
then lvreduce will check fs.  (This is the default.)

if lvm.conf lvreduce_checkfs=0 and --checkfs is not used,
then lvreduce will not check fs.

if --checkfs y then lvreduce will check fs, overriding
any lvm.conf setting.

if --checkfs n then lvreduce will not check fs, overriding
any lvm.conf setting.


Example reducing active LV

# lvs ee
  LV   VG Attr       LSize   
  lv2  ee -wi-a----- 128.00m

# lvreduce -L64M ee/lv2
  Found file system type xfs that cannot be reduced (see --checkfs).

# lvreduce -L64M --checkfs n ee/lv2
  WARNING: Reducing active logical volume to 64.00 MiB.
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce ee/lv2? [y/n]: n
  Logical volume ee/lv2 NOT reduced.


Example reducing inactive LV

# lvs ee
  LV   VG Attr       LSize   
  lv2  ee -wi------- 128.00m

# lvreduce -L64M ee/lv2
  Found file system type xfs that cannot be reduced (see --checkfs).

# lvreduce -L64M --checkfs n ee/lv2
  Size of logical volume ee/lv2 changed from 128.00 MiB (32 extents) to 64.00 MiB (16 extents).
  Logical volume ee/lv2 successfully resized.

# lvs ee
  LV   VG Attr       LSize  
  lv2  ee -wi------- 64.00m

Comment 22 Zdenek Kabelac 2021-04-30 14:52:15 UTC
Review comments:

Thinking about --checkfs   option name might be worth here.

'checkfs' generally evokes  there will be some kind of 'fsck' operation going on to users.

So maybe option name like  --detectfs   or --testfspresence... or something more better describing what is the operation doing.


When  'blkid' is not compiled in - it's not clear to decide what is default  y|n.

But definitelly with option --force it should be bypassing/maybe even skipping detection (with printed log_warn("WARNING:...")

--- Enhancements --- 

Maybe detection should actually 'prompt'  y|n  if user surely wants to possibly corrupt found signatued LV ??

I've no clear opinion whether we should rather list known-to-fail or known-to-work signature names,
but in all cases, this the list should come from lvm.conf
aka user should be able to add 'signature' without binary patching.

Having then list empty might mean 'all goes' aka backward compatible setting 'don't care'

So replacing lvreduce_checkfs  with 

lvreduce_accepts = ["ext4", "ext3", ..... ]

(or permits, allows...  some good name fit goes)

Empty list means  'accept' all - so being backward compatible....

Gives us also possibly to join the option from above to match the list meaning - i.e.:

lvreduce  --useacceptlist....
lvreduce  --withpermission


When FS from set is found it should basically require/motivate user to use -r/--resizefs

So 'lvreduce  --OPTION y'   on ext4 LV - although is a reducible FS, should still probably 'error'  if the user has not specified -r.
Since 'fs' can reduce - but if the user has not reduced it ahead of time - it still can loose data.

(Although our fsadm has quite limited list of supported fs)

Comment 23 David Teigland 2021-04-30 18:44:31 UTC
Good points, --detectfs sounds fine and does seem more accurate than --checkfs.

After more thought, I think lvm.conf lvreduce_checkfs=y|n is unnecessary, so I've dropped that.
FS detection is now only controlled by --detectfs y|n (defaulting to y).

A list of shrinkable fs types in lvm.conf, the only reason I can think of would be to handle a file system that gains the ability to shrink (as xfs will likely do).  Then, a user could add "xfs" to that list in lvm.conf to enable shrinking of xfs.  However, I suspect this may depend on the on-disk version of a particular fs, not just on the type.  e.g. one old xfs on a system may not be shrinkable, while another new xfs on the system may be shrinkable.


Relevant option combinations of:
--detectfs y|n
--resizefs
LV active|inactive
xfs|ext4  (xfs represents a non-shrinkable fs, ext4 represents a shrinkable fs)
fsresize fails|succeeds

LV is inactive:
lvreduce --detectfs y LV_xfs
lvreduce --detectfs n LV_xfs
lvreduce --detectfs y --resizefs LV_xfs
lvreduce --detectfs n --resizefs LV_xfs
lvreduce --detectfs y LV_ext4
lvreduce --detectfs n LV_ext4
lvreduce --detectfs y --resizefs LV_ext4  (resize2fs success|fail)
lvreduce --detectfs n --resizefs LV_ext4

LV is active:
lvreduce --detectfs y LV_xfs
lvreduce --detectfs n LV_xfs
lvreduce --detectfs y --resizefs LV_xfs
lvreduce --detectfs n --resizefs LV_xfs
lvreduce --detectfs y LV_ext4
lvreduce --detectfs n LV_ext4
lvreduce --detectfs y --resizefs LV_ext4  (resize2fs success|fail)
lvreduce --detectfs n --resizefs LV_ext4  (resize2fs success|fail)


Verifying the behavior of each:

LV is inactive
---------------

lvreduce --detectfs y LV_xfs
. reports error fs type cannot be reduced
. does not reduce LV

lvreduce --detectfs n LV_xfs
. reduces LV, data loss

lvreduce --detectfs y --resizefs LV_xfs
. reports error fs type cannot be reduced
. does not reduce LV

lvreduce --detectfs n --resizefs LV_xfs
. reports error LV must be active
. does not reduce LV

lvreduce --detectfs y LV_ext4
. warning prompt "DESTROY DATA y/n"
. n: does not reduce LV
. y: reduces LV, data loss

lvreduce --detectfs n LV_ext4
. reduces LV, data loss

lvreduce --detectfs y --resizefs LV_ext4 (resize2fs success)
. resize2fs shrinks fs success
. reduces LV

lvreduce --detectfs y --resizefs LV_ext4 (resize2fs fails)
. resize2fs shrinks fs fails (fs is full)
. does not reduce LV

lvreduce --detectfs n --resizefs LV_ext4
. reports error LV must be active
. does not reduce LV


LV is active (and mounted)
--------------------------

lvreduce --detectfs y LV_xfs
. reports error fs type cannot be reduced
. does not reduce LV

lvreduce --detectfs n LV_xfs
. warning prompt "DESTROY DATA y/n"
. n: does not reduce LV
. y: reduces LV, data loss

lvreduce --detectfs y --resizefs LV_xfs
. reports error fs type cannot be reduced
. does not reduce LV

lvreduce --detectfs n --resizefs LV_xfs
. reports fsadm error shrinking is unsupported
. does not reduce LV

lvreduce --detectfs y LV_ext4
. warning prompt "DESTROY DATA y/n"
. n: does not reduce LV
. y: reduces LV, data loss

lvreduce --detectfs n LV_ext4
. warning prompt "DESTROY DATA y/n"
. n: does not reduce LV
. y: reduces LV, data loss

lvreduce --detectfs y --resizefs LV_ext4  (resize2fs success)
. prompt to unmount: y
. resize2fs shrinks fs success
. reduces LV

lvreduce --detectfs y --resizefs LV_ext4  (resize2fs fail)
. prompt to unmount: y
. resize2fs shrinks fs fails (fs is full)
. does not reduce LV

lvreduce --detectfs n --resizefs LV_ext4  (resize2fs success)
. prompt to unmount: y
. resize2fs shrinks fs success
. reduces LV

lvreduce --detectfs n --resizefs LV_ext4  (resize2fs fail)
. prompt to unmount: y
. resize2fs shrinks fs fails (fs is full)
. does not reduce LV

Comment 24 David Teigland 2021-04-30 18:57:48 UTC
One combination above that I think we should probably change:
--detectfs y, and --resizefs is not set, and LV is shrinkable.

Above, if the user answers 'y' to the prompt, there will be data loss.  I think we should change this to either require the user to set --detectfs n or set --resizefs.

Comment 25 David Teigland 2021-05-05 16:38:43 UTC
In the list above, I've changed the following case (for both active and inactive LVs):

lvreduce --detectfs y LV_ext4
. warning prompt "DESTROY DATA y/n"
. n: does not reduce LV
. y: reduces LV, data loss

to report an error suggesting --resizefs to avoid data loss:

lvreduce --detectfs y LV_ext4
. reports error fs type can be reduced (see --resizefs)

So, when lvreduce detects a shrinkable fs, the user needs to include --resizefs or set --detectfs n to disable the fs detection.

This does not yet account for the situation where a user has already shrunk the file system, e.g. by running resize2fs themselves, and then runs lvreduce to shrink the LV.  In this situation we're still requiring the user to run lvreduce with --detectfs n or --resizefs.  To better handle this case, lvreduce could detect the size of the file system and check that it is no larger that the reduced LV size.  The libblkid interface for checking the fs size is not yet implemented, but once it is we could add this (mounting the fs to run statfs() is probably a step beyond what lvm should do.)

https://sourceware.org/git/?p=lvm2.git;a=commit;h=7497bca317aba8da8d6f5d625345751d3c414b6e

    lvreduce: check for file system that cannot be reduced
    
    Add a new command line option --detectfs y|n, along with
    the capability to detect if the LV contains a shrinkable
    file system (using libblkid).
    
    The new fs detection is performed by default to avoid
    acccidental data loss by users who reduce an LV without
    knowing that the file system cannot be reduced, or without
    including the --resizefs option to reduce the reducible fs.
    The new errors reported for these cases are:
    
    $ lvreduce -L64M --detectfs y vg/lv
      Detected file system type xfs that cannot be reduced (from --detectfs).
    
    $ lvreduce -L64M --detectfs y vg/lv
      Detected file system type ext4 that can be reduced (see --resizefs).
    
    When detectfs is enabled, the lvreduce command will activate
    the LV to check the fs type.
    
    Using --detectfs n results in the same historical (and
    somewhat dangerous) lvreduce behavior.

Comment 26 Heinz Mauelshagen 2021-05-05 20:15:20 UTC
Dave,

what about adding/supporting an lvm.conf option like devices/detect_filesystem defaulting to 'y' ?

Comment 27 David Teigland 2021-05-05 20:33:10 UTC
(In reply to Heinz Mauelshagen from comment #26)
> what about adding/supporting an lvm.conf option like
> devices/detect_filesystem defaulting to 'y' ?

Good question, I thought the same, it was part of the original patch (comment 21 lvreduce_checkfs).  But, I dropped it when I couldn't think of a realistic scenario where it would be needed, i.e. who would really need to set it in lvm.conf instead of just setting it on the command line?

Comment 28 Zdenek Kabelac 2021-05-05 20:44:43 UTC
Backward compatibility is always for those users - who are aware of the 'risk' but do not want to change any old script and expect them to work without any change - so they set  lvm.conf option to provoke old logic
(aka lvreduction without any detection)

Comment 29 David Teigland 2021-05-07 16:00:41 UTC
I've added back the config option to determine if detectfs is performed when not specified on the command line, although I still wonder if anyone will use it.

I switched what happens when blkid is unavailable, or finds no file system, and have made go ahead with reducing the LV.

When blkid finds a file system, this is comparing it to a list of file systems that we know are shrinkable, and any others are considered non-shrinkable.  My current list of shrinkable fs's are ext4,3,2.  Which others should I include that we know can be shrunk?  Another question is whether others think this change should be targeted at RHEL8 or RHEL9?

https://sourceware.org/git/?p=lvm2.git;a=shortlog;h=refs/heads/dev-dct-lvreduce-check-4

Comment 30 David Teigland 2021-05-07 18:49:55 UTC
After talking with Eric it's clear that we should jump directly to the best solution, which is checking that the fs size is less than the requested LV size.  The blkid FSSIZE interface exists but just needs to be implemented for the specific fs types.

Comment 31 Dave Wysochanski 2021-08-31 15:08:18 UTC
Hey all - what is the status of this?  Can it make 8.6 at least?

Comment 32 David Teigland 2021-08-31 15:50:16 UTC
It became a lot more complicated that I feared (e.g. the old and dubious behaviors around automatically unmounting and remounting the fs from the lvm command.)  I do want to get working on this again soon and hopefully make some progress by 8.6 (maybe not complete since we don't know all the complications that may come up).  

One important prerequisite for the lvm work was getting the blkid FSSIZE interface implemented. This requires some fs internal knowledge that I'm hoping fs developers can more quickly fill in for us (at least for the common file systems.)

Comment 35 David Teigland 2022-02-14 17:12:04 UTC
This would need to be implemented in rhel9 first, then we'd consider if it's suitable for backporting to rhel8.

Comment 36 RHEL Program Management 2022-03-14 07:27:18 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 37 Dave Wysochanski 2022-03-14 09:22:55 UTC
Hey David T (and I guess Eric) - what are the next steps needed here?  I guess I am not following why this got stalled.

Comment 39 David Teigland 2022-03-16 14:30:49 UTC
I still want to do this but it turned into a larger problem.  The last time I was working on it I was blocked by FSSIZE missing for most file systems in in libblkid.

Comment 40 Dave Wysochanski 2022-03-16 15:24:45 UTC
(In reply to David Teigland from comment #39)
> I still want to do this but it turned into a larger problem.  The last time
> I was working on it I was blocked by FSSIZE missing for most file systems in
> in libblkid.

Do we need an upstream issue filed (https://github.com/util-linux/util-linux/issues) or what is the next step(s)?  I briefly looked at the code and it looks like 'lsblk' has this but libblkid does not?  If I'm reading it correctly, maybe we need a RHEL9 libblkid bz, then make this depend on that one, with a 'devel conditional nack: upstream'?  I'm not sure how hard this is to do or even if it's on anyones radar.  If you or Eric know more it would be good to sort that out soon since it sounds like this is what stalled progress on this bug - sorry I didn't notice it until now.

Comment 41 David Teigland 2022-03-16 15:43:13 UTC
I discussed this with Eric at the time but didn't actually file anything.  I've just created bug 2064810 for util-linux for this, which I expect will come back to Eric's group since it requires an understanding of specific on disk file system metadata.

Comment 42 Eric Sandeen 2022-03-16 16:02:54 UTC
It's been a while since I thought about this, so trying to get it paged back in.

It does cross my mind that querying FSSIZE and then making a reduce go/no-go decision isn't really atomic, right - the filesystem could have been grown between checking, and reducing. But I suppose that's a corner case and we could still make this better, even if it's not air-tight, if we had a common way to query filesystem size.

Comment 43 David Teigland 2022-05-16 15:44:00 UTC
I've made substantial progress on this, but it won't be finished quite in time for 8.7/9.1.

Comment 44 David Teigland 2022-06-22 20:52:26 UTC
I'm beginning to test the initial implementation, and here's the preliminary description of the change from the commit summary.

    lvresize: add new options and defaults for fs handling
    
    The new option "--fs String" for lvresize/lvreduce/lvextend
    controls the treatment of file systems before/after resizing
    the LV.  --resizefs is translated to --fs resize_remount.
    
    Possible --fs values:
    
    checksize
      Check the fs size and reduce the LV if the fs is not using
      the affected space, i.e. the fs does not need to be shrunk.
      Fail the command without reducing the fs or LV if the fs is
      using the affected space. (checksize does nothing when
      extending the LV.)
    
    resize_remount
      Resize the fs if needed. Mounts or unmounts the fs as
      required (avoids mounting/unmounting when possible.)
      Attempts to restore the original mount state when finished.
    
    resize_keepmount
      Resize the fs if needed, only if it can be done without
      changing the current mount state.  Fail the command without
      resizing the fs or LV if an fs resize requires mounting or
      unmounting.
    
    resize_unmount
      Resize the fs if needed, only while unmounted. Unmount the
      fs if needed.  Fail the command without resizing the fs
      or LV if an fs resize is needed that requires the the fs
      to be mounted.
    
    resize
      Equivalent to resize_remount.
    
    resize_fsadm
      Use the old method of calling fsadm to do handle the fs (deprecated).
    
    ignore
      Resize the LV without checking for or handling a file system.
    
    Notes on lvreduce:
    
    When no --fs or --resizefs option is specified, lvreduce now
    activates the LV and uses the checksize behavior.
    
    With the exception of --fs resize_fsadm|ignore, lvreduce requires
    the recent libblkid fields FSLASTBLOCK and FSBLOCKSIZE.
    FSLASTBLOCK*FSBLOCKSIZE is the last byte used by the fs on the LV,
    which determines if reducing the fs is necessary.

Comment 48 RHEL Program Management 2022-09-14 07:27:49 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 53 Corey Marthaler 2022-11-29 15:19:12 UTC
Moving this back to ASSIGNED to reflect that this isn't currently waiting on QA per blocking bug 2141837.

Comment 57 Corey Marthaler 2022-12-12 18:01:25 UTC
Marking Verified:Tested based on initial regression test results for this feature with the latest rpms.

kernel-5.14.0-205.el9    BUILT: Fri Dec  2 07:14:37 AM CST 2022
lvm2-2.03.17-3.el9    BUILT: Wed Dec  7 10:41:40 AM CST 2022
lvm2-libs-2.03.17-3.el9    BUILT: Wed Dec  7 10:41:40 AM CST 2022

xfs_active_reduce_no_resizefs --  Create an XFS filesysem and attempt to reduce the size while ACTIVE but w/o --fs resize 
ext4_active_reduce_no_resizefs --  Create an EXT4 filesysem and attempt to reduce the size while ACTIVE but w/o --fs resize 
ext3_active_reduce_no_resizefs --  Create an EXT3 filesysem and attempt to reduce the size while ACTIVE but w/o --fs resize 
xfs_active_reduce_ignorefs --  Create an XFS filesysem and reduce the size while ACTIVE, using --fs ignore 
ext4_active_reduce_ignorefs --  Create an EXT4 filesysem and reduce the size while ACTIVE, using --fs ignore 
ext3_active_reduce_ignorefs --  Create an EXT3 filesysem and reduce the size while ACTIVE, using --fs ignore 
xfs_active_reduce_resizefs --  Create an XFS filesysem and reduce the size while ACTIVE, using --fs resize and --resizefs 
ext4_active_reduce_resizefs --  Create an EXT4 filesysem and reduce the size while ACTIVE, using --fs resize and --resizefs 
ext3_active_reduce_resizefs --  Create an EXT3 filesysem and reduce the size while ACTIVE, using --fs resize and --resizefs 
xfs_inactive_reduce --  Create an XFS filesysem and attempt to reduce the size while INACTIVE 
ext4_inactive_reduce --  Create an EXT4 filesysem and attempt to reduce the size while INACTIVE 
ext3_inactive_reduce --  Create an EXT3 filesysem and attempt to reduce the size while INACTIVE 
xfs_active_reduce_wo_touching_fs_space --  Create an XFS filesysem, extend the volume, and then reduce the space wo touching the filesystem area while ACTIVE 
ext3_active_reduce_wo_touching_fs_space --  Create an EXT3 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while ACTIVE 
ext4_active_reduce_wo_touching_fs_space --  Create an EXT4 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while ACTIVE 
xfs_inactive_reduce_wo_touching_fs_space --  Create an XFS filesysem, extend the volume, and then reduce the space wo touching the filesystem area while INACTIVE 
ext3_inactive_reduce_wo_touching_fs_space --  Create an EXT3 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while INACTIVE 
ext4_inactive_reduce_wo_touching_fs_space --  Create an EXT4 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while INACTIVE 
no_fs_active_reduce --  Create a variety of LV types and attempt to reduce the size while ACTIVE and w/o a filesystem 
no_fs_inactive_reduce --  Create a variety of LV types and attempt to reduce the size while INACTIVE and w/o a filesystem 
no_fs_positive_reduce_attempt --  Verify that a positive reduce attempt doesn't end up extending the volume 
no_fs_negative_extend_attempt --  Verify that a negative extend attempt doesn't end up reducing the volume 
xfs_active_online_traditional_resizefs_extend --  Create an XFS filesystem on a variety of LV types and extend the size while ACTIVE and ONLINE, using the traditional --resizefs flag 
ext3_active_online_traditional_resizefs_extend --  Create an EXT3 filesystem on a variety of LV types and extend the size while ACTIVE and ONLINE, using the traditional --resizefs flag 
ext4_active_online_traditional_resizefs_extend --  Create an EXT4 filesystem on a variety of LV types and extend the size while ACTIVE and ONLINE, using the traditional --resizefs flag

Comment 62 Corey Marthaler 2023-02-02 16:15:00 UTC
Marking final verification of this feature with the latest rpms.

kernel-5.14.0-231.el9    BUILT: Mon Jan  9 08:32:41 PM CET 2023
lvm2-2.03.17-4.el9    BUILT: Tue Jan 10 06:40:12 PM CET 2023
lvm2-libs-2.03.17-4.el9    BUILT: Tue Jan 10 06:40:12 PM CET 2023


Final Test plan for 9.2:
    xfs_active_reduce_no_resizefs --  Create an XFS filesysem and attempt to reduce the size while ACTIVE but w/o --fs resize 
    ext4_active_reduce_no_resizefs --  Create an EXT4 filesysem and attempt to reduce the size while ACTIVE but w/o --fs resize 
    luks_ext4_active_reduce_no_resizefs --  Create an Encrypted LUKs EXT4 filesysem and attempt to reduce the size while ACTIVE but w/o --fs resize 
    ext3_active_reduce_no_resizefs --  Create an EXT3 filesysem and attempt to reduce the size while ACTIVE but w/o --fs resize 
    xfs_active_reduce_ignorefs --  Create an XFS filesysem and reduce the size while ACTIVE, using --fs ignore 
    ext4_active_reduce_ignorefs --  Create an EXT4 filesysem and reduce the size while ACTIVE, using --fs ignore 
    luks_ext4_active_reduce_ignorefs --  Create an Encrypted LUKs EXT4 filesysem and reduce the size while ACTIVE, using --fs ignore which could potentially cause corruption
    ext3_active_reduce_ignorefs --  Create an EXT3 filesysem and reduce the size while ACTIVE, using --fs ignore 
    xfs_active_reduce_resizefs --  Create an XFS filesysem and reduce the size while ACTIVE, using --fs resize and --resizefs 
    ext4_active_reduce_resizefs --  Create an EXT4 filesysem and reduce the size while ACTIVE, using --fs resize and --resizefs 
    luks_ext4_active_reduce_resizefs --  Create an Encrypted LUKs EXT4 filesysem and reduce the size while ACTIVE, using --fs resize and --resizefs (bug 2157591) 
    ext3_active_reduce_resizefs --  Create an EXT3 filesysem and reduce the size while ACTIVE, using --fs resize and --resizefs 
    xfs_inactive_reduce --  Create an XFS filesysem and attempt to reduce the size while INACTIVE 
    ext4_inactive_reduce --  Create an EXT4 filesysem and attempt to reduce the size while INACTIVE 
    luks_ext4_inactive_reduce --  Create an Encrypted LUKs EXT4 filesysem and attempt to reduce the size while INACTIVE 
    ext3_inactive_reduce --  Create an EXT3 filesysem and attempt to reduce the size while INACTIVE 
    xfs_active_reduce_wo_touching_fs_space --  Create an XFS filesysem, extend the volume, and then reduce the space wo touching the filesystem area while ACTIVE 
    ext3_active_reduce_wo_touching_fs_space --  Create an EXT3 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while ACTIVE 
    ext4_active_reduce_wo_touching_fs_space --  Create an EXT4 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while ACTIVE 
    luks_ext4_active_reduce_wo_touching_fs_space --  Create an Encrypted LUKs EXT4 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while ACTIVE (bug 2157591) 
    xfs_inactive_reduce_wo_touching_fs_space --  Create an XFS filesysem, extend the volume, and then reduce the space wo touching the filesystem area while INACTIVE 
    ext3_inactive_reduce_wo_touching_fs_space --  Create an EXT3 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while INACTIVE 
    ext4_inactive_reduce_wo_touching_fs_space --  Create an EXT4 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while INACTIVE 
    luks_ext4_inactive_reduce_wo_touching_fs_space --  Create an Encrypted LUKs EXT4 filesysem, extend the volume, and then reduce the space wo touching the filesystem area while INACTIVE 
    no_fs_active_reduce --  Create a variety of LV types and reduce the size while ACTIVE and w/o a filesystem 
    luks_no_fs_active_reduce --  Create a variety of Encrypted LUKS LV types and reduce the size while ACTIVE and w/o a filesystem 
    no_fs_inactive_reduce --  Create a variety of LV types and attempt to reduce the size while INACTIVE and w/o a filesystem 
    luks_no_fs_inactive_reduce --  Create a variety of Encrypted LUKS LV types and attempt to reduce the size while INACTIVE and w/o a filesystem 
    xfs_inactive_extend_no_resizefs --  Create an XFS filesysem and extend the size while INACTIVE w/o using the --fs resize flag 
    ext4_inactive_extend_no_resizefs --  Create an EXT4 filesysem and extend the size while INACTIVE w/o using the --fs resize flag 
    luks_ext4_inactive_extend_no_resizefs --  Create an Encrypted LUKs EXT4 filesysem and extend the size while INACTIVE w/o using the --fs resize flag 
    ext3_inactive_extend_no_resizefs --  Create an EXT3 filesysem and extend the size while INACTIVE w/o using the --fs resize flag 
    no_fs_active_extend --  Create a variety of LV types and extend the size while ACTIVE and w/o a filesystem 
    luks_no_fs_active_extend --  Create a variety of Encrypted LUKS LV types and extend the size while ACTIVE and w/o a filesystem 
    no_fs_inactive_extend --  Create a variety of LV types and attempt to extend the size while INACTIVE and w/o a filesystem 
    luks_no_fs_inactive_extend --  Create a variety of Encrypted LUKS LV types and attempt to extend the size while INACTIVE and w/o a filesystem 
    no_fs_active_resizefs_extend --  Create a variety of LV types and attempt to extend the size while ACTIVE and w/o a filesystem, but with the use of --fs resize 
    luks_no_fs_active_resizefs_extend --  Create a variety of Encrypted LUKS LV types and attempt to extend the size while ACTIVE and w/o a filesystem, but with the use of --fs resize 
    no_fs_inactive_resizefs_extend --  Create a variety of LV types and attempt to extend the size while INACTIVE and w/o a filesystem, but with the use of --fs resize 
    luks_no_fs_inactive_resizefs_extend --  Create a variety of Encrypted LUKS LV types and attempt to extend the size while INACTIVE and w/o a filesystem, but with the use of --fs resize (bug 2165691) 
    no_fs_positive_reduce_attempt --  Verify that a positive reduce attempt doesn't end up extending the volume 
    no_fs_negative_extend_attempt --  Verify that a negative extend attempt doesn't end up reducing the volume 
    xfs_active_online_traditional_resizefs_fsresize_fsignore_extend --  Create an XFS filesystem on a variety of LV types and extend the size while ACTIVE and ONLINE, using the traditional --resizefs flag, as well as --fs resize, and --fs ignore 
    ext3_active_online_traditional_resizefs_fsresize_fsignore_extend --  Create an EXT3 filesystem on a variety of LV types and extend the size while ACTIVE and ONLINE, using the traditional --resizefs flag, as well as --fs resize, and --fs ignore 
    ext4_active_online_traditional_resizefs_fsresize_fsignore_extend --  Create an EXT4 filesystem on a variety of LV types and extend the size while ACTIVE and ONLINE, using the traditional --resizefs flag, as well as --fs resize, and --fs ignore 
    luks_ext4_active_online_traditional_resizefs_fsresize_fsignore_extend --  Create an Encrypted LUKs EXT4 filesystem on a variety of LV types and extend the size while ACTIVE and ONLINE, using the traditional --resizefs flag as well as --fs resize, and --fs ignore

Comment 67 errata-xmlrpc 2023-05-09 08:23:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (lvm2 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2544


Note You need to log in before you can comment on or make changes to this bug.