Bug 1185117

Summary: Custom partitioning does not allow convenient removal of volume including snapshots (btrfs, LVM)
Product: [Fedora] Fedora Reporter: Chris Murphy <bugzilla>
Component: anacondaAssignee: Anaconda Maintenance Team <anaconda-maint-list>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 26CC: anaconda-maint-list, awilliam, bugzilla, danofsatx, g.kaviyarasu, jonathan, kparal, robatino, vanmeeuwen+fedora
Target Milestone: ---Keywords: CommonBugs, Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: RejectedBlocker https://fedoraproject.org/wiki/Common_F22_bugs#anaconda-hard-to-remove-snapshotted-partitions
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-29 11:49:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
are you sure you want to delete?
none
manual partitioning list of "multiple" opensuse installations
none
manual partitioning list contents of one of the installations
none
anaconda.log
none
program.log
none
storage.log none

Description Chris Murphy 2015-01-23 01:35:39 UTC
Use case description
--------------------

A UEFI computer is configured to dual boot Windows and OpenSUSE 13.2. User wishes to replace openSUSE with Fedora. This is possible with automatic partitioning, but it's impractical to do with with manual partitioning.

Setup steps
-----------

1. UEFI Windows 8 default installation onto baremetal.
2. OpenSuse 13.2 default installation onto the same system. [1]
3. Reboot OpenSuse and use it normally for ~4 hours: do a recommended system update, install a couple of small programs. [2]
4. Reboot using Fedora install media, go to Manual Partitioning. [See screenshots attached.]

Actual results
--------------

- There are over 40 instances of one OpenSuse installation presented in the UI.
- Each instance contains 19 mount points.
- Due to bug 1183880 I cannot "delete all other file systems" or it would render Windows unbootable, a bug anaconda maintainers consider not a bug.
- Due to the UI requiring 4 clicks to delete each mount point one by one, this is over 3000 clicks to remove one instance of OpenSuse 13.2 while also retaining bootablity of Windows 8.1, and installing Fedora.

Since no one would click 3000 times to test whether the installation to replace opensuse in this use case would actually succeed, I nominate it as a final release blocker because it "hinders execution of required Final test plans or dramatically reduces test coverage".

Unfortunately, a user-centric criteria doesn't apply. In fact as designed, a user could click 3000 times and chances are it would work, but since it's totally impractical to test whether that's true, at the moment I think this is the most applicable criteria.

Additionally, this same behavior in anaconda occurs with LVM thinp snapshots. Both thinp snapshots, and Btrfs snapshots can reasonably be in the hundreds, and this functionality is expected to scale to thousands. This openSUSE example system was only about 4 hours old.


Relates to the following anaconda bugs
--------------------------------------

wrongly permits deletion of shared EFI System partition
https://bugzilla.redhat.com/show_bug.cgi?id=1183880

RFE: how to show multiple instances of other (or prior Fedora) installs, instead of snapshots ## This is arguably now a duplicate of this bug.
https://bugzilla.redhat.com/show_bug.cgi?id=1015657

RFE: always create required bootloader partitions in custom partitioning
https://bugzilla.redhat.com/show_bug.cgi?id=1022316


Additional Info
----------------

[1]
By default, it shrinks the largest Windows volume, creating two partitions in its place. One is XFS, mounted at /home. The other is Btrfs with 15 subvolumes.
boot/grub2/i386-pc
boot/grub2/x86_64-efi
opt
srv
tmp
usr/local
var/crash
var/lib/mailman
var/lib/named
var/lib/pgsql
var/log
var/opt
var/spool
var/tmp
.snapshots

[2]
By default, OpenSuse uses snapper. Snapper creates Btrfs snapshots periodically, usually before any software updates or installation. This 4 hour old OpenSystem system now has over 55 snapshots.

Comment 1 Chris Murphy 2015-01-23 01:38:49 UTC
Created attachment 983115 [details]
are you sure you want to delete?

This shortcut checkbox can't be used to remove this single openSUSE installation in one shot due to bug 1183880. This dialog appears for each mount point the user wishes to delete, and the opensuse installation has hundreds of defined mount points according to anaconda UI.

Comment 2 Chris Murphy 2015-01-23 01:40:17 UTC
Created attachment 983116 [details]
manual partitioning list of "multiple" opensuse installations

There is only one opensuse installation, but the UI presents each snapshot as a unique installation.

Comment 3 Chris Murphy 2015-01-23 01:42:59 UTC
Created attachment 983117 [details]
manual partitioning list contents of one of the installations

After one mount point is deleted, the UI automatically closes that snapshot's reveal view. One click to reveal the contents, one click to select the mount point, one click for the minus button, and one click to Delete It.

Comment 4 Fedora Blocker Bugs Application 2015-01-23 01:46:20 UTC
Proposed as a Blocker for 22-final by Fedora user chrismurphy using the blocker tracking app because:

 Since no one would click 3000 times to test whether the installation to replace opensuse in this use case would actually succeed, the bug "hinders execution of required Final test plans or dramatically reduces test coverage".

Comment 5 David Shea 2015-01-23 13:30:39 UTC
Please attach the logs from /tmp

Comment 6 Chris Murphy 2015-01-23 16:57:47 UTC
Created attachment 983460 [details]
anaconda.log

Comment 7 Chris Murphy 2015-01-23 16:58:05 UTC
Created attachment 983461 [details]
program.log

Comment 8 Chris Murphy 2015-01-23 16:58:20 UTC
Created attachment 983462 [details]
storage.log

Comment 9 Dan Mossor [danofsatx] 2015-01-26 17:48:52 UTC
Discussed at Fedora Blocker Review Meeting 2015-01-26

http://meetbot.fedoraproject.org/fedora-blocker-review/2015-01-26/f22-blocker-review.2015-01-26-17.00.log.txt

AcceptedBlocker for Beta - This bug is a violation of the Beta criterion: "Correctly interpret, and modify as described below, any disk with a valid ms-dos or gpt disk label and partition table containing ext4 partitions, LVM and/or btrfs volumes..."

Comment 10 Adam Williamson 2015-01-26 20:20:57 UTC
anaconda folks, please let us know if you disagree with the determination here - we explicitly considered that it's still quite a while before Beta and there's time for us to re-consider the decision if necessary.

Comment 11 Jaroslav Reznik 2015-03-03 16:45:20 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22

Comment 12 Brian Lane 2015-03-16 15:56:32 UTC
I really don't think this is a blocker. You've got a situation where a user has created a system that's going to be really complicated for Anaconda to try and clean up without deleting something they want to keep. I think in this case the burden should fall on them to get the system into a state where it can be installed.

Comment 13 Chris Murphy 2015-03-16 18:38:11 UTC
(In reply to bcl from comment #12)
> You've got a situation where a user
> has created a system that's going to be really complicated for Anaconda to
> try and clean up without deleting something they want to keep. 

a.) The user stated they want to replace openSUSE with Fedora. Deletion of the entire Btrfs volume is implied, but the custom UI doesn't permit this. It requires an unreasonably laborious deletion of thousands of snapshots first.

b.) The user didn't create the system. It's an out of the box UEFI laptop with Windows 8.1 preinstalled, followed by a default openSUSE install that's less than a day old.

c.) Guided UI doesn't have this problem. It permits the entire Btrfs volume to be marked for deletion with two clicks.

> I think in
> this case the burden should fall on them to get the system into a state
> where it can be installed.

How is that at all compatible with this very long standing basic, reasonable release criteria?

"When using the custom partitioning flow, the installer must be able to: Remove existing storage volumes"

It only lets me remove subvolumes. I can't remove a whole Btrfs volume.

The same problem exists with LVM thin provisioning too, because I can't delete a Volume Group, only individual LVs.

Comment 14 Adam Williamson 2015-03-23 21:19:25 UTC
I think Chris makes a reasonable point, there; it seems legitimate to me that custom part should (like guided part) make it reasonably viable to remove an entire volume even when this kind of snapshotting feature has been used (which is a standard feature for LVM and btrfs, AIUI).

Comment 15 David Lehman 2015-03-30 15:23:35 UTC
Is it not possible to delete the subvolume that contains the snapshots?

Comment 16 Chris Murphy 2015-03-30 17:02:30 UTC
I don't know, probably. It assumes the user knows what to look for though. Feel free to mark this with insufficient data and kick it to Fedora 23. Getting the UI to enable pool removal (Btrfs volumes, not subvols; and LVM VG's, not LV's) may be more invasive than the Fedora 22 timeframe permits.

LVM thin volume snapshots can't be nested, so a way to remove VG's seems necessary. While Btrfs snapshots could be nested, it's not required. And there's no spec or standard for these layouts.

So my prediction is Btrfs volumes can't be shared among distros in the short-medium term, and the user will just want to obliterate the whole volume. And that means some way to remove the whole pool. In the meantime, it's much easier to just use CLI tools to destroy an LVM VG, or Btrfs volume (or use guided partitioning) than playing hide and seek with piles of snapshots and whatever subvolume might contain them (or not).

Comment 17 David Lehman 2015-03-30 17:56:50 UTC
We should definitely defer this until we have extra time to optimize the flow when a great number of snapshots are present.

This bug should never have been a blocker. This is a situation most users will never encounter. On top of that, it's a matter of how convenient it is -- not whether it is possible.

Comment 18 Chris Murphy 2015-03-30 18:56:15 UTC
(In reply to David Lehman from comment #17)
I agree with your conclusion, I don't agree with the assessment though. The blocker worthiness is reasonable, it's just impractical given the timeframe.

- It actually isn't possible with LVM thinp snapshots because they aren't nested, and the installer UI lacks a way to delete/remove VGs. And snapper support thinp snapshots in additional to Btrfs. It's still 14 subvolumes that aren't snapshots I'd have to delete, which itself takes 56 trackpad clicks. That's impractical to the point it might as well be impossible.

- Criterion says removal of volumes (not merely sub portions) is required functionality, it goes back to oldUI.

- No criterion comes close to saying most (a majority) of users have to hit a problem for it to qualify as a blocker.

- Docker, and now even systemd-219 create subvolumes piles of snapshots for container management. This isn't an obscure issue isolated to just the default openSUSE installation example.

Comment 19 Adam Williamson 2015-03-30 20:05:56 UTC
Note that when the criteria use the term 'volume' it is not in the sense of a btrfs 'volume' or an LVM 'volume'. It is a more generic sense, referring to any sort of thing you could put a filesystem on - it's the word we use instead of 'partitions' because 'partitions' isn't strictly accurate in all cases. I picked this usage of the term up from anaconda itself a while back.

Comment 20 Chris Murphy 2015-03-30 22:36:57 UTC
Whether "volume" means strictly the block device an fs is created on, or includes the pool concept, Btrfs volumes can't be removed in the current UI.

The UI let's the user directly create pools in the form of VGs and Btrfs volumes, but not directly remove them. A complete manual tear down is required to release their space. It's always been a criterion problem, it's just had a work around: tear downs aren't typically that tedious. Well now they are.

Further, "volume" meaning block device only can be construed to mean any such individual block device used to create a pool. That's a resize operation which I think is out of scope for Anaconda, and why I'm specifically referencing pool destruction.

Comment 21 Adam Williamson 2015-04-01 23:17:33 UTC
As anaconda team disagreed with the blocker determination on this bug and would like it re-considered, I'm dropping the AcceptedBlocker mark, which makes it back into a proposed blocker. It will be discussed on test@ and/or at the next blocker review meeting.

Comment 22 Adam Williamson 2015-04-06 16:29:09 UTC
Discussed at 2015-04-06 blocker review meeting: https://meetbot.fedoraproject.org/fedora-blocker-review/2015-04-06/f22-blocker-review.2015-04-06-16.00.log.txt . With anaconda team's input that this is considered new development, not bug fixing, and unlikely to be viable in the Fedora 22 timeframe, we agreed that this can be considered not a blocker.

It's a conditional violation of the criterion (the 'condition' being 'an existing layout that uses some form of snapshotting'), so the judgment is subjective, and we can incorporate factors like the complexity of fixing the 'bug', time required, likelihood of breaking other things, and so on.

Note that the bug could become a blocker for future releases as use of LVM/btrfs snapshotting becomes more prevalent.

Comment 23 Fedora End Of Life 2016-07-19 12:41:58 UTC
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 24 Adam Williamson 2016-08-19 21:12:42 UTC
I believe this is still valid, right Chris?

Comment 25 Chris Murphy 2016-08-19 21:33:09 UTC
Pretty sure it's at least somewhat fixed, if not mostly fixed, although I haven't tested it. There was a Fedora Magazine or Fedora People blog post by one of the developers describing the ability to multiselect LV's and Btrfs subvolumes and delete them together. Could you do this for 1000 snapshots? Egads... but I guess we just have to see how that plays out when there's more snapshotting down the road.

Comment 26 Fedora End Of Life 2017-02-28 09:40:05 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 27 Fedora End Of Life 2018-05-03 08:06:25 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 28 Fedora End Of Life 2018-05-29 11:49:24 UTC
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.