Bug 2186158 - Custom partitioning: A btrfs partition can't be reformatted and a subvolume removal fails
Summary: Custom partitioning: A btrfs partition can't be reformatted and a subvolume r...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: 39
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Radek Vykydal
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker https://discussion.fe...
Depends On:
Blocks: F39BetaBlocker
TreeView+ depends on / blocked
 
Reported: 2023-04-12 09:05 UTC by Kamil Páral
Modified: 2023-09-15 11:29 UTC (History)
10 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2023-08-24 13:44:12 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
screencast demonstration of the bug (2.76 MB, video/webm)
2023-04-12 09:06 UTC, Kamil Páral
no flags Details
Same behavior at a F33 install (145.50 KB, image/jpeg)
2023-04-12 13:36 UTC, Geraldo Simião
no flags Details
trying to remove btrfs subvolumes in Custom part (727.10 KB, video/webm)
2023-04-12 16:11 UTC, Kamil Páral
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github rhinstaller anaconda pull 4812 0 None open Improve btrfs handling in custom partitioning 2023-06-05 15:06:55 UTC

Description Kamil Páral 2023-04-12 09:05:41 UTC
Description of problem:
A partition which contains a btrfs filesystem can't be reused during installation, because Anaconda doesn't allow to reformat that partition.

In this example, I have a default Fedora Workstation installation (in BIOS mode), where vda2 is /boot and vda3 is / (and /home as a subvolume):

$ lsblk -o NAME,SIZE,FSTYPE,TYPE,PARTTYPENAME /dev/vda
NAME   SIZE FSTYPE TYPE PARTTYPENAME
vda     20G        disk 
├─vda1   1M        part BIOS boot
├─vda2   1G ext4   part Linux filesystem
└─vda3  19G btrfs  part Linux filesystem


Anaconda allows me to reformat vda1 and vda2, but doesn't allow me to reformat vda3. That means I can't use vda3 to put a root filesystem there (either a fresh btrfs, or a different filesystem). Anaconda requires a root filesystem to be reformatted. It says "You must create a new filesystem on the root device". Catch 22.

This affects both "Custom partition" and "Blivet-GUI partitioning". In Custom, the File System -> Reformat checkbox isn't active (nor Volume -> Modify). In Blivet-GUI, the Format option isn't active, and it doesn't help even when I remove all of btrfs filesystem from that partition (I assume that should definitely help, but it doesn't).

If I want to use that space, I have to delete the whole partition and create it again. Of course, that might not be what I want. I should be able to re-use an existing partition, if I'm OK with reformatting the filesystem (as is the case with other filesystems than btrfs).

I'm attaching a screencast video.

Version-Release number of selected component (if applicable):
anaconda-38.23.4-2.fc38.x86_64
Fedora-Workstation-Live-x86_64-38-1.4.iso

How reproducible:
always

Steps to Reproduce:
1. Create a default Fedora Workstation 37/38 installation (using btrfs). I only tested BIOS mode, but it doesn't probably matter.
2. Run the installer and try to use the former root partition (vda3 in my case). Reformatting the filesystem is expected.
3. Anaconda doesn't allow you to use the partition, you have to completely re-create it.

Additional info:
This problem already existed in Fedora 36 and 37, Anaconda behaves the same way there.

Comment 1 Kamil Páral 2023-04-12 09:06:38 UTC
Created attachment 1957109 [details]
screencast demonstration of the bug

See this video to illustrate the problem.

Comment 2 Kamil Páral 2023-04-12 09:11:19 UTC
I believe this falls under this release criterion:
https://fedoraproject.org/wiki/Fedora_38_Final_Release_Criteria#Disk_layouts
although the footnote says "Installation Destination spoke attempts to let you do" and this bug is exactly about not letting me do it. But it seems reasonable to let me wipe a btrfs filesystem.

At the same time, I don't have expectations that this would block F38 when it behaves the same way for F36 and F37, we're already at F38 RC time, and we didn't get huge complaints. Perhaps we can consider it for F39, though.

Comment 3 Geraldo Simião 2023-04-12 13:36:36 UTC
Created attachment 1957161 [details]
Same behavior at a F33 install

It seems we have anaconda doing this since btrfs landed as fedora default filesystem, at F33.

Comment 4 Vojtech Trefny 2023-04-12 13:37:01 UTC
This is a known problem in the way btrfs subvolumes are presented in the custom partitioning. From our (blivet) point of view a btrfs volume with two subvolumes (root and home, default btrfs partitioning in Fedora) is a "device with two children" so you cannot simply reformat the /dev/vda3 btrfs volume to ext4 for example, you need to remove all the "devices" on top of that. It's not ideal, but it also prevents you from reformatting your root subvolume and deleting your home by accident simply because you didn't realize the two subvolumes actually share a single device. (And you cannot reformat just the root subvolume, because it's actually not a separate device.) A better representation of the btrfs devices is definitely needed, but for now, I would consider this is working "correctly".

> In Blivet-GUI, the Format option isn't active, and it doesn't help even when I remove all of btrfs filesystem from that partition (I assume that should definitely help, but it doesn't).

Removing the btrfs subvolume should definitely allow you to reformat the device, I would consider this to be a bug in blivet-gui.

Comment 5 Adam Williamson 2023-04-12 15:39:06 UTC
-4 in https://pagure.io/fedora-qa/blocker-review/issue/1152 , marking rejected blocker.

Comment 6 Kamil Páral 2023-04-12 16:11:46 UTC
Created attachment 1957224 [details]
trying to remove btrfs subvolumes in Custom part

Thanks, Vojtech. So this is a bug in blivet-gui, OK. How about the Custom part? How am I supposed to reformat the partition? If I try to remove the remaining subvolumes except the root one, which should work according to your description, it still seems broken. When I try to remove the /home subvolume, it stays present, but its name and label gets renamed. If I click on it again (which makes an extra "boink" sound) and remove it again, the whole btrfs device disappears. There's no vda3 present. See the attached video.

Comment 7 Vojtech Trefny 2023-04-26 09:03:42 UTC
(In reply to Kamil Páral from comment #6)
> Created attachment 1957224 [details]
> When I try to remove the /home
> subvolume, it stays present, but its name and label gets renamed. If I click
> on it again (which makes an extra "boink" sound) and remove it again, the
> whole btrfs device disappears. There's no vda3 present. See the attached
> video.

tl;dr This is a bug in Anaconda, the home subvolume is deleted after the first action. Anaconda then shows the entire volume as /home mountpoint which is incorrect. Removing the "second /home" removes the entire volume.

This is again a result of the implementation of btrfs support in blivet and anaconda. This is how blivet sees the default Fedora btrfs partitioning:

------
  existing 9 GiB partition vda3 (28) with existing btrfs filesystem
    existing 9 GiB btrfs volume fedora (37) with existing btrfs filesystem
      existing 9 GiB btrfs subvolume home (48) with existing btrfs filesystem
      existing 9 GiB btrfs subvolume root (44) with existing btrfs filesystem
        existing 9 GiB btrfs subvolume root/var/lib/portables (52) with existing btrfs filesystem
------

We have the /dev/vda3 partition formatted to btrfs, on top of it we construct a btrfs volume with 3 subvolumes. Anaconda shows only the top/leaf devices, so in this case /home, / and /var/lib/portables subvolumes and hides the volume and the partition. This is similar logic to LVM -- the partition with the lvmpv format and the volume group are hidden, only the logical volumes (with mountpoints) are shown. With btrfs this is more complicated because there aren't separate devices for the volume and subvolume, everything is just a single partition (or multiple partitions for multidevice setups) with btrfs format. We need the abstraction internally to deal with one device having multiple mountpoint and possibly spanning multiple disks.

After deleting the /home subvolume Anaconda should display only the 2 remaining subvolumes, but for some reason it decides to show the (previously hidden) volume and assign it the /home mountpoint (even though the home subvolume was deleted) and when you chose to delete this "device", the entire volume was removed (this includes the remaining 2 subvolumes and the /dev/vda3 partition).

Comment 8 Kamil Páral 2023-04-26 10:52:04 UTC
Thanks, Vojtěch. I created bug 2189899 to track the bug in Blivet-GUI, and let's keep this bug just related to the Custom partitioning issue.

I'm reproposing this issue as a blocker against Fedora 39. It was rejected on the grounds that this is an expected deficiency in anaconda. However, from further conversation with Vojtěch, it seems clear that he was referring to the inability to reformat btrfs without *removing all subvolumes first*. And as it was showcased here, even if you do that, it's still not possible to reformat that partition, because of bugs present in the Custom partitioning, as described above. Those bugs weren't "expected deficiency" and I think it's correct to propose it as a blocker again. Not only the the partition can't be reformatted, but also a subvolume removal isn't working correctly, as shown above. Both of these seem to violate https://fedoraproject.org/wiki/Fedora_38_Final_Release_Criteria#Disk_layouts .

Comment 9 Kamil Páral 2023-04-26 10:55:12 UTC
Actually, this probably also violates https://fedoraproject.org/wiki/Fedora_38_Beta_Release_Criteria#Custom_partitioning , particularly:
> Correctly interpret, and modify as described below, any disk with a valid ms-dos or gpt disk label and partition table containing ext4 partitions, LVM and/or btrfs volumes, and/or software RAID arrays at RAID levels 0, 1 and 5 containing ext4 partitions
> Remove existing storage volumes

Comment 10 Radek Vykydal 2023-06-05 08:49:32 UTC
Hello, @kparal 
I've opened a PR that should improve the situation with btrfs subvolumes removing and adding, allowing for example reusing of /home partition on btrfs subvolume:
https://github.com/rhinstaller/anaconda/pull/4812
Would you be OK with such a solution to remove the blocking nature of the BZ?
(Or would you require support for []Reformat checkbox in Custom partitioning for btrfs devices? It can be rather complex issue.).

Comment 11 Kamil Páral 2023-06-05 15:06:36 UTC
Hi Radek. First of all, please note that this is just a proposed blocker, not accepted. If you'd like to know the blocker status asap (to plan your work), we can poke people to cast their votes soon [1].

If your patches fix subvolume removal, the F39 Beta blocker proposal would certainly be dropped. Regarding the partition reformat problem... if I understand it correctly, it would still not be possible to reformat btrfs as a fresh new btrfs, reformat btrfs as ext4, etc. That might or might not be problem, depending on how you read the release criterion, so a vote would be needed to decide. It's certainly good that Anaconda doesn't crash or anything, it just doesn't offer an option which seems logical to be present. It would be even better if it communicated it better to the user (i.e. a tooltip on that disabled reformat checkbox, guiding the user what to do instead). I've looked at the videos at your PR, and even re-creating the root subvolume is quite opaque, I must say (but that's applicable to the whole Custom part UI, sadly). So if you could communicate these better, that would be great, and the actual blocker status needs to be discussed in that blocker ticket. It would be also good to know whether Custom part will be kept in the upcoming Web UI for Anaconda, or whether a completely different UI is planned. Of course we don't want to push you to work on something that's going to be thrown away soon. In that case, some documentation e.g. in Common Issues could be more appropriate.

Thanks for looking into this well in advance!


[1] https://pagure.io/fedora-qa/blocker-review/issue/1156

Comment 12 Radek Vykydal 2023-06-06 06:50:04 UTC
Thank you for quick feedback. I agree that the process of re-creating of the root subvolume is rather non-straightforward. I was focusing on making it possible at all first. And yes, a tooltip for the disabled reformat option would be a great improvement at this point.

As for the Web UI, the Custom partitioning in this form will not be kept. Currently we are focusing on two use cases (workflows) that can be related: preserving /home partition (this will be most probably addressed a bit later), and mount point assignment (which offers reformatting of a mounted target). For both we want to handle the default Fedora Workstation configuration - btrfs - in the first place, so we will need to figure out how to "reformat" btrfs mountpoint / subvolume in this context. The UI for the cases is being designed from scratch.
Nevertheless, if we improve handling of btrfs mount points / subvolumes for WebUI, we might be able to easily use the solution also in the current Custom Partitioning UI.

Comment 13 Radek Vykydal 2023-06-07 06:53:34 UTC
(In reply to Radek Vykydal from comment #10)
> Hello, @kparal 
> I've opened a PR that should improve the situation with btrfs subvolumes
> removing and adding, allowing for example reusing of /home partition on
> btrfs subvolume:
> https://github.com/rhinstaller/anaconda/pull/4812

The patch was merged into rawhide. It should appear in anaconda 39.19.

Comment 14 Adam Williamson 2023-06-07 07:14:46 UTC
But until https://bugzilla.redhat.com/show_bug.cgi?id=2212121 is resolved, we'll have to keep untagging new anaconda versions...

Comment 15 Fedora Release Engineering 2023-08-16 07:13:21 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.

Comment 16 Adam Williamson 2023-08-23 19:47:43 UTC
This should have been fixed some time ago. Kamil, can you confirm?

Also, it has +3 in https://pagure.io/fedora-qa/blocker-review/issue/1156 , so marking accepted blocker.

Comment 17 Kamil Páral 2023-08-24 13:44:12 UTC
I tested with Fedora-Workstation-Live-x86_64-39-20230823.n.0.iso. Btrfs volumes and subvolumes can now be removed without any issues in the Custom part. That's the reason why this is currently accepted as an F39 beta blocker, and that is solved.

A btrfs partition can now be reformatted, mostly, but the process is opaque even considering Anaconda UI standards. You need to remove the / subvolume, but it mustn't be the last subvolume that exists (otherwise the whole partition disappears, so you keep your /home subvolume for now). Then create a new / mount point, it automatically is placed as a btrfs subvolume. Then you can finally remove or reformat the non-root subvolumes. So with this approach, if you want to stay on btrfs, you'll have a fresh new root, but can keep e.g. /home. You are not able to reformat the whole btrfs though, only manipulate subvolumes.
If you want to change the partition to a different filesystem, it's possible to use the same approach, only as the very last step, keep only the new / subvolume (remove all the rest), and then change the partition type to a standard partition with e.g. ext4.
You can get some idea from this video: https://rvykydal.fedorapeople.org/btrfs/rhbz2186158/reuse_home_by_removing_and_adding_btrfs_root.webm

So all use cases are mostly available now, even though some of them are very confusing and complicated. That's not likely a blocker, though. Also considering that Custom partitioning is getting phased out. Closing as mostly fixed.


Note You need to log in before you can comment on or make changes to this bug.