Bug 2263964 - Cockpit storage: cannot "Remove a planned storage volume from the planned layout" as required in the release criteria
Summary: Cockpit storage: cannot "Remove a planned storage volume from the planned lay...
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: distribution
Version: 42
Hardware: All
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Aoife Moloney
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: AnacondaWebUITracker
TreeView+ depends on / blocked
 
Reported: 2024-02-13 01:29 UTC by Adam Williamson
Modified: 2025-09-19 04:18 UTC (History)
17 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2024-02-15 17:49:32 UTC
Type: Bug
Embargoed:
blc: mirror-


Attachments (Terms of Use)

Description Adam Williamson 2024-02-13 01:29:12 UTC
This was discussed in the blocker review meeting this morning, but in reviewing the criteria, I realized it is actually a violation of the release criteria as written. We have this in the Beta criteria:

"When using both the installer-native and the blivet-gui-based custom partitioning flow, the installer must be able to: 
...
Remove a planned storage volume from the planned layout"

I have now edited the preamble there to read "When using both the installer-native and the blivet-gui-based custom partitioning flow on the GTK-based installer, and the Cockpit-based custom partitioning flow on the webui-based installer, the installer must be able to:", to accommodate the cockpit-based flow. But of course, the cockpit-based flow does *not* satisfy this requirement. As we talked about at the meeting, it is based on udisks, and applies any changes immediately. It does not have the "plan then apply" workflow that blivet provides.

It did not feel right to simply unilaterally edit the criteria to not require the cockpit-based flow to meet this requirement; there already seemed to be a strong feeling that we should consider whether an "immediate apply" model is acceptable for the installer workflow, and this criterion provides a good peg to hang that discussion on. So I'm filing this bug and proposing it as a blocker.

Filing against distribution rather than anaconda or cockpit because this is kind of "as designed", AIUI, so this is more of a "do we find this design acceptable?" question than a "there is a bug" situation.

Comment 1 Neal Gompa 2024-02-13 13:23:56 UTC
It is seriously concerning that Cockpit storage applies changes immediately. None of the desktop counterparts (KDE Partition Manager, Blivet-GUI, GParted, etc.) do that.

I do not find this design choice acceptable. In fact, I consider it seriously dangerous.

Comment 2 Stephen Gallagher 2024-02-13 13:34:23 UTC
(In reply to Neal Gompa from comment #1)
> It is seriously concerning that Cockpit storage applies changes immediately.
> None of the desktop counterparts (KDE Partition Manager, Blivet-GUI,
> GParted, etc.) do that.

To be fair, anything using the udisks interface (such as gnome-disks) DOES behave that way.


> I do not find this design choice acceptable. In fact, I consider it
> seriously dangerous.

I agree that this is dangerous for the installer; catastrophic data-loss is a simple mis-click away for someone attempting a dual-boot setup.

Comment 3 Brandon Nielsen 2024-02-13 16:21:57 UTC
I think Cockpit definitely needs a "plan then apply" workflow and the criteria should stand as given in the description in this bug.

The current immediate apply workflow makes it more likely for accidental data loss to occur either because the user made a mistake, or because a UI issue / unexpected UI behavior resulted in the user doing something they didn't intend. I also feel one GUI installer behaving different than the others increases the likelihood of user error.

In defense of the new Cockpit installer, it does warn both at the top of the custom partitioning screen, and with every action you pick, that the action cannot be reversed.

Comment 4 Adam Williamson 2024-02-13 16:25:12 UTC
To Stephen's point, it's worth emphasizing that with the current design, this "instant-apply" UI is the *only* UI offered for doing a classic "resize a Windows install and install alongside it" operation, in the new webui workflow. The "guided partitioning" workflow that exists alongside the "custom partitioning" workflows in gtkui no longer exists as a separate thing; if you need to free up space, this is the only in-line UI we offer to do it.

Comment 5 Zbigniew Jędrzejewski-Szmek 2024-02-13 17:14:17 UTC
(In reply to Neal Gompa from comment #1)
> I do not find this design choice acceptable. In fact, I consider it
> seriously dangerous.

Sadly, I very much agree.

Apart from the danger of data loss, this approach is harder to use,
in particular for less advanced users. The ability to plan the entire
set of disk layout changes in advance in your head is hard. It is much
much easier to instead "play" with the layout, try things and simply
"see" if everything that you want fits in there, and then click "proceed"
after looking at the final version for a minute or two.

Also, the planning is "instantaneous", but the execution may take a while
with some fs and disk types. When actions are applied immediately,
the workflow becomes to do some important choices interspersed with
unpredictably long pauses. This makes it easier to make mistakes.

The solution with a single execution step at the end is simply much better
UX.

Comment 6 Adam Williamson 2024-02-13 17:50:22 UTC
Doing a quick survey of other distros:

* Ubuntu (23.04) has an in-line custom partitioner (UI quite similar to Cockpit) which uses a plan-then-apply model
* SUSE (OpenSUSE Tumbleweed) has an in-line custom partitioner (UI quite similar to blivet-gui or gparted) which uses a plan-then-apply model
* Mint (21.3) has an in-line custom partitioner (looks like the Ubuntu one) which uses a plan-then-apply model
* Arch (2024.02.01)...has fdisk, mkfs and mount, and tells you how to use them! Never change, Arch: https://wiki.archlinux.org/title/installation_guide#Partition_the_disks

Comment 7 Jiri Konecny 2024-02-13 20:14:18 UTC
Definitely a good points here. However, I would like to mention that the design of the web UI is that users should use Guided solution for most of the tasks which means that this tool should be used only by advanced users who are not able to use guided solution for some reason. If you take this into consideration than I would IMHO say that "this approach is harder to use, in particular for less advanced users" is not correct.

I wonder if it would be satisfying if we make this even more obvious that this should be used only when you really know what you are doing. My personal feeling is that we are still aligned (even though it's questionable) with the original change https://fedoraproject.org/wiki/Changes/AnacondaWebUIforFedoraWorkstation#Guided_partitioning the only difference is that we give user an easy way to use one of the possible tooling easily accessible. And yes, I know that could be a big difference. My point is if we can change the logic to make this obvious that we gave you a possibility nothing more...

Comment 8 Adam Williamson 2024-02-13 21:54:12 UTC
The main concern I have with that is the one stated above: the current webui implementation provides no other way to resize or even selectively delete existing partitions, which is a pretty common task. By offering only the cockpit approach for this, we encourage people down that path, which means it is going to be used rather more than it would be otherwise.

In gtkui, the 'guided' path offers a relatively easy/safe UI for doing resizes and selective deletions (the "free up space" UI) which is distinct from either of the "custom partitioning" UIs.

Comment 9 Jiri Konecny 2024-02-15 00:01:31 UTC
I would like to clarify reasoning behind this change which is already partially covered at comment 7.

Origin proposal as mentioned before was to use external tool. However, IIRC after starting integration work with GIS we have found that people don't have access to the given tooling in limited environment so we added an easier way to reach this tool - blivet-gui. However, this tool wasn't the final variant because, aside other reasons, it won't work for remote installations.

For that reason we were always looking for Cockpit storage because it's the best tool for adoption (and the only one to my knowledge). It's based on the same technology and has active team willing to help. From the begging the plan was to only make easier access to the storage tool and more accessible but never it was intended as the main installation method which should be used for partitioning. For that reason it's hidden behind a button and there are a lot of warnings everywhere. We rejected the idea to have this tool as part of the wizard workflow.

I would also like to make clear one thing. Anaconda development team is not looking for creating our own solution for partitioning as we had GTK UI custom partitioning. Reasons are simple:
1. extremely hard to maintain
2. we don't have capacity to reasonably create it

For that reason our approach is to leave this hard part to specialized tool. Instead of that we want to focus on the smart solution which (based on our data we had collected) is used by majority of users. We would like to add new workflows to our Guided partitioning based on the feedback to cover what users really want.



Based on what is written above I would like to change this discussion to how we can improve the current solution to make it acceptable? We are probably not able to change behavior to planning actions, however, as I said this tool was always meant for advanced users only. Also, we still support to choose tools of their choice outside of Anaconda if they don't want to use Cockpit Storage. What we can do to make the intended workflow more obvious any suggestions?

Comment 10 Jiri Konecny 2024-02-15 00:03:40 UTC
(In reply to Stephen Gallagher from comment #2)
> (In reply to Neal Gompa from comment #1)
> > It is seriously concerning that Cockpit storage applies changes immediately.
> > None of the desktop counterparts (KDE Partition Manager, Blivet-GUI,
> > GParted, etc.) do that.
> 
> To be fair, anything using the udisks interface (such as gnome-disks) DOES
> behave that way.
> 
> 
> > I do not find this design choice acceptable. In fact, I consider it
> > seriously dangerous.
> 
> I agree that this is dangerous for the installer; catastrophic data-loss is
> a simple mis-click away for someone attempting a dual-boot setup.

To my knowledge should be always confirmation for any destructive action. I don't think mis-click is probable to happen.

Comment 11 Neal Gompa 2024-02-15 07:17:33 UTC
Then the solution here is to have Cockpit Storage change to match the workflow of everything else. "Apply immediately" is a very bad workflow.

Comment 12 Matej Marušák 2024-02-15 17:49:32 UTC
> Then the solution here is to have Cockpit Storage change to match the workflow of everything else. "Apply immediately" is a very bad workflow.

Cockpit can not and will not change to have "apply later" workflow. It is not just switch that can happen it is the whole concept and technology stack that is behind it. To have such solution a completely new plugin would have to be created based on some different backend solutions. So there is no-op from cockpit side - closing.

My 2cents to the above discussion:
Cockpit storage is not dangerous tool where `catastrophic data-loss is a simple mis-click away`. There is always blocking alert where user has to confirm any time they want to do any destructive operation. Cockpit storage is very well established tool and if it would be that simple to destroy your data we would probably hear about it before :) 

All the talk about blivet I was certain that the changes users do in this tool are never committed and only once user clicks "install" button storage starts to change. However that is not the case. You have to commit to (possibly breaking) changes right after you return back to installer. So not dramatically later than doing it with "immediate changes workflow". Btw. I checked Anaconda today and there is big warning on top saying "Changes made here will immediately affect the system. There is no 'undo'."

Surely tool like blivet is better to "play around" for users that don't know what they want to do at the beginning and are just trying different combinations since it is easy to just reset all the changes. However users can undo and redo and play around with Cockpit as well. It is not that if user assigns something there is no way back (unless of course they format their data). Admittedly I have no idea about Anaconda UI userbase and what kind of users use this tool but to my understanding "Modify storage" in Anaconda is for "advanced users" where some knowledge is expected and recommended. For someone who is ready to twice confirm "Yes, format my data" and then be surprised they lost their data there is not tool to protect them.

Comment 13 Adam Williamson 2024-02-15 19:48:32 UTC
Please don't unilaterally close proposed blocker bugs as NOTABUG.

Comment 14 Adam Williamson 2024-02-15 19:49:08 UTC
I intentionally filed this on distribution, not cockpit, because it is kinda philosophically broader than just a claim that this is a concrete "bug" in cockpit.

Comment 15 Adam Williamson 2024-02-15 19:59:38 UTC
> All the talk about blivet I was certain that the changes users do in this tool are never committed and only once user clicks "install" button storage starts to change. However that is not the case. You have to commit to (possibly breaking) changes right after you return back to installer.

In the gtkui+blivet workflow, it works as you describe. Changes are not actually committed to disk until you hit the Install button. The webui+blivet workflow changed quite a lot for the short time it existed and was never really finalized, so it's hard to make definite arguments or comparisons about it.

> Admittedly I have no idea about Anaconda UI userbase and what kind of users use this tool but to my understanding "Modify storage" in Anaconda is for "advanced users" where some knowledge is expected and recommended.

The broader issue is that we really have no way to claim that. It is the way the developers seem to want this to be seen, but it is not necessarily how it will *be* seen. You could claim this is already the case for gtkui - the "default" workflow is guided partitioning, after all, and you have to click a "scary" button marked "Custom" or "Advanced Custom" to access either of the custom workflows. But in practice, it seems to be important. People click on those buttons, and have Strong Opinions (usually not positive) about what they see when they do. This has definitely been a significant factor in determining people's opinions on gtkui.

Note also that, in gtkui, the "guided" workflow allows you to selectively delete and resize partitions (resizing Windows installs is apparently a bit murky these days due to Bitlocker, but the capability is technically there, and it can certainly be used on existing *Linux* installs). In current webui, there is no way to do this besides through whatever the current "custom" flow is. The only "non custom" options are "install to existing free space", "delete one or more entire disks", or "assign mount points to existing storage volumes, and optionally reformat them". To do *anything* other than that, the only offered operation is to use the "custom" flow (i.e. cockpit, currently). I have seen a design board in which even the "assign mount points to existing storage volumes" UI is subsumed into the cockpit-storage UI, which actually makes a deal of sense in a way (since cockpit-storage can assign mount points to existing storage volumes, why also have a different UI for doing that?), but further extends the 'reach' of this flow.

Comment 16 Matej Marušák 2024-02-16 08:49:02 UTC
> Please don't unilaterally close proposed blocker bugs as NOTABUG.
> I intentionally filed this on distribution, not cockpit, because it is kinda philosophically broader than just a claim that this is a concrete "bug" in cockpit.

Indeed, it is philosophically so not sure why it was thrown over to Cockpit. Since it was "not a bug" and I did not want to throw the hot potato around I did the right thing from Cockpit point of view. Makes sense to reopen if you want to continue to polemicize. At this moment it is just argument about how it used to be and whether it should always stay the same. If there would be specific action item where Cockpit can help (and we are keen if there are actionable items!) with please CC me back, thanks!

Comment 17 Jan Stodola 2024-02-16 09:49:03 UTC
Since the partitions are created immediately by cockpit-storage, it is now also important in what order are the partitions created by the user. If you spend time on creating for example an LVM setup with encryption and find out you forgot to create the BIOS boot partition, you have to remove everything and start over with the BIOS boot partition created first. I hit this problem in bug 2264517.

Comment 18 Jiri Konecny 2024-02-16 12:01:13 UTC
(In reply to Adam Williamson from comment #6)
> Doing a quick survey of other distros:
> 
> * Ubuntu (23.04) has an in-line custom partitioner (UI quite similar to
> Cockpit) which uses a plan-then-apply model
> * SUSE (OpenSUSE Tumbleweed) has an in-line custom partitioner (UI quite
> similar to blivet-gui or gparted) which uses a plan-then-apply model
> * Mint (21.3) has an in-line custom partitioner (looks like the Ubuntu one)
> which uses a plan-then-apply model
> * Arch (2024.02.01)...has fdisk, mkfs and mount, and tells you how to use
> them! Never change, Arch:
> https://wiki.archlinux.org/title/installation_guide#Partition_the_disks

It's great to know what other distributions are using, however, I don't think that should be important for decision here. I don't think Fedora wants to base decision on others because if we do that we would end up of blocking all the innovations or new ideas (or it is fine to do the installations in command line for us :D :D).

On the same note, this whole bug is a bit absurd to me. These release criteria were created to not regress between the Anaconda versions and as guidelines for testers to know what needs to be working. They shouldn't be used as a blocker for workflow change or innovations. For example SilverBlue never can adapt to these to be installed from Network installation image because it's breaking release criteria about:

    When installing with the generic network install image, interactively selecting a package set other than the default must work.

Does that mean that Fedora SilverBlue never can be officially supported by Anaconda? What I want to show here is that with a change of workflow (as proposed in the current/old change) we can't use the old criteria, that is just not possible or again we stick with blocking any innovation.


I thought that Fedora is embracing innovations and is not worried about testing new ideas, however, this bug is exactly about opposite, so I'm not that sure with discussion here.

I honestly agree with closing this bug as NOTABUG because we are not discussing here blocker for users (it's not blocking them to finish the installation in any way) but about personal preference and as explained above the whole bug is based on a wrong reason.

Comment 19 Jiri Konecny 2024-02-16 12:11:10 UTC
(In reply to Adam Williamson from comment #15)
> > All the talk about blivet I was certain that the changes users do in this tool are never committed and only once user clicks "install" button storage starts to change. However that is not the case. You have to commit to (possibly breaking) changes right after you return back to installer.
> 
> In the gtkui+blivet workflow, it works as you describe. Changes are not
> actually committed to disk until you hit the Install button. The
> webui+blivet workflow changed quite a lot for the short time it existed and
> was never really finalized, so it's hard to make definite arguments or
> comparisons about it.

It was never a plan to support full integration with blivet-gui in Anaconda web UI, at least to my knowledge.
 
> > Admittedly I have no idea about Anaconda UI userbase and what kind of users use this tool but to my understanding "Modify storage" in Anaconda is for "advanced users" where some knowledge is expected and recommended.
> 
> The broader issue is that we really have no way to claim that. It is the way
> the developers seem to want this to be seen, but it is not necessarily how
> it will *be* seen. You could claim this is already the case for gtkui - the
> "default" workflow is guided partitioning, after all, and you have to click
> a "scary" button marked "Custom" or "Advanced Custom" to access either of
> the custom workflows. But in practice, it seems to be important. People
> click on those buttons, and have Strong Opinions (usually not positive)
> about what they see when they do. This has definitely been a significant
> factor in determining people's opinions on gtkui.

We never ever can make everyone happy with the partitioning. That is just not possible because it's too much about personal preference. (Which is another reason why I'm unhappy about this whole bug). 

For that reason Anaconda don't want to try, we want to go opposite way and that is to tell users if you are not happy about what we provide you, feel free to do the partitioning beforehand with any tool of your choice. The Cockpit storage solution is just an option which we are able to integrate reasonably but it's not blocking users from using other tool if they prefer to.

> 
> Note also that, in gtkui, the "guided" workflow allows you to selectively
> delete and resize partitions (resizing Windows installs is apparently a bit
> murky these days due to Bitlocker, but the capability is technically there,
> and it can certainly be used on existing *Linux* installs). 

Based on what I'm hearing from the storage experts it's most probable that soon Linux won't be able to resize default Windows installations for a long time. More precisely it seems that bitlocker won't work on Linux anytime soon.

> In current
> webui, there is no way to do this besides through whatever the current
> "custom" flow is. The only "non custom" options are "install to existing
> free space", "delete one or more entire disks", or "assign mount points to
> existing storage volumes, and optionally reformat them". To do *anything*
> other than that, the only offered operation is to use the "custom" flow
> (i.e. cockpit, currently). I have seen a design board in which even the
> "assign mount points to existing storage volumes" UI is subsumed into the
> cockpit-storage UI, which actually makes a deal of sense in a way (since
> cockpit-storage can assign mount points to existing storage volumes, why
> also have a different UI for doing that?), but further extends the 'reach'
> of this flow.

As mentioned above, mount point assignment will be supported and we want to use this solution to give users an option to do the partitioning in a way they prefer to. So, the Cockpit storage will get the integration but it won't block users from using anything else.

Comment 20 Adam Williamson 2024-02-16 17:08:22 UTC
> I don't think Fedora wants to base decision on others because if we do that we would end up of blocking all the innovations or new ideas

I don't think we should base decisions on others, but we *should* be aware of how other distributions behave and how our behaviour compares to them, because *other people will make that comparison* in evaluating our distribution. This is the lesson we learned from gtkui: it is harder to convince people of the value and quality of a tool that doesn't behave like all the other tools in its class, the way people are expecting. It's possible, but you have to be *really good*. Something that's probably about as good as the alternatives, but that works differently, is going to cause friction. Even if you decide to go with it anyway, it is valuable to be prepared for the feedback you're going to get.

> These release criteria were created to not regress between the Anaconda versions and as guidelines for testers to know what needs to be working.

As someone who's been involved with the criteria from the start, I disagree. The intent of the criteria has always been to try and capture the functional requirements we think should be met by a Fedora release. When it comes to something like OS installation, there *is* a tension there, because it's very hard to phrase absolutely everything in sort of 'generic' terms, as if we had no idea how the tool that currently implements them behaves, so there does tend be an inevitable degree of 'capture' where criteria start being more tied to specific tool functionality, but it's not the *intent* of the criteria and we do try to keep a lid on it.

> They shouldn't be used as a blocker for workflow change or innovations.

I agree, and if you check, you will see we have frequently changed the criteria over time. Each criterion has a "References" footer which gives a potted history of these changes, in fact. But equally, we shouldn't just say "well, any time a developer wants to do something that breaks the criteria, we should just go ahead and change them". There has to be a balance there. This bug report is intended as a mechanism for having a necessary discussion about the situation. Not every bug that's *proposed* as a blocker is *accepted* as one.

> Does that mean that Fedora SilverBlue never can be officially supported by Anaconda?

No. For a start, that's exactly why the criterion starts "When installing with the generic network install image" - that wording makes it clear that the criterion applies only to "the generic network install image", i.e. Everything-boot.iso. So it's irrelevant to Silverblue. Again, if you go through the history of the criteria, you will see several waves where we adjusted them to significant changes (there was a significant revision for IoT becoming a blocking edition, for instance; ditto for FCOS). But those were all discussed and communicated, and there *were* cases where we said "no, for this criterion, it makes sense to make the new product comply with the criterion, not change the criterion to fit the product".

> I thought that Fedora is embracing innovations and is not worried about testing new ideas, however, this bug is exactly about opposite, so I'm not that sure with discussion here.

Again, I think this is just a question of balance and being intentional as a whole project. I disagree that this bug is "exactly about opposite". Embracing innovations and testing new ideas is great, but we do need to exercise some kind of oversight and quality control on them, that is what we have these processes for. As I wrote above, the point of this bug is to have the discussion and to provide an opportunity for the wider Fedora community to be aware of the changes and to provide input and feedback. What was the alternative? I just go ahead and change the release criteria, and drop a mail to test@ saying I'd done it? I don't believe that would have been a good process.

Comment 21 Adam Williamson 2024-02-16 17:26:31 UTC
> For that reason Anaconda don't want to try, we want to go opposite way and that is to tell users if you are not happy about what we provide you, feel free to do the partitioning beforehand with any tool of your choice. The Cockpit storage solution is just an option which we are able to integrate reasonably but it's not blocking users from using other tool if they prefer to.

As I said, I understand that this is how you're looking at it, but I am concerned that it's not going to be how the people who use the installer see it. My experience with this suggests to me that if you put the thing in, people will take it as a thing they're meant to do, and consider it an important part of the tool. This was our experience with the custom flows in gtkui. This was our experience with things like adding btrfs support - remember how the old anaconda team had a predilection for hiding experimental features behind magic, lightly-documented kernel cmdline parameters? That's kinda the level of obfuscation you need to make people *not* think of the feature as just another, equally-important bit of the installer.

Again as a personal opinion, at least in *theory* I liked the original design that anaconda had no custom partitioning mode, only assign partitions. *That* design indeed makes the intent you are trying to communicate clear. I understand that it became difficult to provide a viable workflow for that given the way the integration with gnome-initial-setup was designed. I also think that, sadly, things like EFI system partitions and biosboot partitions mean it might fundamentally not be a viable approach: even people who *think* they're partitioning experts probably will often not know or remember to put those in, and the experience of forgetting such a 'required partition' when the partitioning tool is not part of the installer is inevitably going to be extremely clunky and awful.

But - referring to our chat discussion yesterday - I'm worried that this is an instance of the danger I see in kind of planning things on the fly. Yes, if you're thinking about the problem "we can't find a good way to give the user an opportunity to launch their own partitioning tool then come back to our installer", then "OK, so let's put a partitioning tool in the installer" is a logical response. But now you've suddenly got an installer that's quite different from your original idea - you've got an installer with a custom partitioning workflow - but you're still sort of thinking about it in the initial way - "the partitioning isn't really our problem, it's someone else's". That's what I'm seeing in the line of response that goes "oh well, most people aren't going to use this custom partitioning thing anyway, we hope".

Also, it seems like every revision to webui at present is kind of rowing in the *opposite* direction from communicating to the user "custom partitioning is a kinda weird path! you probably shouldn't use it!" First, we had a button to launch blivet-gui as an explicitly separate tool. Then we got a kinda more integrated mode of blivet-gui. Then we got cockpit-storage, still launched from a different-looking button, but also very integrated into the UI. The next revision (from reading the PR) seems like it will make blivet-gui fully responsible for assigning mount points, if you use it, and send you from blivet-gui straight to the rest of the installer - which is definitely a design improvement, but *also* is moving away from communicating "this is a separate thing, not part of the installer". And then I saw a design board which looked like it dropped the "Assign mount points" item in the "three choices" on the installer front page and replaced it with an option that would launch cockpit-storage - which *again* I think is actually a UI improvement from many perspectives, but *at the same time* is again making the custom flow seem both more important (it would then be on an equal footing with the current two 'guided' choices) and more integrated.

Comment 22 Adam Williamson 2024-02-16 17:59:17 UTC
BTW, I think the comparison to FCOS and IoT is kinda illustrative. In those cases, we worked through this stuff several weeks or even months before Beta. We had advance warning of like a release cycle that there was a plan to make them release-blocking Editions. There was a Change in each case, and as part of the Change scope, we had tickets for updating the release criteria and test matrices, which we worked on together with the IoT and FCOS teams well in advance of release: https://pagure.io/fedora-qa/issue/622 , https://pagure.io/fedora-qa/issue/623 , https://github.com/coreos/fedora-coreos-tracker/issues/1239 etc. All of that stuff was prepared way before we reached branching for the release where the editions actually became release-blocking, and we had builds of FCOS and IoT to test and write the criteria against, whose behaviour did not subsequently change significantly.

We actually got off to a good start with this for webui! There was a Change, and we talked about this stuff a long way ahead. There is an old ticket - https://pagure.io/fedora-qa/issue/740 - from such a planning round. But...as you can see, that ticket says "The new anaconda web UI is planning to drop custom partitioning 🥳 . This will require revisions to the release criteria, as there are specific "custom partitioning" requirements that, as currently written, apply to all installer images." So *that* was the change we were planning at the time.

And then the scope started shifting. First we got blivet-gui in the installer, which was not really very concerning from a criteria or test case perspective, because it was a known quantity. We already have an installer that can launch blivet-gui, after all, so we already have release criteria and test cases that cover such a design. I figured all we'd maybe need to do is some very minor tweaks.

But then the cockpit-storage change landed, very late, and not communicated as part of the process at all. I know we've discussed this, and it wasn't intentional, and I don't want to keep hitting you with it - but it *is* significant in a lot of contexts, and this is one of them. If we'd known cockpit-storage was landing three months ago, and maybe had an image to see how that looked, we would have had time to have these discussions in a much more leisurely and convivial way - tickets on the tracker, casual chats on Matrix, and so on. With an image, at least, we would have noticed this 'no more plan-then-apply' change, and we could maybe have informally sounded FESCo out on it, had some time to think it through and work it out. This would not have had to be a big panic emergency drill. It's feeling like a big panic emergency drill *because* this change is, unfortunately and unintentionally, landing late and surprising us.

Comment 23 Martin Pitt 2024-02-19 03:02:49 UTC
Some background information: This was crystal clear from the very start, when moving away from GTK Anaconda to a web-based installer was discussed and decided in fall 2021 (FTR, I was not an active part of that discussion/decision). There are three choices:

 1. Keep Anaconda GTK with the blivet based "make a plan and execute" partitioner
 2. Cockpit team throws a few person-months of work on the "immediate exec" udisks based cockpit-storage page (Anaconda mode, btrfs support, etc.), and Anaconda team uses that as the installer storage backend. Most of this has happened now.
 3. Anaconda team develops an entirely new blivet-based partitioning Cockpit page. (This would take significantly more resources)

It has always been clear that cockpit-storage is and will remain udisks based (and hence "insta-apply"), and can and will not turn into 3 -- that would become an entirely new project.

From what I saw as the outcome of the discussions, Fedora/RH has decided that option 3 is too expensive, and doesn't want 1. any more. Also, the Anaconda team has even done a public user survey [1]. I assume that has factored into the decision as well. But purely practically, given how little dev capacity we had/have for all of this, the only realistic option is 2. now. Of course RH is free to decide that it wants to do 1. or 3. after all, but TBH I don't see that happening.

> If we'd known cockpit-storage was landing three months ago

... and that surprises me. This was apparently a great miscommunication :-(

[1] https://fedoramagazine.org/anaconda-installer-partitioning-and-storage-survey-results/

Comment 25 Zbigniew Jędrzejewski-Szmek 2024-02-19 12:12:47 UTC
> https://fedoramagazine.org/anaconda-installer-partitioning-and-storage-survey-results

Hmm, this is very interesting. Q14 is the most interesting:
"How do you prefer to create partitions during the installation process?"
and the two answers that would map to the "advanced workflow" that we are discussing
here are "I modify the partitions proposed by the installer" 27%, and
"I create the partitions manually via the installer tooling" 26%.
To that's 53% of responses in this survey!

(Note that the plot has an annotation that 70% use "Auto partitioning", but out of those,
27 p.p. modify the automatic proposal, so they fall into the "advanced" category.
Also note that the responses in this survey seem to be biased towards power users,
with mean experience of 11 years. I guess we have more less experienced users in reality.)

Comment 26 Adam Williamson 2024-02-19 16:09:56 UTC
Martin: well, there definitely seems to be a lot of confusion. There was certainly a fourth choice: have no full-on "custom partitioning" interface in the installer at all, and require the user to do any custom partitioning outside of the installer workflow, using a tool of their choice. This was the official plan as of the time the Change page for Fedora was written:

"Because of those issues, we decided to choose another approach, which we are calling guided partitioning. This type of partitioning is giving users paths with explanations of what will happen but does not overload them with too many options at once. These paths could be then customized. ... We will provide the recommended solution and improved customization based on the users feedback. However, in case someone is not happy about the recommended solution, we are going to provide a way to guide users, to create their partitioning themselves (with a tool of their choice) and then tell Anaconda how to use it."

https://fedoraproject.org/wiki/Changes/AnacondaWebUIforFedoraWorkstation#Guided_partitioning

That page was written in June 2023, a long time after the survey and Fall 2021 date you cite. That was the plan the rest of the Fedora project was aware of. As far as I know, nobody in Fedora outside of the installer and cockpit teams was aware of the cockpit-storage plan until about two weeks ago, when it was casually mentioned to me on a chat channel (though I kind of didn't initially realize its significance at the time).

I guess another aspect of this is that the "guided partitioning" approach described there hasn't really come to fruition either. That sounds rather like what the current SUSE installer does - it actually analyzes your disk and provides a suggestion of what to do. For instance, when I tested it on a disk containing a Fedora install, it successfully suggested an approach of replacing Fedora with SUSE, re-using the existing partitions (I was quite impressed). However, none of that really seems to have made it to the current webui implementation; as noted, you can only use installer auto-partitioning with existing free space or by wiping entire disks, or re-assign existing partitions manually (without much 'guidance' beyond the "these are the required partitions" rules).

I am aware of the Red Hat background here. As the point of this bug is for public understanding and discussion of these changes and their consequences, I think it'd be really good to avoid private comments on it. Non-RHers are aware private comments exist, because the numbering becomes non-sequential - they can see a comment #23 and a comment #25, so they know there is a comment #24 they cannot see. This tends to engender mistrust and it's really good if we can avoid it.

Comment 27 Jiri Konecny 2024-02-19 16:48:32 UTC
I understand your point Adam, it came late. What I don't like about this bug it is not saying it's coming late, but it's saying NO to Cockpit storage which I said is subjective based on my opinion. For me, it feel like it is creating a promise for future that immediate actions are just not something what will be accepted to Fedora (also based on replies here - not your replies Adam). And yes, I understand your point that criteria could be changed but in that case I really don't understand this bug as it is filed currently.

Comment 28 Adam Williamson 2024-02-19 17:06:20 UTC
What the bug is meant to "say" is: as things stand, we have a violation of the release criteria, so this is proposed as a release blocker. There are various paths we can take from there. But this close to Beta's release, it really needs to be tracked as a bug. If we had done this a few months ago, that would not have been necessary.

Comment 29 Adam Williamson 2024-02-19 22:56:45 UTC
FESCo today voted to defer the entire webUI Change to Fedora 41: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/4CYBUJ7BFBV7ZV5AICKTHIZGKWO3X4FI/ . I'm thus removing blocker status from this bug. Technically it's still a blocker for F41 Beta, but it really doesn't make sense to track this as "release-blocking bug" over such a timeframe. There's now time for folks to have a more relaxed conversation about what the hell we're doing.

Comment 30 Chris Murphy 2024-02-20 05:17:52 UTC
The relationship between this bug and the late change, is that the lateness delayed the necessary conversation and decision whether to change the release criterion violated by this bug. And that discussion would have included the "planned" vs "immediate" partition execution tradeoffs.

Ordinarily the need to remove/modify/add release criterion would have been exposed before FESCo approved the change. But I don't see any indication in the change proposal that this could have been realized until cockpit-storage landed two weeks ago. I also don't see release criterion mentioned in the change proposal process docs or template. But any change that intersects a release criterion is something that needs to be addressed.

https://fedoraproject.org/wiki/Basic_Release_Criteria
https://fedoraproject.org/wiki/Fedora_40_Beta_Release_Criteria
https://fedoraproject.org/wiki/Fedora_40_Final_Release_Criteria

The change proposal process docs don't define what constitutes a material change in an approved proposal that should result in immediately informing FESCo. This situation doesn't often happen, is probably why. The motivation section convincingly tells us that proposers aren't expected to know or predict everything, Fedora development is complex. Full disclosure of the change being broadcast helps give all of Fedora's stakeholders the opportunity to point out previously undiscovered liabilities or fallout from a change. The inclusion of cockpit-storage UI into the installer seems like a significant/material change in the proposal to me. But how can we unambiguously define it, rather than it being opinionated?

The change proposal process docs also don't say where or when proposers should inform of material changes to their proposal, or whether FESCo or working groups should investigate change processes more often. I also wonder if this situation is a consequence of not having a Fedora Program Manager to more frequently poke for status updates on changes?

Anyway, I'd say there's no one single problem that got us here, and I'm not sure how to help avoid this from happening in the future.

Comment 31 Aoife Moloney 2025-02-26 12:58:00 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 42 development cycle.
Changing version to 42.


Note You need to log in before you can comment on or make changes to this bug.