Bug 860677

Summary: installer lacks support for creating new devices from existing containers
Product: [Fedora] Fedora Reporter: Eric Hopper <eric-bugs2>
Component: anacondaAssignee: David Lehman <dlehman>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: awilliam, bugzilla, clydekunkel7734, duffy, g.kaviyarasu, ipilcher, jonathan, joshua, vanmeeuwen+fedora
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-01-04 19:10:12 EST Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:

Description Eric Hopper 2012-09-26 09:06:28 EDT
I have a partitioning scheme in which I have one big btrfs filesystem with subvolumes for /, /var and /home. With the anaconda gui in its current state, I can't figure out how to make it create a new subvolume for the new /root and /var for Fedora 18, and then subsequently install into those subvolumes.

I also can't see how I would use LVM in any reasonable way to do the same thing (which is what I did before btrfs) with the current GUI.

What I'm going to have to do is install the whole entire system into a new partition and then manually move the installed system myself into the appropriate locations and fix up /etc/fstab and the grub2 configuration myself.

While the new anaconda is very nice in many respects, and a big step forward, the way it handles storage in the GUI is quite broken.

This brokenness prevented me from being able to participate in any of the test days because installing Fedora went from a couple of hour process to a 6-8 hour process.
Comment 1 Jesse Keating 2012-09-26 11:59:51 EDT
The BTRFS stuff isn't ready yet, hopefully by beta.
Comment 2 Eric Hopper 2012-09-26 13:43:14 EDT
I can't even figure out how to use the new GUI to do something like create an LVM volume group from an old partition that currently contains and ext4 filesystem with an old install on it. It seems very bent on trying to use one of the layouts I currently have to install into. But even then it's not clear exactly what will be installed where.

The old gui was kind of clunky, but quite serviceable for most things. The only thing I couldn't do with it was install into a btrfs subvolume or create an encrypted swap partition that will be reformatted and re-keyed with a random key on boot.
Comment 3 David Lehman 2012-09-26 15:21:40 EDT
(In reply to comment #2)
> I can't even figure out how to use the new GUI to do something like create
> an LVM volume group from an old partition that currently contains and ext4
> filesystem with an old install on it. 

What you want to do in this case is _remove_ that old partition with the ext4 filesystem on it. Then you can create a new device and, if you like, change its device type LVM.

The new custom partitioning interface defines devices based on what you actually want instead of making you build things up from disk to partition to lvm &c.

If you no longer want a partition/device, remove it by selecting it and clicking the minus button. If you want a new device, click the plus button. It will create a device of the default type, which you can change by clicking Customize and then choosing a new entry from the Device Type list. This is a far cleaner and simpler approach -- it's just going to take some getting used to for some people.


We will add support for using existing devices very soon, but support for creating new devices from existing containers will take a bit longer. If that's your chosen method, sorry -- you have chosen a method that requires some of the most advanced possible functionality from an OS installer. Your use-case was deemed both significantly less likely and significantly more difficult to handle than others.
Comment 4 Eric Hopper 2012-09-27 09:33:37 EDT
(In reply to comment #3)
> We will add support for using existing devices very soon, but support for
> creating new devices from existing containers will take a bit longer. If
> that's your chosen method, sorry -- you have chosen a method that requires
> some of the most advanced possible functionality from an OS installer. Your
> use-case was deemed both significantly less likely and significantly more
> difficult to handle than others.

It's how I've set up every RedHat and Fedora system I've installed since LVM was available on the install screen. My modus operandi has been to create new filesystems for the new /, /var, (and sometimes /var/log) partitions from the LVM volume group holding everything, use the existing swap and /tmp partitions (though recently I've taken to greatly increasing the size of swap and using tmpfs for /tmp), and then point at the existing /home partition and specify that it not be formatted.

But, I have no idea how anybody else does it. So it's quite possible that I might be highly unusual in this regard.
Comment 5 Eric Hopper 2012-09-27 13:40:48 EDT
And, one last thing, then no more complaining by me in this bug.

I tried this one more time...  There is absolutely no way I can install this with the current state of Anaconda. It won't even use the existing /boot partition if I ask for a new /. I have no downgrade path. Additionally, the GUI doesn't make it clear that nothing else on my disk will be touched. Near as I can tell, if I create a new partition in free space and install into it, it will erase everything else on the disk. At least any hint that it exists completely disappears from the GUI once I ask for a new partition in the free space.

This is really distressing to me. I could just install and hope that the GUI is lying about all the other partitions disappearing. But then in order to get the thing to show the previous revision in the boot menu I have to basically create my own grub config file by hand. And grub2 is so complex I have no clue how to do that anymore. And I'd have to manually move all the stuff back into my careful and deliberate partitioning scheme after I installed, and that's providing the original partitions are left intact, which it looks like they'll all be wiped out.

Perhaps, for someone who's never installed Fedora in their lives, or only installs onto pristine fresh HDs and throws out the one for the old revision, or who blindly (and IMHO stupidly) trusts the upgrade process to not make a mistake, perhaps this current thing is just fine. But it's not for me, and it never will be. I've adapted to a lot of Fedora changes over the years and been generally very pleased, but this I just can't stomach.

It doesn't tell me devices names or partition numbers or anything. I have no clue what it's really doing or why. I feel like I'm going to destroy my whole HD just by touching the wrong button and I won't get any warning that I'm doing it.

This is the exact reason I've refused to use Ubuntu. Every time I try to install it it won't let me do what I want and insists on setting up all the partitions and everything its way. And if I try to adjust things to be my way later nothing works right anymore.

I've chosen my way because it allows me to carefully upgrade my system to new revisions while preserving the ability to boot back into the old one should it prove complex to upgrade software configurations (which I always do by hand because no automated process is going to handle that properly). My way carefully manages risk and always gives me an out should something go wrong somewhere in the process.

I feel blind, deaf, and dumb, like I'm fumbling in the dark with hands wrapped in plaster casts. I have no control. I have no ability to manage risk aside from pulling the HD and putting in a brand new one (which is a tricky proposition when you've always used RAID 1 and have an SSD as well). This is really saddening and upsetting. I don't know which distro I can use or trust anymore.
Comment 6 Máirín Duffy 2012-09-27 14:23:19 EDT
Hi Eric, thanks for your bug report. Just as a follow-up from our IRC conversation, what you're trying to do (preserve your old partitions, *not* reusing the containers but just not touching them, except for sharing /home with the new install) is ultimately planned to be possible, but as the new ui is currently under heavy development, it's not possible right now in F18 alpha.

So I really hope we'll be able to meet your needs soon when the new development is finished, and really apologize for the current state of things and that you had to miss out on test days. Hopefully using a live USB stick next time will help for doing nouveau testing.

Thanks!!
Comment 7 David Lehman 2012-11-21 15:35:17 EST
Patch posted that adds support for managing an LV's VG.
Comment 8 David Lehman 2012-12-03 13:15:01 EST
*** Bug 882447 has been marked as a duplicate of this bug. ***
Comment 9 joshua 2012-12-03 17:47:19 EST
Can we get a Beta2 please?  If I can't literally can't install anything until RC, something is wrong.
Comment 10 David Lehman 2012-12-06 12:36:33 EST
I've made an updates image against the beta that enables the use of a preexisting VG. To try it, add the following to your boot commandline:

 updates=http://dlehman.fedorapeople.org/updates/updates-f18b-860677.0.img

It's untested, so let me know if there are issues.
Comment 11 Ian Pilcher 2012-12-06 16:19:51 EST
(In reply to comment #10)
> I've made an updates image against the beta that enables the use of a
> preexisting VG. To try it, add the following to your boot commandline:
> 
>  updates=http://dlehman.fedorapeople.org/updates/updates-f18b-860677.0.img
> 
> It's untested, so let me know if there are issues.

My first attempt gave me an immediate traceback.  It's in bug 884861.
Comment 12 Chris Murphy 2012-12-06 17:03:02 EST
(In reply to comment #11)
Also get a traceback for anaconda 18.29-2 and 18.36-1.
Comment 13 Ian Pilcher 2012-12-07 16:44:09 EST
The "smoke 4" DVD mentioned in bug 884861 worked for me.
Comment 14 Adam Williamson 2013-01-04 18:57:49 EST
I just confirmed that with Final TC4, you can create new LVs in existing VGs no problem. You can even shrink an existing LV and create a new one in the space that's freed up and install to that and it all works.

Is there any need for this bug to be open any more? Is there still work it's covering, or did it just get lost in the mix? I don't know the status with btrfs, but current LVM status looks decent.
Comment 15 Adam Williamson 2013-01-04 19:10:12 EST
I believe the fix for this was:

https://lists.fedorahosted.org/pipermail/anaconda-patches/2012-November/002199.html

acked by vpodzime on 11-29:

https://lists.fedorahosted.org/pipermail/anaconda-patches/2012-November/002235.html

committed 11-30:

http://git.fedorahosted.org/cgit/anaconda.git/commit/?id=af8703cd6bafc9320429660003d0bd8f9ede07e3

said commit is present on f18-branch, and I just confirmed by testing that it works. So I'm pretty sure we can close this out. The btrfs case probably needs to be handled separately (there's likely already a bug to track it).
Comment 16 Eric Hopper 2013-01-04 19:22:41 EST
So, where can I find the ISO to download and test this for myself?
Comment 17 Eric Hopper 2013-01-04 19:23:16 EST
And the btrfs case is pretty important to me. So if there's a separate bug to track it, I'd like to know what it is please.
Comment 18 Adam Williamson 2013-01-04 19:23:53 EST
https://dl.fedoraproject.org/pub/alt/stage/18-TC4/ . give me a shout on IRC if you need any help with the interface, adamw on freenode (#fedora-qa).
Comment 19 Adam Williamson 2013-01-04 19:27:31 EST
https://bugzilla.redhat.com/show_bug.cgi?id=886691 looks like the bug you want for btrfs.
Comment 20 Chris Murphy 2013-01-04 19:48:29 EST
I think this bug can be closed as everything in the description for LVM and Btrfs was working as described in 18.37.8-1. It's even possible to put /boot in LVM and GRUB2 boots it; and /boot onto a boot subvol and GRUB likewise boots it.
Comment 21 Eric Hopper 2013-01-04 22:52:14 EST
The UI is still very confusing. And can't easily get a picture of which drives have which stuff on them. And my attempt to delete a partition and make a new one in the empty space resulted in it seemingly deleting another critical partition.

I think it decided that if I was deleting the one partition that must mean the other was fair game since the OS installed seemed to need both partitions to boot, which wasn't the case.

I still think Fedora 18 will not be installable for me. And I'm guessing no future revisions of Fedora will be installable. The UI appears to be going in the direction of trying to be very smart instead of be crystal clear and letting you do what you want. And the last thing I want when I'm partitioning drives is something that tries to outsmart me and deletes all of my data.

This is really vexing because I really don't like Ubuntu. Maybe I'll have to switch to CentOS.
Comment 22 Adam Williamson 2013-01-04 23:38:35 EST
I'm impressed by the quality of your crystal ball. I have trouble figuring what's going to happen next week.

Moving out of crazy future prediction territory and back to the land of sanity:

"I think it decided that if I was deleting the one partition that must mean the other was fair game since the OS installed seemed to need both partitions to boot, which wasn't the case."

No, it wouldn't do that. There is a checkbox for 'delete all other partitions that are part of this OS too', to save you time if you're wiping a whole install, but it is not checked by default. Unless you checked it, anaconda would not do that. I cannot tell what was actually going on without screenshots and/or a better description, but it is almost certainly not what you think.

"And can't easily get a picture of which drives have which stuff on them."

Yes, that's the key difference you have to understand with the new layout: it does not base its display around 'what devices are in what physical location on what drive' but around 'what devices are which parts of what existing OS installs'. It's just a different principle. You need to grok that principle and then how it works becomes a lot clearer.
Comment 23 Chris Murphy 2013-01-05 00:24:59 EST
I have seen an unselected mount point (partition) under Unknown vanish when the "delete all others" check box was not checked; but it was a while ago and always was followed by a crash. Version of anaconda? If reproducible with 18.37.[8,9,10] you need to file a bug with the installer logs found in /tmp.

The UI shows mount points, which may be comprised of one or more disks or partitions if the Device Type is LVM, RAID, or Btrfs. It can show true partitions  under "Unknown" if it doesn't have a way to piece together a previous install. But you need to think in terms of mount points first, device type and file system second, capacity third, and on what device these things are located last.

As for UI confusion, I find it's a problem for using existing pooled storage (dmraid, md raid, LVM, Btrfs) expressly because the user must know, and isn't aided in finding, what physical drives make up that pooled storage. 

Anyway, you're better off simply asking how the UI works, or how to do what you want, rather than critiquing because the mooring lines on F18 are being cut. UI feedback is in the realm of Rawhide/F19 now. And I'd ask them on test@ at this point unless you can clearly describe how this bug isn't fixed, because so far I've tested this pretty thoroughly and I'm able to get it to work.

Last, if you want to actively use an experimental file system, you need to use a distribution that's cutting out much newer kernels than Ubuntu does. I'm running 3.7.1-2 at the moment. Very few Btrfs fixes and almost no enhancements are being backported because of how active development is right now.
Comment 24 Eric Hopper 2013-01-05 00:30:57 EST
(In reply to comment #22)
> Yes, that's the key difference you have to understand with the new layout:
> it does not base its display around 'what devices are in what physical
> location on what drive' but around 'what devices are which parts of what
> existing OS installs'. It's just a different principle. You need to grok
> that principle and then how it works becomes a lot clearer.

So I get to forever unplug all the disks, install into a blank disk just for this purpose, then put in the disks back in after the install is finished so I can copy the data to the appropriate places?

Other than that, I can't see how I will be able to install with confidence. It already made an error in identifying systems because I have a partial root partition left over that I never deleted from when I created a special blank partition just to install Fedora 17 into and copy everything back into its proper places last time because btrfs support had gone completely missing from Anaconda.

And since those two partitions were basically the same install of the same OS, it's difficult telling them apart because it's such a pain to see what partition anything is on.

This is my crystal ball. That statement right up there. That way madness lies. You can't ignore the physical layout of your system, or obscure it. You'll just end up deleting things you never intended to because you can't tell where anything is.

Anyway, I'll make the stupid screenshots just to show you what I mean. It'll probably be tomorrow when I do it because I'm really sick and feeling very exhausted this evening. If posting links to them in an IRC channel is the right thing to do because Fedora 18 isn't going to be changing, that's what I'll do.
Comment 25 Adam Williamson 2013-01-05 04:17:32 EST
"So I get to forever unplug all the disks, install into a blank disk just for this purpose, then put in the disks back in after the install is finished so I can copy the data to the appropriate places?"

Um. What? No. That's not what I said at all. You can perfectly adequately modify, delete and/or re-purpose existing devices from the newUI custom part UI. It's just that it uses a different organizational principle for *displaying* them. Instead of thinking 'what disk is the partition I want to delete / reformat / reuse on and which number partition is it on that disk?', think 'what existing OS is the partition I want to delete / reformat / reuse a part of?', and you will find it in the partition group for that OS. You can select it and then perform any kind of operation you like on it - schedule it for removal or reformatting, and/or assign it a mount point within the new install.

"And since those two partitions were basically the same install of the same OS, it's difficult telling them apart because it's such a pain to see what partition anything is on."

That's a pretty unusual case, you've gotta admit. If you have a case that newUI currently doesn't handle terribly well, it's not as if it's terribly difficult to prepare things in parted first. If you can't tell which of two partitions you want to delete in newUI, well, wipe the correct one with parted first, then go ahead with the F18 install.
Comment 26 Chris Murphy 2013-01-05 14:39:43 EST
(In reply to comment #24)
F18 is in the oven already, no changing the batter. Please post details of your layout: number and size of disks, partition info, any md raid or LV info, Btrfs layout, so that it can be reproduced; might be easiest to boot from a Live CD, and 'yum install system-storage-manager', then 'ssm list' and post that. And then describe what your final goal is, and then describe the sequence you're using to arrive at that goal such that its reproducible by another. Your descriptions thus far are incoherent.

The only thing anaconda doesn't let me do with Btrfs volume reuse, is reuse an existing subvolume for Root. That must be created new, is intentional. You can reuse any other subvol for home, boot, usr, or var, although it's confusing because if you click on existing home, as soon as you click on the Mount Point field to type in /home, the previously selected subvol is deselected so it's unclear what you're modifying. So you have to remember what you just clicked on, and type in the proper mount point for each subvol you're reusing, then click Apply Changes (for which there's no feedback that it's had an effect, you have to go to New Fedora Installation to see the assignment has taken effect).

There is a dracut bug 890577 in TC4 and earlier that causes boot failure if /usr is on a usr subvol, with a work around and fix pushed to stable.