Bug 1347379

Summary: Unable to Install Fedora on a BTRFS Subvolume in a LUKS Container
Product: [Fedora] Fedora Reporter: Naftuli Tzvi Kay <rfkrocktk>
Component: anacondaAssignee: Anaconda Maintenance Team <anaconda-maint-list>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 24CC: anaconda-maint-list, butirsky, g.kaviyarasu, io, jonathan, rfkrocktk, vanmeeuwen+fedora
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-20 15:54:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Error message none

Description Naftuli Tzvi Kay 2016-06-16 16:25:11 UTC
Created attachment 1168792 [details]
Error message

Description of problem:
When attempting to install Fedora in an encrypted LUKS container, using two BTRFS subvolumes for the root mount and the home mount, the installer doesn't allow installation to proceed.

Version-Release number of selected component (if applicable):
This applies to both Fedora 23 and Fedora 24 Beta installers.

How reproducible:
Consistently reproducible, happens every time.

Steps to Reproduce:
Boot into a Fedora LiveCD. Create filesystems as follows:

sudo parted /dev/sda mklabel gpt
# create GRUB BIOS partition
sudo parted /dev/sda mkpart primary 2048s 2MiB
# create EFI partition
sudo parted /dev/sda mkpart primary 2MiB 256MiB
sudo mkfs.vfat /dev/sda2
# create Linux boot partition
sudo parted /dev/sda mkpart primary 256MiB 512MiB
sudo mkfs.btrfs /dev/sda3
# create LUKS container
sudo parted /dev/sda mkpart primary 512MiB 100%
echo -n password | sudo cryptsetup luksFormat --key-file - /dev/sda4
echo -n password | sudo cryptsetup luksOpen /dev/sda4 rootcrypt
# create BTRFS filesystem in LUKS container
sudo mkfs.btrfs /dev/mapper/rootcrypt
# create BTRFS subvolumes
sudo mount -t btrfs -o defaults /dev/mapper/rootcrypt /mnt
# create subvolume for root
sudo btrfs subvolume create /mnt/@
# create subvolume for home
sudo btrfs subvolume create /mnt/@home
# unmount and close LUKS
sudo umount /mnt
sudo cryptsetup close rootcrypt

After following the above procedure, start the Fedora installer and configure partitioning manually. Set /dev/sda3 as /boot, set /dev/sda2 as /boot/efi, and then decrypt the LUKS container with "password."

Attempt to register the BTRFS subvolume "@" as the root "/" mount point. 

The installer reports an error, saying "You must create a new file system on the root device." Installation cannot proceed.

Actual results:
I am unable to install Fedora.


Expected results:
I should be able to install Fedora, as I don't believe my expectations are out of band; I'm able to install Ubuntu under the exact same setup.


Additional info:
I've walked through the above in Fedora 23 and 24 in a VM and on physical devices and captured the screenshots here: https://imgur.com/a/mnRqa

Comment 1 David Lehman 2016-06-20 15:54:20 UTC
The use of a pre-existing filesystem for the root filesystem is not allowed. I don't know why you prefer to do all of that work yourself instead of letting the installer do it for you, but you're going to have to create the root filesystem in the installer.

Comment 2 Naftuli Tzvi Kay 2016-06-21 04:21:17 UTC
I'm pretty particular about the size of my filesystems.

Does Fedora want a completely empty disk by default? I think it's kind of strange to not allow users to use existing partitions; if previous data is the concern, mandate reformatting.

I'm installing on a MacBook, and I need to maintain a specific partition layout and sizing so that the machine will actually boot. 

If I go through the above steps identically and instead use ext4, everything just works. BTRFS subvolumes seem to be the issue here.

Comment 3 David Lehman 2016-06-21 15:51:10 UTC
(In reply to Naftuli Tzvi Kay from comment #2)
> I'm pretty particular about the size of my filesystems.
> 
> Does Fedora want a completely empty disk by default? I think it's kind of
> strange to not allow users to use existing partitions; if previous data is
> the concern, mandate reformatting.

That is the requirement. You are free to create the devices if you prefer, even if there is really no reason for it -- anaconda is more than capable of creating the layout you're making there.

> 
> I'm installing on a MacBook, and I need to maintain a specific partition
> layout and sizing so that the machine will actually boot. 

I'm fairly sure we have taken that into account already.

> 
> If I go through the above steps identically and instead use ext4, everything
> just works. BTRFS subvolumes seem to be the issue here.

Yes, because a btrfs subvolume cannot be "reformatted". You could create the root subvolume in anaconda, though. I don't know if it would allow you to name it "@", though. (I doubt it.)

Comment 4 Naftuli Tzvi Kay 2016-06-21 16:09:52 UTC
To anyone finding this, there does appear to be a way to have Anaconda do everything with LUKS and BTRFS. I'll install it in a VM as closely as possible to my desired way, and then copy /root/anaconda.ks and start making changes from there to do what I'm looking to do, similar to this: https://blog.laimbock.com/2014/12/27/howto-kickstart-fedora-with-luks-encrypted-partitions/

Comment 5 David Lehman 2016-06-21 17:33:56 UTC
I am confused as to why you think all that is necessary. The following should work:

ignoredisk --onlyuse=sda
clearpart --all --drives=sda --disklabel=gpt

part biosboot --size=2
part /boot/efi --fstype=vfat --size=256
part /boot --size=512
part btrfs.1 --size=1024 --grow --fstype=btrfs --encrypted --passphrase=mypass
btrfs none --label=fedora btrfs.1
btrfs / --subvol --name=root LABEL=fedora
btrfs /home --subvol --name=home LABEL=fedora


If it doesn't work, either there's an error in the above or a bug in anaconda or a related component. There is no reason for you (or the author of the blog you referenced) to be doing all of this manually. If there are problems, file bugs. We've done all this work for you a long time ago.

Comment 6 David Lehman 2016-06-21 17:41:40 UTC
You may have problems because you're trying to create partitions for both BIOS and UEFI booting. If you could make up your mind how you want to boot things would be simpler for you since none of the OS tools will automatically update both BIOS and UEFI configurations automatically AFAIK. If you are booting via UEFI you should not create biosboot and you should call the /boot/efi fstype 'efiboot' instead of 'vfat'. If you're booting via BIOS (in legacy mode), you should omit the /boot/efi completely. If you insist on trying to keep both you may end up having to do things manually, including bootloader configuration (every time you update the kernel, too).

Comment 7 Naftuli Tzvi Kay 2016-06-21 21:49:48 UTC
@David Lehman: I wasn't sure re: GRUB and the biosboot partition, I usually use rEFInd as a bootloader, so yeah.

I want GPT and EFI booting.

I basically want to create the LUKS partition by hand so I can specify different ciphers. The way that Kickstart/Anaconda does things regarding encrypted filesystems is confusing at best.

Comment 8 David Lehman 2016-06-23 15:39:21 UTC
You could get almost exactly what you want with this as your whole storage config:

ignoredisk --onlyuse=sda
clearpart --all drives=sda
autopart --type=btrfs --encrypted --cipher=acipher --passphrase=apassphrase


As far as booting goes, your system will be configured appropriately for however you booted the system to perform the installation. We've done a lot of work to make this all easy for you. You can find reasons to make it hard if you want, but it doesn't have to be that way.

Comment 9 Naftuli Tzvi Kay 2016-06-24 03:08:39 UTC
Sorry, I guess this is coming from an 8 year Ubuntu veteran who always had to do things by hand in order to make sure that I got the right filesystem setup. 

Perhaps Anaconda _does_ always do the right thing, I've just found it incredibly unwieldy to work with. For example, what if I wanted to maintain my home mount while formatting the root mount to install Fedora in place of where Ubuntu was? With the problem described above, I'd be pretty much hosed.

What I'd really like would be to find and have a way to install Fedora outside of Kickstart and Anaconda. Anaconda, as described above, won't let you install onto a preexisting root filesystem that it can't format, e.g. a BTRFS subvolume. Kickstart is kind of problematic for me as it's hard to test that a Kickstart will do what I expect it to do without doing a full integration test. I setup VirtualBox and some automated scripts here to bootstrap a VM and send it the Kickstarter config over HTTP, but I'm finding that it's slow and cumbersome to test. 

While I fully understand that this is not a technical support forum, is there a way to install Fedora without Kickstart or Anaconda? I've messed around with using DNF natively to create something akin to a stage 3: https://gist.github.com/rfkrocktk/83e6b843b7cd366c75deae3095ac3214
but I've had quite a few issues with it.

What I'd like to do is described above as far as filesystem layout is concerned, but I also usually initialize my LUKS partition before formatting it with random data, c.f. https://gist.github.com/rfkrocktk/83e6b843b7cd366c75deae3095ac3214#file-luks-setup-ks-cfg-L51-L61

In any case, if I'm using Kickstart or Anaconda to create everything, I'll be re-encrypting my LUKS partition anyway to make sure my KDF time, cipher, etc. are all set to my preference. 

If there's a faster/better way to test Kickstart scripts, I think that'd be the best thing so I could tweak what I already have in order to get something working in the way that I want. Alternatively, a way to create a base install of Fedora without Kickstart/Anaconda would be desirable.

At this point, I may just use Anaconda to create a system in a VM and then create a tarball of it so I can just rsync it in the future to a new install, tweak crypttab/fstab/kernel boot line, and then be on my merry way.

Comment 10 Mikel PĂ©rez 2019-10-05 19:30:45 UTC
I want to reopen this bug. I installed fedora 31 without knowing samba-dc was broken (#1757071),
and made a fedora-30 subvolume to install it there but anaconda wouldn't let me.

even if that is fixed, it should be possible to clean install on an already existing
btrfs subvolume (as long as the target subvolume is empty)