Description of problem: On accessing the MANUAL PARTITIONING screen, my hard disk layout is displayed as follows Unknown ext4 sda1 Encrypted (LUKS) sda2 BIOS boot sda9 On selecting the "Encrypted (LUKS)" partition, I can enter my passphrase and decrypt the device. Anaconda now shows my hard disk layout as follows Unknown physical volume (LVM) luks-123.... ext4 sda1 BIOS boot sda9 I can find no way to access and use my LVM configuration on the LVM physical volume. However, from the shell I can see the partition has been decrypted and Volume Group and all the Logical Volumes contained have been activated. It seems like in Fedora 24 Anaconda simply does not try to detect any LVM configuration within a LUKS-encrypted partition when it is decrypted. Version-Release number of selected component (if applicable): (as shipped in Fedora-Workstation-netinst-x86_64-24-1.2.iso) How reproducible: Steps to Reproduce: 1. LUKS-encrypt a disk partition 2. Decrypt the partition and create LVM Volume Group using the partion. 3. Create some Logical Volumes in the VG 4. Reboot using Fedora 24 installation media and try to access the LVs when creating a manual partition layout. Actual results: After decrypting the LUKS partition, Anaconda only recognizes the contents as an LVM Physical Volume and so the only option available is to reformat. Expected results: Anaconda detects the LVM configuration on the Physical Volume, and allows the LVs to be added to the filesystem tree. Additional info: I was able to use this disk layout when installing Fedora 23 (admittedly I have converted the hdd format from MBR to GDM since then).
Please attach the logs from /tmp/*log as individual text/plain attachments.
Created attachment 1170431 [details] anaconda.log
Created attachment 1170432 [details] dnf.librepo.log
Created attachment 1170433 [details] dnf.log
Created attachment 1170434 [details] dnf.rpm.log
Created attachment 1170435 [details] hawkey.log
Created attachment 1170436 [details] ifcfg.log
Created attachment 1170437 [details] packaging.log
Created attachment 1170438 [details] program.log
Created attachment 1170439 [details] sensitive-info.log
Created attachment 1170440 [details] storage.log
Cannot attach "syslog" and "X.log" - perhaps 10-attachment limit? If you require either of these, can you nominate a file to be replaced?
Note that the LUKS partition is the only PV used by VG "vg_miles": # pvs PV VG Fmt Attr PSize PFree /dev/mapper/luks-5b484048-e56e-4c04-be51-bba5553a3752 vg_miles lvm2 a-- 464.75g 280.75g
Same problem here, LVM volumes within LUKS aren't recognized by Anaconda after successfully unlocking the LUKS volume. Correct entries exist in /dev/mapper, pvs output is correct, ... but Anaconda never notices. Affects the netinst media for both Workstation and Server, haven't tried anything else thus far.
I think I might see the issue - in "/tmp/storage.log" (attached), it actually finds the devices used by LVM volumes listed in /dev/mapper: > 23:01:40,132 DEBUG storage.ui: DeviceTree.getDeviceByName: hidden: False ; incomplete: False ; name: vg_miles-libvirt ; > 23:01:40,133 DEBUG storage.ui: DeviceTree.getDeviceByName returned None > 23:01:40,134 INFO storage.ui: vg_miles-libvirt is an lvm logical volume However, it fails to find a corresponding device for the LVM volume group: > 23:01:40,150 DEBUG storage.ui: DeviceTree.getDeviceByName: hidden: False ; incomplete: False ; name: vg_miles ; > 23:01:40,152 DEBUG storage.ui: DeviceTree.getDeviceByName returned None > 23:01:40,152 ERR storage.ui: failed to find vg 'vg_miles' after scanning pvs This is to be expected as LVM volume groups have no corresponding device special file ("DeviceTree.getDeviceByName" will always fail for a VG name?) In the "/dev" file system LVM volume groups only exist as directories containing links from the names of the logical volumes they contain to the corresponding "dm" devices: # find /dev -name vg_miles /dev/vg_miles # ls -lh /dev/vg_miles total 0 lrwxrwxrwx. 1 root root 7 Jun 23 07:40 F21Root -> ../dm-5 lrwxrwxrwx. 1 root root 7 Jun 23 07:40 F23Root -> ../dm-1 lrwxrwxrwx. 1 root root 7 Jun 23 07:40 F24Root -> ../dm-7 lrwxrwxrwx. 1 root root 7 Jun 23 07:40 Home -> ../dm-6 lrwxrwxrwx. 1 root root 7 Jun 23 07:40 lfs -> ../dm-3 lrwxrwxrwx. 1 root root 7 Jun 23 07:40 libvirt -> ../dm-4 lrwxrwxrwx. 1 root root 7 Jun 23 07:40 Swap -> ../dm-2
It looks like we can't get information about the recently-decrypted PV. Now that we're using the lvm-dbus plugin I can't see what the PV info looks like -- maybe the path for a device-mapper PV isn't using the map name (using 'dm-2' instead?).
Created attachment 1171614 [details] syslog File permissions were preventing me uploading this...
Created attachment 1171615 [details] X.log
(In reply to David Lehman from comment #16) > It looks like we can't get information about the recently-decrypted PV. Now > that we're using the lvm-dbus plugin I can't see what the PV info looks like > -- maybe the path for a device-mapper PV isn't using the map name (using > 'dm-2' instead?). The problem here is that the lvmdbusd doesn't notice and react on the even of the LUKS device being opened. So while the 'pvs' utility knows about the PV, which means that the pvscan was performed, the DBus API doesn't know about it. Reassigning this to LVM.
The following updates.img should work around this issue by avoiding the use of the DBus API: http://vpodzime.fedorapeople.org/1348688_updates.img
I've succeeded in reusing encrypted partitions after specifying this updates= image file.
The 1348688_updates.img patch also allowed me to install F24 using my LVM configuration on the LUKS-encrypted volume after decryption. Thanks!
lvm fedora package isn't being built with '--enable-notify-dbus', thus the lvm dbus service is not getting notifications from lvm to refresh its state. So at the moment the lvm dbus service is quite broken. One potential workaround is to change the service file to add back in the '--udev' option until a new package with '--enable-notify-dbus' is available.
I found a bug when running with 'enable-notify-dbus'. When doing a lv move, vg move or lv snapshot merge the client will hang if they specify that they willing to wait definitely. Immediate work around is to use --udev option when running service instead of compiling with notify dbus support. Once we get a fix in for this hang then we should compile with 'enable-notify-dbus'.
Hang when using 'enable-notify-dbus' has been committed upstream. To reiterate, the issue which pertains to the bz can be addressed by 2 different ways: 1. Repackage existing version, but patch the lvmdbusd service file to use --udev 2. Release a new version of lvm which contains https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=dd5d865020acd545712d4bcc0f3236143de4d76d and utilize 'enable-notify-dbus' when building lvm.
*** Bug 1350947 has been marked as a duplicate of this bug. ***
Fedora 24 anaconda installer is fixed in current version using update image from https://fedoraproject.org/wiki/Common_F24_bugs#lvm-on-luks-reuse-broken. I'll also provide updated lvm2 package in Fedora 24 as well.
Should this be fixed in F25, does anyone know?
Yesterday in a pub, I heard this has been fixed shortly after 24 GA compose. Drunken people are oftentimes honest, so I tend to believe things can get better for Fedora 25, assuming the scotch keeps coming.
(In reply to Šimon Lukašík from comment #29) > Yesterday in a pub, I heard this has been fixed shortly after 24 GA compose. > > Drunken people are oftentimes honest, so I tend to believe things can get > better for Fedora 25, assuming the scotch keeps coming. LOL http://giphy.com/gifs/maury-the-maury-show-l3V0wkQ2KKcAeW8Cs
(In reply to Šimon Lukašík from comment #29) > Yesterday in a pub, I heard this has been fixed shortly after 24 GA compose. I can confirm this...should be fixed. :) > > Drunken people are oftentimes honest, so I tend to believe things can get > better for Fedora 25, assuming the scotch keeps coming. And this I can confirm too. :)
Since my search skills failed me in finding this report and the updates file, my workaround was to do something dangerous and use the Fedora 25 pre-release netinst, with the 24 repo. I came here to document the LVM issue, but then found this report. Anyway, in addition to that updates image, it appears to work just fine in the Fedora 25 pre-release. I just thought I'd point that out. Hopefully it will continue to work when Fedora 25 is released.
Just want to inform you that this bug appears in F26. Could you look at it and fix it? I guess it was build from 27. 04.
Just additional info: This is whole process how to use already created partitions on F25 installer: 1) select disk + custom partitioning 2) rescan disk (automatically go back to disk selection...) 3) go again to partitioning - now you see unkown parititons 4) unlock luks partition 5) rescan disk again (again you are switched to disk selection ....) 6) and again, go back to partitioning -- now you can finally continue and select your paritions.. But this works only on F25. For F26 you stuck on step 5. Is it really necessary? Is there some specific purpose for this pain? Why this is not merged only to two steps (1, 4)? Or at least 1,2,4 (in case that "rescan" could take a lot of time and you don't want to reuse any partition)? If you really want to keep the process so unfriendly, there would be at least info message informing user they have to rescan disk again and again manually.
Heh. I tried that again on F25 and behaviour is different again. So now I didn't have to do steps 5,6. So there are probably some another troubles, when sometimes it requires another rescan and sometimes it doesn't.
So, it is because of different behaviour between use of the Unlock button and hit the Enter key. See https://bugzilla.redhat.com/show_bug.cgi?id=1446924
I haven't tried myself, but based on comment 33 proposing as a beta blocker: https://fedoraproject.org/wiki/Fedora_26_Beta_Release_Criteria#Custom_partitioning
Can someone provide a clear explanation of how to reproduce this, starting from scratch (i.e. a VM with an empty hard disk)? I tried to reproduce this by doing an F25 install using guided partitioning with "encrypt my data" selected, then booting the most recent F26 nightly and going to custom partitioning. It showed the encrypted device under Other, and when I selected it and entered the passphrase, it worked correctly - custom now shows a 'Fedora Linux 25 for x86_64' entry with the relevant LVs in it. So does this involve a different way of setting up the original install?
I have reproduced in a VM on a Fedora 25 host with qemu-system-x86 (other non @^minimal-environment packages needed for this process include wget, gdisk, qemu-img, and a desktop environment unless you use SPICE or VNC to view QEMU). # # I used this installer # wget 'https://mirrors.kernel.org/fedora/development/26/Everything/x86_64/iso/Fedora-Everything-netinst-x86_64-26-20170506.n.0.iso' # # Pre-partition an empty disk # (using host via qemu-nbd) # # Create disk image qemu-img create -f qed fedora.bug.1348688.test.qed 32G # Open disk image on host sudo modprobe nbd max_part=64 sudo qemu-nbd -c /dev/nbd0 -f qed fedora.bug.1348688.test.qed # Partition disk sudo sgdisk /dev/nbd0 \ -Z \ -n=1::+1M -t=1:ef02 -c=1:'BIOS boot partition' \ -n=2::+64M -t=2:ef00 -c=2:'EFI System Partition' \ -n=3::+512M -t=3:8300 -c=3:'boot' \ -n=4:: -t=4:8300 -c=4:'' \ -p # Create LUKS sudo cryptsetup luksFormat /dev/nbd0p4 sudo cryptsetup open --type luks /dev/nbd0p4 cryptroot # Create VG (named after its own UUID) sudo lvm pvcreate /dev/mapper/cryptroot sudo lvm vgcreate vg00 /dev/mapper/cryptroot eval "$(sudo lvm vgs --nameprefixes --no-headings -o uuid vg00)" sudo lvm vgrename vg00 "vg_$LVM2_VG_UUID" # Create root LV sudo lvm lvcreate -l 100%FREE -n root "vg_$LVM2_VG_UUID" # Close everything sudo lvm vgchange -an "vg_$LVM2_VG_UUID" sudo cryptsetup close cryptroot sudo qemu-nbd -d /dev/nbd0 # # Start VM with just-partitioned disk # qemu-system-x86_64 \ -nodefaults -nodefconfig -no-user-config \ -machine q35,accel=kvm -cpu host -smp cores=$(nproc) -m 2G \ -drive file="/usr/share/edk2/ovmf/OVMF_CODE.fd",format=raw,if=pflash,readonly=on \ -device virtio-vga \ -device virtio-net-pci,netdev=eth0 -netdev user,id=eth0 \ -drive file="Fedora-Everything-netinst-x86_64-26-20170506.n.0.iso",format=raw,media=cdrom,if=none,id=cdrom0 \ -device virtio-scsi-pci,id=scsi -device scsi-disk,drive=cdrom0,bus=scsi.0 \ -drive file="fedora.bug.1348688.test.qed",format=qed,if=virtio \ -monitor stdio In installer: - Accept fate - Select Installation Destination - Storage Configuration: Custom - Done (Installation Destination) - Expand "Unknown" category - Select "Encryoted (LUKS)", unlock Notice vg_whatever-root is not an option (was immediately an option in F25). Woraround: - Main menu with Done (Manual Partitioning) twice - Select Installation Destination - Click "Refresh..." in bottom right - Rescan Disks - Done (Installation Destination) Notice vg_whatever-root is now available.
Discussed in 2017-05-08 Blocker review meeting. This bug is a violation of the Beta criterion: "When using the custom partitioning flow, the installer must be able to: ... Correctly interpret, and modify as described below, any disk with a valid ms-dos or gpt disk label and partition table..."
I'm setting this back to ASSIGNED since it's an active issue again, though really this bug shouldn't have been reopened, I don't think, a new bug should've been filed...
Vrato, the following patch was suggested as potential fix: https://sourceware.org/git/?p=lvm2.git;a=commit;h=e53454d6de07de56736303dd2157c3859f6fa848 However, that is changing GetManagedObjects and LookUpByLvmId calls from synchronous to asynchronous. How big is that a deal? Is it a change in API or are dbus libs somehow abstracting it away?
(In reply to Marian Csontos from comment #42) > Vrato, the following patch was suggested as potential fix: > https://sourceware.org/git/?p=lvm2.git;a=commit; > h=e53454d6de07de56736303dd2157c3859f6fa848 > > However, that is changing GetManagedObjects and LookUpByLvmId calls from > synchronous to asynchronous. How big is that a deal? Is it a change in API > or are dbus libs somehow abstracting it away? Shouldn't affect the means Anaconda->Blivet->libblockdev get data from the DBus API.
Except of one bug in original, the backport was straightforward. [Untested yet] scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=19605558 (sorry, for not using better "Release:"...) Vrato, are you able to test it? IIUC this requires an update.img which last time I tried I failed to produce working one. ..may be I could do something in %pre script in kickstart. Sorry, out of time today...
(In reply to Marian Csontos from comment #42) > Vrato, the following patch was suggested as potential fix: > https://sourceware.org/git/?p=lvm2.git;a=commit; > h=e53454d6de07de56736303dd2157c3859f6fa848 > > However, that is changing GetManagedObjects and LookUpByLvmId calls from > synchronous to asynchronous. How big is that a deal? Is it a change in API > or are dbus libs somehow abstracting it away? This change did not change the external behavior. The dbus calls from the client are still synchronous, they block until the result is returned (if that was their previous mode of operation). Only the implementation changed. However, clients are free to block or not block on the response as needed for their use case and what client side library they are using and the functionality it provides.
I already realized it would likely be fine as DBus is all about messaging, so the client must either block or use callbacks while waiting for the response. Thanks Tony for confirmation. I will attach updates.img. And try to test it...
Created attachment 1280350 [details] updates To test add the following to boot options: updates=https://mcsontos.fedorapeople.org/bz1348688/updates-bz1348688.img
Tested. I now see the LV instead of PV only. Building a package.
lvm2-2.02.168-6.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-dc0776cb70
Should show up in next nightly, I imagine?
No, nightlies are built from stable (they don't include updates-testing packages). If you can test with the updates.img and provide feedback on the update it'd be very helpful, thanks!
lvm2-2.02.168-6.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-dc0776cb70
Steve, can you re-test with Beta RC4 (Beta-1.4) and see if it works with that? Thanks! https://dl.fedoraproject.org/pub/alt/stage/26_Beta-1.4/
From my testing, it seems to work fine with 1.4
Using "Fedora-Workstation-netinst-x86_64-26_Beta-1.4.iso" I was able to unlock my LUKS-encrypted partition and access/use the logical volumes contained to install Fedora 26 Beta. This was possible using both the "Custom" and "Advanced Custom" options on the "Installation Destination" panel.
lvm2-2.02.168-6.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.