Bug 680100

Summary: second luks-encrypted partition lost during pivot
Product: [Fedora] Fedora Reporter: David Juran <djuran>
Component: dracutAssignee: dracut-maint
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 14CC: bodhi.zazen, harald, jonathan
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-01-23 09:14:27 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
udevadm info --query=property --name=/dev/sda3 none

Description David Juran 2011-02-24 11:01:12 UTC
Description of problem:
On my F14 laptop, the VG that holds my root volume resides on a PV that is a luks-encrypted device on /dev/sd2. Now sda2 is running out of space so I created /dev/sd3, ran luksFormat, created a PV on the new dm-crypt device, added it to the VG and rebuilt the initrd. So far so good. But when I reboot the laptop, only the original /dev/sda2 gets opened during boot and the root VG complains that there is a missing PV. 

Looking a bit closer at what actually happens, it seems that the dm-crypt device is lost during pivot. If I break the initrd before pivot (rdbreak=pre-pivot) then /dev/sda3 is open and active but if I break after pivot (by just adding rdbreak to the kernel command line) then at that point only /dev/sda2 is active but not sda3

Version-Release number of selected component (if applicable):
dracut-006-6.fc14

How reproducible:
every time

Steps to Reproduce:
Start out with a luks volume on /dev/sda2 on which the VG holding your root file system resides.

1. cryptSetup luksFormat /dev/sda3
2. cryptSetup luksOpen /dev/sda3 crypt3
3. vgextend <your-root-VG> /dev/mapper/crypt3
4. dracut -f  /boot//initramfs-2.6.35.11-83.fc14.x86_64.img 2.6.35.11-83.fc14.x86_64
5. reboot
6. note that your root VG is missing the new PV

Comment 1 Harald Hoyer 2011-02-25 08:26:09 UTC
you have to add it to the kernel command like the other one is, or remove any "rd_LVM*" arguments.

Comment 2 David Juran 2011-02-25 12:42:51 UTC
I don't have any rd_* parameters at all (my grub entry below). But do keep in mind that the volume gets unlocked during intird, it's open if you break pre-pivot but gone again right after pivot...

title Fedora (2.6.35.11-83.fc14.x86_64)
        root (hd0,0)
      kernel /vmlinuz-2.6.35.11-83.fc14.x86_64 ro root=UUID=cfb3f940-d5f3-48bd-92c9-c5406678a6a8 rhgb SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=fi
      initrd /initramfs-2.6.35.11-83.fc14.x86_64.img

Comment 3 Harald Hoyer 2011-02-25 13:09:18 UTC
can you install "dracut-tools" and show me the output of 

# dracut-gencmdline

Comment 4 David Juran 2011-02-25 13:29:48 UTC
[djuran@localhost Desktop]$ sudo dracut-gencmdline
rd_NO_MD rd_LVM_VG=HelpdeskRHEL4 rd_LUKS_UUID=luks-0a5363f2-e029-46da-8949-73e33595b639 KEYTABLE=fi SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 root=UUID=cfb3f940-d5f3-48bd-92c9-c5406678a6a8 


And in there it's only getting the original /dev/sda2. /dev/sda3 has UUID 0e82b193-b6e6-4861-be16-cdf5492b3acb but isn't mentioned...

I also tried adding both UUID:s to the command-line (using rd_LUKS_UUID twice) and rebuilding the initrd but it still didn't work.

Comment 5 Harald Hoyer 2011-02-25 14:02:14 UTC
(In reply to comment #4)
> [djuran@localhost Desktop]$ sudo dracut-gencmdline
> rd_NO_MD rd_LVM_VG=HelpdeskRHEL4
> rd_LUKS_UUID=luks-0a5363f2-e029-46da-8949-73e33595b639 KEYTABLE=fi
> SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8
> root=UUID=cfb3f940-d5f3-48bd-92c9-c5406678a6a8 
> 
> 
> And in there it's only getting the original /dev/sda2. /dev/sda3 has UUID
> 0e82b193-b6e6-4861-be16-cdf5492b3acb but isn't mentioned...

are you sure the UUID is 0e82b193-b6e6-4861-be16-cdf5492b3acb?

$ udevadm info --query=property --name=/dev/sda3

> 
> I also tried adding both UUID:s to the command-line (using rd_LUKS_UUID twice)
> and rebuilding the initrd but it still didn't work.

you should not have to rebuild the initramfs

just add the parameter to the _kernel_ command line in /etc/grub.conf:

rd_NO_MD rd_LVM_VG=HelpdeskRHEL4 rd_LUKS_UUID=luks-0a5363f2-e029-46da-8949-73e33595b639 rd_LUKS_UUID=luks-0e82b193-b6e6-4861-be16-cdf5492b3acb KEYTABLE=fi SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 root=UUID=cfb3f940-d5f3-48bd-92c9-c5406678a6a8

Comment 6 David Juran 2011-02-25 14:50:36 UTC
About the UUID, at least that's what "cryptsetup luksDump" reports and udevadm tends to agree (at least in the ID_FS_UUID and ID_FS_UUID_ENC fileds). Full output attached

And adding the parameters you mentioned makes no difference. During the bootup, /dev/sda3 gets unlocked but once the system is up again, the device is not opened, only /dev/sda2 is active.

Comment 7 David Juran 2011-02-25 14:51:44 UTC
Created attachment 481011 [details]
udevadm info --query=property --name=/dev/sda3

Comment 8 David Juran 2011-03-23 13:49:25 UTC
Digging a bit deeper into this issue, I think the problem lies in the crypt-cleanup.sh scripts in pre-pivot in the initrd. They basically try to close _all_ luks volumes and there isn't anything that ever re-opens them after pivot. 
  
 The only reason that a machine with a luks volume at all works seem to be by accident, since the first luks volume is busy when crypt-cleanup.sh is running so it fails to close.

I tried removing these scripts and then everything works fine. What is the reason for having these scripts in the first place? Can we just dump them?

Comment 9 David Juran 2011-03-29 13:30:46 UTC
Discussing with Harald Hoyer I understand that in dracut-008 (F15), if a luks volume is specified on the kernel commandline (with rd_LUKS_UUID), then it'll survive through pivot.

For now, the workaround is to expand the root FS onto every device you want to have open after boot.

Comment 10 bodhi.zazen 2011-06-20 20:52:05 UTC

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 11 Fedora Admin XMLRPC Client 2011-10-20 16:21:37 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.