Description of problem: Fedora 16 out of box installation fails after the first reboot when installing on iSCSI LUN using a NIC that supports iBFT (not PXE!). Boot from the Live CD succeeds and the file system is partitioned and files are installed. Upon reboot the system presents grub menu and proceeds to boot. Boot is interrupted because the root filesystem can't be mounted, and the user is dropped to an emergency shell. This is the grub entry that the Fedora 16 installer generated: menuentry 'Fedora Linux, with Linux 3.1.0-0.rc3.git0.0.fc16.x86_64' --class fedora --class gnu-linux --class $ load_video set gfxpayload=keep insmod part_msdos insmod ext2 set root='(hd0,msdos1)' search --no-floppy --fs-uuid --set=root 52b9fdbc-67ce-44e2-93c5-9ec0948a4721 echo 'Loading Linux 3.1.0-0.rc3.git0.0.fc16.x86_64 ...' linux /vmlinuz-3.1.0-0.rc3.git0.0.fc16.x86_64 root=/dev/mapper/vg_vmhost01-lv_root ro iscsi_firmwar$ echo 'Loading initial ramdisk ...' initrd /initramfs-3.1.0-0.rc3.git0.0.fc16.x86_64.img } Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Install iBFT capable network card (in this case, Intel CT PCIe adapter with iSCSI boot firmware) 2. Install Fedora on iSCSI LUN 3. Reboot Actual results: You are dropped to a dracut shell with a very limited filesystem (appears to be running from RAM). No debugging tools available in this shell. Expected results: System should boot into the normal first boot screen for Fedora 16. Additional info: I have confirmed that the hardware is working; a different operating system installs and boots from the same hardware with no problems.
Clarification on the grub entry: "iscsi_firmware rd.dm=0 rd.lvm.lv=vg_vmhost01/lv_swap KEYTABLE=us SYSfont=latarcyrheb-sun16 rhgb rd.md=0 rd.luks=0 LANG=en_US.UTF-8 rd.lvm.lv=vg_vmhost01/lv_root" Part of the grub entry was cut off in the original comment. The additional comment gives the rest of the line. The actual error message just before I am dropped to a debug shell is: dracut Warning: No root device "block:/dev/mapper/vg_vmhost01-lv_root" found dracut warning: LVM vg-vmhost01/lv_swap not found dracut warning: LVM vg_vmhost01/lv_root not found dracut warning: LVM vg_vmhost01/lv_root not found Dropping to debug shell. sh: can't access tty; job control turned off
(In reply to comment #1) > Clarification on the grub entry: > > "iscsi_firmware rd.dm=0 rd.lvm.lv=vg_vmhost01/lv_swap KEYTABLE=us > SYSfont=latarcyrheb-sun16 rhgb rd.md=0 rd.luks=0 LANG=en_US.UTF-8 > rd.lvm.lv=vg_vmhost01/lv_root" > > Part of the grub entry was cut off in the original comment. The additional > comment gives the rest of the line. > > > The actual error message just before I am dropped to a debug shell is: > > dracut Warning: No root device "block:/dev/mapper/vg_vmhost01-lv_root" found > dracut warning: LVM vg-vmhost01/lv_swap not found > dracut warning: LVM vg_vmhost01/lv_root not found > dracut warning: LVM vg_vmhost01/lv_root not found > > Dropping to debug shell. > > sh: can't access tty; job control turned off I thought that specifying iscsi_firmware for iBFT boot was all that's needed. Harald, is that correct or are we missing something?
Discussed at 2011-09-09 blocker review meeting. We agreed this is a final blocker per criterion "The installer must be able to complete an installation using any network-attached storage devices (e.g. iSCSI, FCoE, Fibre Channel)". it's not a beta blocker, but we'll take it as NTH, as obviously we should fix this for beta if we can.
Dan can you please attach the installation logs? Those are the /tmp/*log files after the installation finishes but before reboot.
Also Harald (the Dracut maintainer) asks whether you could append "rd.debug=1 rd.shell=1" to the kernel boot line and paste the output?
Moving to dracut, Anaconda correctly adds 'iscsi_firmware' to dracut parameters.
Dan, we need the information requested in comment #4 and #5 to move forward on this bug: can you please provide it? Thanks.
Created attachment 529134 [details] anaconda.log
Created attachment 529135 [details] ifcfg.log
Created attachment 529136 [details] program.log
Created attachment 529137 [details] storage.log
I'm not sure what's changed, but I can't even get to a grub prompt now after the install (so I can't get the rd.debug and rd.shell items). I'm using the newest F16 Live CD downloaded from the Fedora website as of 10/19/2011. Dan
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
The failure to get to a grub prompt looks like another victim of 15:34:42,508 ERR program: cat: /boot/grub2/video.lst: No such file or directory https://bugzilla.redhat.com/show_bug.cgi?id=736993 The workaround for that is to boot to rescue mode, chroot to /mnt/sysimage , do grub2-install /dev/foobar then grub2-mkconfig -o /boot/grub2/grub.cfg - can you try that and see if it shifts things? -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
Actually, I'm wrong in comment #14, please ignore that. The critical error for 736993 is "Serial terminal not available on this platform." and you don't have that. So there's some other reason yours didn't boot.
so, we're really kind of at a loss as to what's going wrong here. dan, sorry to bug you, but instead of using nightlies, can you please give TC2 a shot, as that's our current 'known' test image? http://dl.fedoraproject.org/pub/alt/stage/16.TC2/ it might be good to test the DVD as well as the live image, too. thanks!
I tested with a custom boot.iso (anaconda-16.23-1 and dracut-013-16 so final TC2-ish) and an intel NIC that supports booting from iSCSI. I was not able to reproduce the issue. I was able to do a minimal install to an iSCSI target and it did boot after installing. I'll retest again once TC3 is released in order to verify with a more official image but I don't think that the results will be any different.
given that we have no further info from the maintainer and tflink can't reproduce, i'm not sure this is blocker material any more. -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
I did a full graphical install to an iSCSI target with the TC3 x86_64 DVD. After installation, I have no problems booting into the system. There is one oddity, though. The grub configuration as generated at install time doesn't use iscsi_firmware even though this system is capable of doing so (I tried adding it in manually). Instead, the kernel options use netroot and iscsi_initiator. I'm not sure if that makes much of a difference since the system does boot but it doesn't seem quite right.
Can you describe your iSCSI adapter? I can boot from iSCSI if I use gPXE but not the built in iBFT function of the NIC. Dan
Dan: few more questions for you - can you re-test with TC3: http://dl.fedoraproject.org/pub/alt/stage/16.TC3/ and see where things are at? If you can get to dracut can you give the earlier requested info? Can you provide a few more details about all the hardware in the system? and can you try with the regular DVD / net installer instead of live, and see how that goes? we kind of expected that people using iSCSI would install via the traditional installer rather than the live image; is there a particular reason you're using the live image? thanks! -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
Sure - I'll download 16.TC3. I normally use the LiveCD because I don't add a lot of packages after the install and it's a smaller download. I'm on Comcast and I'm close to my 250 Gb limit :-) Dan
(In reply to comment #20) > Can you describe your iSCSI adapter? I can boot from iSCSI if I use gPXE but > not the built in iBFT function of the NIC. Not exactly sure what you're looking for here but: Intel PRO/1000 PT Server Adapter Flashed with iSCSI boot ROM
netinst.iso + minimal install is probably the most efficient for bandwidth... -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
I just tried doing another installation with the TC3 x86_64 Desktop Live CD and was able to reproduce the error. It seems to have something to do with the live installer, then since I wasn't able to reproduce with either a netinstall iso or DVD.
Discussed at 2011-10-28 blocker review meeting. We agreed that on our current understanding of this bug we're downgrading its status from blocker to NTH as it appears to affect only live installs, and live installs are primarily intended for desktop deployment, not exotic stuff like this. Please re-propose if our understanding of this changes and the bug appears to affect DVD/netinst installs as well.
If it is bound to the live installer, maybe not all packages are available for the dracut netroot boot. dracut-network has to be installed for this to work!!!
I had this problem on an older intel MB/CPU 64 capable combo. I tried every option. That is it failed to reboot after fresh install using 32, 64 net install, live cd, and full dvd. All failed indicating that this is not an installation specific issue.
bruce: yeah, our working theory now is this will always happen if you try and do an iSCSI install from the live image, but we think 'don't do iSCSI installs from the live image' is a reasonable 'workaround'. The live installer isn't really intended for deploying complex enterprise storage scenarios, it's meant for typical desktops. We should commonbugs this, though. -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
-- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
harald: is this truly CANTFIX forever, or just for F16 since it's been released? if it's only CANTFIX for F16, should we re-open and bump it to Rawhide, so we can fix F17? -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
(In reply to comment #31) > harald: is this truly CANTFIX forever, or just for F16 since it's been > released? if it's only CANTFIX for F16, should we re-open and bump it to > Rawhide, so we can fix F17? Quoting yourself: The live installer isn't really intended for deploying complex enterprise storage scenarios, it's meant for typical desktops.
that was a justification for not considering the bug an f16 release blocker, not a reason never to fix it. -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
(In reply to comment #33) > that was a justification for not considering the bug an f16 release blocker, > not a reason never to fix it. > Dracut supports all... the live cd just has to include dracut-network. Not dracut's fault.