Hide Forgot
Description of problem: Anaconad didn't bring up iBFT NIC up when boot from iBFT iSCSI when using ksdevice=bootif is another NIC. kernel iscsi option will also incorrect point to bootif rather than iBFT NIC. This will cause os cannot boot up. Version-Release number of selected component (if applicable): RHEL 6.1 GA How reproducible: 100% Steps to Reproduce: 1. Find a server with iBFT NIC and another NIC for kickstart. 2. Install OS on that server. 3. Verify the kernel option, check whether it point to correct iBFT NIC. Actual results: kernel iscsi option point to ksdevice rather than iBFT. iBFT NIC also not bring up automatically during anaconda installation which might cause this issue. Expected results: anaconad should respect /sys/firmware/ibft/ethernet0/ when found OS boot from iBFT NIC. Additional info: let me know if you need that server to test.
Please attach /tmp/anaconda.log and /tmp/syslog. The files can be copied from installer environment using scp from shell in tty2. On installed system they can be found in /var/log/anaconda as anaconda.log and anaconda.syslog. We support activation of additional devices (e.g. iSCSI device additionaly to device used to fetch kickstart file) only using kickstart network command. See https://bugzilla.redhat.com/show_bug.cgi?id=638131#c22 for example of such configuration, you should be able to use "bootif" instead of "<mac address of INSTDEV>" in the example.
Created attachment 517086 [details] anconda log on dell-pe2950-01.rhts.eng.nay.redhat.com
Created attachment 517087 [details] syslog of anaconda on dell-pe2950-01.rhts.eng.nay.redhat.com
The network setup of this server: eth0: 00:15:17:C8:C3:DA iBFT NIC, static IP, 10.66.87.185/28, target: 10.66.87.184 eth2: 00:19:B9:DC:5A:1A PXE boot NIC, bootif, dhcp, 10.66.86.21/23, Both eth0 and eth2 can reach iscsi target within subnet. The kernel option will be setuped as: ===== iscsi_firmware ip=eth2:dhcp ifname=eth2:00:19:b9:dc:5a:1a ===== This is the iscsi session detail: ==== sh-4.1# iscsiadm -m session -P 1 Target: iqn.1994-04.com.redhat:nay.hds.2100.iscsid3 Current Portal: 10.66.87.184:3260,1 Persistent Portal: 10.66.87.184:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:ibft-dell-pe2950-01 Iface IPaddress: 10.66.86.21 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 1 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ==== The OS is using the eth2 for iscsi rather eth0 (iBFT NIC).
Only eth2 was activated during install and as iscsi target is reachable via eth2 so anaconda assumed that eth2 is the device that should be used for iscsi in target system (dracut/kernel options). As I said, anaconda won't bring up ibft device automatically (nor use ibft to configure it) if you specify another device to be used for installation (here ksdevice). You have to ask for activation and (configuration using ibft) of ibft device in kickstart with command network --device=ibft --bootproto=ibft --onboot=yes --activate --nodefroute Could you please post also kickstart you are using?
I noticed this line in its kickstart file which might cause the problem: === network --onboot no --device eth0 --noipv4 --noipv6 === manual option of cobbler is not like its literal meaning. I tried again which overide the cobller kickstart template. But the anaconda still using eth2 for iscsi connection and eth0 is still down. The kickstart command you provide still can __not__ boot that OS up. I am using this kickstart file for installation. === http://lacrosse.corp.redhat.com/~fge/tmp/anaconda-ks.cfg === The kernel option after installation seems correct now: === iscsi_firmware ip=ibft ifname=eth0:00:15:17:c8:c3:da === But I got a black screen after press "b" (for boot) in grub (I have wait for 30 minutes). Grub menu.lst has been correctly loaded, so iBFT BIOS firmware should be OK. Generally, the problems are: 1. For manual install (via DVD or via simple ks file with repo only), initramfs setup incorrect iscsi session, it should respect "iscsiadm -m fw" information and setup NIC correctly. 2. For kickstart option, it cannot work on this server. BTW, I also have a workstation with iBFT NIC only, it installed via DVD (Intel NIC don't support iBFT and PXE at the same time) and works well via manual install. It's kernel option is === iscsi_firmware ip=ibft ifname=eth0:d8:d3:85:80:a1:c5 ===
(In reply to comment #6) > === > The kernel option after installation seems correct now: > === > iscsi_firmware ip=ibft ifname=eth0:00:15:17:c8:c3:da > === If the generated line is correct than it seems as problem of dracut. Could you please attach logs of the installation generating supposedly correct dracut parameters? /tmp/syslog, /tmp/anaconda.log, /tmp/storage.log, /tmp/ks.cfg, /tmp/program.log, also grub.conf generated at the end of the installation and output of "iscsiadm -m -fw" from installer environment could be useful, I am not sure if output of "iscsiadm -m session -P 1" not having hw address is ok. Thanks. > But I got a black screen after press "b" (for boot) in grub (I have wait for 30 > minutes). Grub menu.lst has been correctly loaded, so iBFT BIOS firmware should > be OK. > Generally, the problems are: > 1. For manual install (via DVD or via simple ks file with repo only), > initramfs setup incorrect iscsi session, it should respect "iscsiadm -m fw" > information and setup NIC correctly. Automatic default setup is not supported when using multiple network devices during installation (e.g. combining regular and iBFT/iSCSI network device). Also configuration in UI is very limited in this case (see https://bugzilla.redhat.com/show_bug.cgi?id=710366#c5).
Created attachment 517748 [details] logfiles for iBFT with correct kickstart.
(In reply to comment #7) > Automatic default setup is not supported when using multiple network devices > during installation (e.g. combining regular and iBFT/iSCSI network device). > Also configuration in UI is very limited in this case (see > https://bugzilla.redhat.com/show_bug.cgi?id=710366#c5). That's acceptable, but anaconda setup iscsi session via incorrect NIC, that's a BUG. If anaconda don't support manual install with mixed NICs, we could disable iBFT when found iBFT NIC mix with regular NIC. Return nothing is better than return incorrect data. iscsiadm -m fw output with correct kickstart file: ====== sh-4.1# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.66.87.176 0.0.0.0 255.255.255.240 U 1 0 0 eth0 10.66.86.0 0.0.0.0 255.255.254.0 U 1 0 0 eth2 0.0.0.0 10.66.87.254 0.0.0.0 UG 0 0 0 eth2 sh-4.1# iscsiadm -m fw # BEGIN RECORD 2.0-872 iface.initiatorname = iqn.1994-05.com.redhat:ibft-dell-pe2950-01 iface.hwaddress = 00:15:17:c8:c3:da iface.bootproto = STATIC iface.ipaddress = 10.66.87.185 iface.subnet_mask = 255.255.255.240 iface.gateway = 0.0.0.0 iface.vlan = 0 iface.net_ifacename = eth0 node.name = iqn.1994-04.com.redhat:nay.hds.2100.iscsid3 node.conn[0].address = 10.66.87.184 node.conn[0].port = 3260 node.boot_lun = 00000000 # END RECORD ======
(In reply to comment #6) > === > The kernel option after installation seems correct now: > === > iscsi_firmware ip=ibft ifname=eth0:00:15:17:c8:c3:da > === > But I got a black screen after press "b" (for boot) in grub (I have wait for 30 > minutes). Grub menu.lst has been correctly loaded, so iBFT BIOS firmware should > be OK. > I can't see anything suspicious in anaconda logs. We need to debug dracut, could you please try to boot using this: http://fedoraproject.org/wiki/How_to_debug_Dracut_problems#Identifying_your_problem_area and this: http://fedoraproject.org/wiki/How_to_debug_Dracut_problems#Network_root_device_related_problems
Radek, sorry for late. I tried the rdshell kernel option for it. And comfirmed there is no "quiet" or "rhgb" kernel option. Still got nothing (neither console nor KVM )after I press B in grub. Any idea?
(In reply to comment #12) > The server is dell-pe2950-01.rhts.eng.nay.redhat.com > Let me know if you need the check it by yourself. It seems I'll have to, though I have no other post-reboot debugging ideas and I'll probably need some guidance for beaker. One more question about your ibft configuration: (In reply to comment #9) > sh-4.1# iscsiadm -m fw > # BEGIN RECORD 2.0-872 > iface.initiatorname = iqn.1994-05.com.redhat:ibft-dell-pe2950-01 > iface.hwaddress = 00:15:17:c8:c3:da > iface.bootproto = STATIC > iface.ipaddress = 10.66.87.185 > iface.subnet_mask = 255.255.255.240 > iface.gateway = 0.0.0.0 Is the gateway setting correct? I can imagine anaconda using non-ibft eth2 during installation to reach the target and then after reboot, where only eth0 would go up based on "iscsi_firmware ip=ibft ifname=eth0:00:15:17:c8:c3:da" dracut options routing may fail.
No route needed for iscsi. both target and ibft is in same subnet. initiator ip address: 10.66.87.185 target ip address: 10.66.87.184 The line: == iface.gateway = 0.0.0.0 == you found in that output is because I didn't setup the gateway in BIOS firmware and leave it as blank. That server isn't owned by storage-qe. I will setup it up once it back to me. Sorry for the late reply.
After adding "edd=off" into kernel option, OS boot up correctly. 1. Is it a linux kernel, Dell BIOS or Intel iBFT firmware issue? I tried edd=on and earlyprintk=ttyS0, but still got no valuable log, might be BIOS issue. 2. For incorrect kernel option with multiple NICs for manually install, if you think that is not supported, we can close this bug or make it as a document only bug.
After I add edd=off into boot kernel option, even with incorrect kernel iscsi option (point to pxe NIC instead of ibft NIC), OS still can boot up. Just like anaconda initramfs, OS initramfs setup iscsi session on PXE NIC. You still think the current incorrect but worked way is not a bug or we might fix it in the coming future?
From the point of network configuration this is not a bug. If you want to add some kind support for automatic configuration for multiple NICs with iBFT/iSCSI, please open a new RFE bug with clear description and use case of your request. Perhaps this RFE for iscsi interface binding: https://bugzilla.redhat.com/show_bug.cgi?id=500273 meets your requirements. Without the interface binding network layer decides which device is used to access iscsi and anaconda doesn't support configuration of routing (except for --nodefroute option useful in some cases of multiple NICs/iSCSI cases, i.e. using separate subnets). Also for other issues (wrong kernel boot options generated by anaconda?) please open a separate bug (one issue per bug) with description and logs so that it can be also reproduced/verified by QE. Thanks.
(In reply to comment #16) > After I add edd=off into boot kernel option, even with incorrect kernel iscsi > option (point to pxe NIC instead of ibft NIC), OS still can boot up. > I am seeing this too with some iSCSI cards.This is either a kernel problem or a bios problem.
(In reply to comment #17) > From the point of network configuration this is not a bug. If you want to add > some kind support for automatic configuration for multiple NICs with > iBFT/iSCSI, please open a new RFE bug with clear description and use case of > your request. Perhaps this RFE for iscsi interface binding: > https://bugzilla.redhat.com/show_bug.cgi?id=500273 meets your requirements. > Without the interface binding network layer decides which device is used to > access iscsi and anaconda doesn't support configuration of routing (except for > --nodefroute option useful in some cases of multiple NICs/iSCSI cases, i.e. > using separate subnets). Yes, it look like bug #500273 is what I need. I will wait and check whether bug #500273 fix the NIC iscsi binding issue or not. > > Also for other issues (wrong kernel boot options generated by anaconda?) please > open a separate bug (one issue per bug) with description and logs so that it > can be also reproduced/verified by QE. For this, I will submit a new for bug for this once Bug #500273 fixed. > > Thanks. Close this bug as not a bug.