Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1289341

Summary: Multi-NIC server fails to kickstart when only one NIC is cabled
Product: Red Hat Enterprise Linux 7 Reporter: Dylan Gross <dgross>
Component: dracutAssignee: dracut-maint
Status: CLOSED ERRATA QA Contact: Release Test Team <release-test-team-automation>
Severity: high Docs Contact:
Priority: high    
Version: 7.2CC: bbreard, dgross, dracut-maint-list, dsafford, harald, jdickers, jkurik, jotodd, jreznik, jruemker, jstodola, mbanas, mkolaja, mmatsuya, ohudlick, rvykydal, vanhoof
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1315709 (view as bug list) Environment:
Last Closed: 2016-11-04 08:02:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1203710, 1289485, 1313485, 1315709    

Description Dylan Gross 2015-12-07 21:39:34 UTC 
Description of problem:

  Kickstarting a server with multiple NICs where only one has a link, the first few un-cabled cards are configured with the network setup of the targeted card.

  Server with 4 (un-cabled) on-board NICs and one (cabled) 10Gb NIC.

  enp3s0f0
  enp3s0f1
  enp3s0f2
  enp3s0f3
  ens1     (This NIC is Cabled, with link - When finally started, produces error that "enp3s0f0" has the IP address.   Ip is applied to enp3s0f0, then dracut attempts to apply same static ip info to ens1 and fails, as it is already assigned.  



Version-Release number of selected component (if applicable):

  RHEL 7.2


How reproducible:


Steps to Reproduce:
1. kickstart multi-NIC server with only one NIC cabled using boot line args:    ipv6.disable ksdevice=link nameserver=<DNS> netmask=<NETMASK> ip=<IP Address> gateway=<Gateway> hostname=<Hostname>



Actual results:

   Drops to a dracut shell after an error about Duplicate IP being held by the first discovered (uncabled) device

Expected results:

   Cabled NIC is assigned the correct IP and boots.

Additional info:

  - This kickstart is happening against a Satellite 5.7 using a boot ISO.  No DHCP/pxe.

  --device=link was working previously, but no longer works with 7.2
  - The ksdevice was working in 7.0 but stopped working in 7.1
Comment 1 Ben Breard 2015-12-07 21:53:06 UTC 
For the same environment we've also tested the following: 


initrd=initrd-7.img inst.ks=http://[satellite]/cblr/svc/op/ks/profile/RHEL-7-Prod:1 ipv6.disable nameserver=10.10.17.10 ip=10.10.10.142::10.10.10.129:255.255.255.128:hostname:ens1:none


This allows the kickstart file to be downloaded, but towards the end of the install the network cannot start as all interfaces ifcfg files are configured as static with the same ip address.
Comment 2 Jason Dickerson 2015-12-07 22:22:40 UTC 
Addition to comment 1.  The build works, but the static ifcfg-ens1 file is not created correctly by the network include kickstart snippet written by the customer.  That snippet depends on the old syntax where ip, netmask, gw, hostname, are all separate variables.  This results in a built server with no network.  (I am not sure if the kickstart postinstall is completing correctly or not.)  However, the result is a server that can boot.
Comment 3 Radek Vykydal 2015-12-08 14:58:29 UTC 
Could you please post anaconda logs for the case from comment #1 ? Also the kickstart file used for installation. The logs we need are:

/tmp/anaconda.log
/tmp/syslog
/tmp/ifcfg.log
Comment 4 Radek Vykydal 2015-12-08 15:33:22 UTC 
I am not able to reproduce the issue from the Description. I see the duplicate IP message but after some time (waiting for link on unplugged device) dracut proceeds to installation.

Could you please attach rd.debug option and try to gather output of journalctl -a from dracut shell?
Comment 6 Radek Vykydal 2015-12-09 11:51:02 UTC 
I suppose the kickstart used is the one attached in the customer case.

(In reply to Dylan Gross from comment #5)
> Created attachment 1103710 [details]
> journalctl -a output from case 01550617

This is the output for this case:

ip=161.166.234.142::161.166.234.129:255.255.255.128:osel400639::none

I'd like to see also journalctl -a with rd.debug for the original problem from the description, ie

ksdevice=link nameserver=161.173.7.10 netmask=255.255.255.128 ip=161.166.234.142 gateway=161.166.234.129 hostname=osel400639
Comment 7 Radek Vykydal 2015-12-09 12:01:54 UTC 
(In reply to Dylan Gross from comment #5)
> Created attachment 1103710 [details]
> journalctl -a output from case 01550617

It seems that enp3s0f0 behaves as it actually has a link, so it gets the static address (before ens1). From the log:

Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@396(): do_static
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@156(do_static): strstr 161.166.234.142 '*:*:*'
Dec 08 14:39:43 localhost dracut-initqueue[735]: /lib/dracut-lib.sh@29(strstr): '[' 161.166.234.142 '!=' 161.166.234.142 ']'
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@158(do_static): linkup enp3s0f0
Dec 08 14:39:43 localhost dracut-initqueue[735]: /lib/net-lib.sh@607(linkup): wait_for_if_link enp3s0f0
Dec 08 14:39:43 localhost dracut-initqueue[735]: /lib/net-lib.sh@608(linkup): ip link set enp3s0f0 up
Dec 08 14:39:43 localhost dracut-initqueue[735]: /lib/net-lib.sh@609(linkup): wait_for_if_up enp3s0f0
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@163(do_static): ip route get 161.166.234.142
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@164(do_static): read a rest
Dec 08 14:39:43 localhost dracut-initqueue[735]: RTNETLINK answers: Network is unreachable
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@165(do_static): '[' '' = local ']'
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@169(do_static): return 0
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@172(do_static): '[' -n '' ']'
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@173(do_static): '[' -n '' ']'
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@174(do_static): strstr 161.166.234.142 '*:*:*'
Dec 08 14:39:43 localhost dracut-initqueue[735]: /lib/dracut-lib.sh@29(strstr): '[' 161.166.234.142 '!=' 161.166.234.142 ']'
Dec 08 14:39:43 localhost dracut-initqueue[735]: /sbin/ifup@179(do_static): arping -f -q -D -c 2 -I enp3s0f0 161.166.234.142
Dec 08 14:39:44 localhost kernel: floppy0: no floppy controllers found
Dec 08 14:39:44 localhost dracut-initqueue[735]: /sbin/ifup@183(do_static): ip addr flush dev enp3s0f0
Dec 08 14:39:44 localhost dracut-initqueue[735]: /sbin/ifup@184(do_static): ip addr add 161.166.234.142/255.255.255.128 brd + dev enp3s0f0

compared to other devices:

Dec 08 14:40:47 osel400639 dracut-initqueue[735]: /sbin/ifup@158(do_static): linkup enp3s0f1
Dec 08 14:40:47 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@607(linkup): wait_for_if_link enp3s0f1
Dec 08 14:40:47 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@608(linkup): ip link set enp3s0f1 up
Dec 08 14:40:47 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@609(linkup): wait_for_if_up enp3s0f1
Dec 08 14:41:06 osel400639 dracut-initqueue[735]: /sbin/ifup@159(do_static): warn 'Could not bring interface enp3s0f1 up!'
Dec 08 14:41:06 osel400639 dracut-initqueue[735]: /lib/dracut-lib.sh@60(warn): echo 'Warning: Could not bring interface enp3s0f1 up!'
Dec 08 14:41:06 osel400639 dracut-initqueue[735]: Warning: Could not bring interface enp3s0f1 up!
Dec 08 14:41:06 osel400639 dracut-initqueue[735]: /sbin/ifup@160(do_static): return 1

ec 08 14:41:07 osel400639 dracut-initqueue[735]: /sbin/ifup@158(do_static): linkup enp4s0f0
Dec 08 14:41:07 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@607(linkup): wait_for_if_link enp4s0f0
Dec 08 14:41:07 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@608(linkup): ip link set enp4s0f0 up
Dec 08 14:41:07 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@609(linkup): wait_for_if_up enp4s0f0
Dec 08 14:41:27 osel400639 dracut-initqueue[735]: /sbin/ifup@159(do_static): warn 'Could not bring interface enp4s0f0 up!'
Dec 08 14:41:27 osel400639 dracut-initqueue[735]: /lib/dracut-lib.sh@60(warn): echo 'Warning: Could not bring interface enp4s0f0 up!'
Dec 08 14:41:27 osel400639 dracut-initqueue[735]: Warning: Could not bring interface enp4s0f0 up!
Dec 08 14:41:27 osel400639 dracut-initqueue[735]: /sbin/ifup@160(do_static): return 1


Dec 08 14:41:28 osel400639 dracut-initqueue[735]: /sbin/ifup@396(): do_static
Dec 08 14:41:28 osel400639 dracut-initqueue[735]: /sbin/ifup@156(do_static): strstr 161.166.234.142 '*:*:*'
Dec 08 14:41:28 osel400639 dracut-initqueue[735]: /lib/dracut-lib.sh@29(strstr): '[' 161.166.234.142 '!=' 161.166.234.142 ']'
Dec 08 14:41:28 osel400639 dracut-initqueue[735]: /sbin/ifup@158(do_static): linkup enp4s0f1
Dec 08 14:41:28 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@607(linkup): wait_for_if_link enp4s0f1
Dec 08 14:41:28 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@608(linkup): ip link set enp4s0f1 up
Dec 08 14:41:28 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@609(linkup): wait_for_if_up enp4s0f1
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@159(do_static): warn 'Could not bring interface enp4s0f1 up!'
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /lib/dracut-lib.sh@60(warn): echo 'Warning: Could not bring interface enp4s0f1 up!'
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: Warning: Could not bring interface enp4s0f1 up!
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@160(do_static): return 1


Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@396(): do_static
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@156(do_static): strstr 161.166.234.142 '*:*:*'
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /lib/dracut-lib.sh@29(strstr): '[' 161.166.234.142 '!=' 161.166.234.142 ']'
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@158(do_static): linkup ens1
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@607(linkup): wait_for_if_link ens1
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@608(linkup): ip link set ens1 up
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /lib/net-lib.sh@609(linkup): wait_for_if_up ens1
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@163(do_static): ip route get 161.166.234.142
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@164(do_static): read a rest
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@165(do_static): '[' local = local ']'
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@166(do_static): warn 'Not assigning 161.166.234.142 to interface ens1, cause it is already assigned!'
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /lib/dracut-lib.sh@60(warn): echo 'Warning: Not assigning 161.166.234.142 to interface ens1, cause it is already assigned!'
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: Warning: Not assigning 161.166.234.142 to interface ens1, cause it is already assigned!
Dec 08 14:41:48 osel400639 dracut-initqueue[735]: /sbin/ifup@167(do_static): return 1

So even the one-string ip= configuration should work without specifying the device, given only one device has link.
That would require changing kickstart %pre script to parse this form of configuration though.

I suspect the multiple values configuration (ip= gateway= ...) is not working for the same reason, though we'd need to see the log requested in comment #6 to confirm.

Now the question is: is the ens1 really the only plugged device? And if so, why linkup on enp3s0f0 succeeds?
Comment 12 Radek Vykydal 2015-12-10 09:11:47 UTC 
From the logs in comment #9 I'd say both forms of boot options (multiple values with ksdevice, and ip= dracut boot option) fail due to the same root cause identified by Masahiro (great job!).

Reassigning to dracut based on comment #10.
Comment 25 Mike McCune 2016-03-28 22:31:02 UTC 
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions
Comment 26 Harald Hoyer 2016-06-20 12:30:21 UTC 
please verify, if this is fixing your problem:

http://people.redhat.com/harald/downloads/dracut/dracut-033-360.el7_2.2/
Comment 32 errata-xmlrpc 2016-11-04 08:02:01 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2530.html