Bug 1266215

Summary: %post script in kickstart, on HP Gen9, includes non-chroot /etc/sysconfig/network-scripts for network commands
Product: Red Hat Enterprise Linux 6 Reporter: jcastran
Component: initscriptsAssignee: David Kaspar // Dee'Kej <deekej>
Status: CLOSED INSUFFICIENT_DATA QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.5CC: danw, dcbw, deekej, jruemker, rkhan, thaller
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Release Note
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-12 17:46:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1269194, 1356047    
Attachments:
Description Flags
sosreport
none
Customer Kickstart
none
Customer Network Script Called in %post. Script completes, then install pauses none

Description jcastran 2015-09-24 19:24:02 UTC
Created attachment 1076687 [details]
sosreport

Description of problem:
   In the kickstart, user defines a static ip for eth0 and installs the system. In a %post script, user creates new ifcfg files for required network configuration:
    bond0   slaves=eth0,eth1

ifcfg-eth0, in chrooted environment does not contain any ip information, only that it is a slave.

After newly created files, running "service network restart" shows it interacting with more interfaces then normal. Additionally, eth0 is set as a slave and is incorrectly given its IP from the anaconda install. Any attempt to reset the interfaces leaves eth0 with its old IP and the default gateway sticks to eth0 (instead of going to bond0).

Running "ip route del default" will remove the eth0 default gateway.
Running "ifup bond0" will then correctly set the default gateway to bond0.

Resetting the environment with the following commands, while in chroot, will cause the "service network restart" and "ifup,ifdown" commands to work properly and no longer apply an IP to eth0

   . /root/.bashrc
   . /root/.bash_profile

Rebooting the system resolves these issues and network works as it should. This does not occur on the customers Gen8 hardware. Only gen9.


Version-Release number of selected component (if applicable):
rhel 6.5


---Steps to Reproduce---   
This is post script environment. system mounted at /mnt/sysimage. %post runs in chrooted environment. You must include script or pause install at post and "ctrl+alt+f2" to access terminal

0.service NetworkManager stop; chkconfig NetworkManager off
1.Use static ip during kickstart install for eth0
2.%post script recreates ifcfg-eth0 and ifcfg-bond0 files
3."service network restart"
4."route -n" (((Check Default gateway interface. It should be bond0, not eth0)))
5."ifdown eth0; ifup bond0" (((Default gateway does not update again)))
6."ip route del default"
7."ifup bond0" (((Default gateway is now bond0)))

Actual results:
Default Gateway does not update to correct bond0 interface and eth0 retains anaconda install IP.

Expected results:
"service network restart" brings all interfaces down and back up with correct configuration. eth0 should be a slave with no IP.

Additional info:
The workaround we discovered was to run these two commands after the network scripts finishes. (But before the oracle script)

   /sbin/ip route del default
   /sbin/ifup bond0

This will delete the default gateway linked to "eth0". Then when it brings up bond0, it will set the default gateway to the correct interface. Resetting networking or bringing up eth0 will result in the default gateway switching back to using eth0 and networking will fail.

Comment 1 jcastran 2015-09-24 19:25:02 UTC
Created attachment 1076688 [details]
Customer Kickstart

Comment 2 jcastran 2015-09-24 19:25:49 UTC
Created attachment 1076689 [details]
Customer Network Script Called in %post. Script completes, then install pauses

Comment 4 Brian Lane 2015-09-29 00:02:11 UTC
This isn't an Anaconda issue, and I'm not sure if doing this inside the %post chroot is expected to be reliable, moving to NetworkManager for comment.

Comment 5 Jirka Klimes 2015-09-29 08:48:11 UTC
> 0.service NetworkManager stop; chkconfig NetworkManager off

I am reassigning to initscripts, because NetworkManager was switched off and thus ifup should handle the networking.

For debugging, it may help to add "-x" option to bash [1] in /etc/sysconfig/network-scripts/ifup and /etc/sysconfig/network-scripts/ifup.

I can't see the bonding attempt in anaconda.syslog. Where are the logs suppose to be? 

[1] http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_03.html

Comment 6 jcastran 2015-10-06 13:35:42 UTC
I have requested the customer enable post script logging so we can get a better log. I did not find one myself in the sosreport

Specifically is there any command output or additional logs/debug you would like to see? I troubleshot this when the script hung by going to the second tty. I should be able to add any commands we ran or need to run to see the output. Then log that and attach it here.