Created attachment 898352 [details] logs_from_host_with_failed_install Description of problem: if engine uses VLAN tagged management network (rhevm) and host is being added over VLAN tagged interface using DHCP to obtain its IP, host install fails Reproducer: 1) data center with bridgeless, VLAN tagged (I used VLANID=172) rhevm 2) host: install rhel6.5 set up interface em1.172 with DHCP cat /etc/sysconfig/network-scripts/ifcfg-em1 DEVICE="em1" BOOTPROTO="none" IPV6INIT="no" MTU="1500" NM_CONTROLLED="no" ONBOOT="yes" TYPE="Ethernet" cat /etc/sysconfig/network-scripts/ifcfg-em1.172 DEVICE=em1.172 ONBOOT=yes VLAN=yes BOOTPROTO=dhcp DEFROUTE=yes NM_CONTROLLED=no 3) add host to rhevm result: engine loses connectivity to host since the host loses IP Version-Release number of selected component (if applicable): Red Hat Enterprise Virtualization Manager Version: 3.4.0-0.21.el6ev How reproducible: 100% Actual results: install fails Expected results: install successful Additional info: there seem to be 2 problems 1) first is udev spawning dhclient for newly defined em1.172, this can be solved by adding HOTPLUG=no in ifcfg file apuimendo already posted a patch for this http://gerrit.ovirt.org/27980 cat /etc/sysconfig/network-scripts/ifcfg-em1.172 DEVICE=em1.172 ONBOOT=yes VLAN=yes BOOTPROTO=dhcp DEFROUTE=yes NM_CONTROLLED=no HOTPLUG=no 2) second issue seems to be that if dhclient is already running on the host, host fails to obtain address (see logs bellow) logs: MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,105::ifcfg::537::root::(writeConfFile) Writing to file /etc/sysconfig/network-scripts/ifcfg-em1.172 configuration: # Generated by VDSM version 4.14.7-2.el6ev DEVICE=em1.172 ONBOOT=yes VLAN=yes BOOTPROTO=dhcp DEFROUTE=yes NM_CONTROLLED=no MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,169::utils::642::root::(execCmd) '/sbin/ip route show to 0.0.0.0/0 table all' (cwd None) MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,171::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,175::utils::642::root::(execCmd) '/sbin/ip -d link show dev em1.172' (cwd None) MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,177::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,179::utils::642::root::(execCmd) '/sbin/ifup em1.172' (cwd None) MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,207::utils::662::root::(execCmd) FAILED: <err> = 'dhclient(7333) is already running - exiting. \n\nThis version of ISC DHCP is based on the release available\non ftp.isc.org. Features have been added and other changes\nhave been made to the base software release in order to make\nit work better with this distribution.\n\nPlease report for this software via the Red Hat Bugzilla site:\n http://bugzilla.redhat.com\n\nexiting.\n'; <rc> = 1 MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,207::utils::642::root::(execCmd) '/sbin/ifdown em1.172' (cwd None) sourceRoute::DEBUG::2014-05-21 13:13:16,315::sourceRouteThread::20::root::(process_IN_CLOSE_WRITE_filePath) Responding to DHCP response in /var/run/vdsm/sourceRoutes/1400670796 sourceRoute::INFO::2014-05-21 13:13:16,316::sourceRoute::175::root::(remove) Removing gateway - device: em1.172 sourceRoute::DEBUG::2014-05-21 13:13:16,316::utils::642::root::(execCmd) '/sbin/ip rule' (cwd None) sourceRoute::DEBUG::2014-05-21 13:13:16,326::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 sourceRoute::ERROR::2014-05-21 13:13:16,326::sourceRoute::162::root::(_getRules) Routing rules not found for device em1.172 MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,394::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|Thread-13::INFO::2014-05-21 13:13:16,395::ifcfg::339::root::(restoreAtomicNetworkBackup) Rolling back logical networks configuration (restoring atomic logical networks backup) MainProcess|Thread-13::INFO::2014-05-21 13:13:16,395::ifcfg::379::root::(restoreAtomicBackup) Rolling back configuration (restoring atomic backup) MainProcess|Thread-13::INFO::2014-05-21 13:13:16,395::ifcfg::388::root::(restoreAtomicBackup) Restored /etc/sysconfig/network-scripts/ifcfg-em1.172 MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,395::utils::642::root::(execCmd) '/sbin/ifup em1.172' (cwd None) MainProcess|Thread-13::DEBUG::2014-05-21 13:13:16,510::utils::662::root::(execCmd) FAILED: <err> = '/etc/sysconfig/network-scripts/ifup-eth: line 282: 9426 Terminated /sbin/dhclient ${DHCLIENTARGS} ${DEVICE}\n'; <rc> = 1 MainProcess|Thread-13::ERROR::2014-05-21 13:13:16,511::ifcfg::431::root::(_startDevices) Failed to ifup device em1.172 during rollback. Traceback (most recent call last): File "/usr/share/vdsm/netconf/ifcfg.py", line 428, in _startDevices ifup(dev) File "/usr/share/vdsm/netconf/ifcfg.py", line 789, in ifup rc, out, err = _ifup(iface) File "/usr/share/vdsm/netconf/ifcfg.py", line 778, in _ifup out[-1] if out else '') ConfigNetworkError: (29, 'Determining IP information for em1.172... failed.') MainProcess|Thread-13::ERROR::2014-05-21 13:13:16,511::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks Traceback (most recent call last): File "/usr/share/vdsm/supervdsmServer", line 98, in wrapper res = func(*args, **kwargs) File "/usr/share/vdsm/supervdsmServer", line 202, in setupNetworks return configNetwork.setupNetworks(networks, bondings, **options) File "/usr/share/vdsm/configNetwork.py", line 651, in setupNetworks implicitBonding=True, **d) File "/usr/share/vdsm/configNetwork.py", line 186, in wrapped return func(*args, **kwargs) File "/usr/share/vdsm/configNetwork.py", line 288, in addNetwork netEnt.configure(**options) File "/usr/share/vdsm/netmodels.py", line 123, in configure self.configurator.configureVlan(self, **opts) File "/usr/share/vdsm/netconf/ifcfg.py", line 89, in configureVlan ifup(vlan.name, vlan.ipConfig.async) File "/usr/share/vdsm/netconf/ifcfg.py", line 789, in ifup rc, out, err = _ifup(iface) File "/usr/share/vdsm/netconf/ifcfg.py", line 778, in _ifup out[-1] if out else '') ConfigNetworkError: (29, 'Determining IP information for em1.172... failed.') sourceRoute::DEBUG::2014-05-21 13:13:16,522::sourceRouteThread::20::root::(process_IN_CLOSE_WRITE_filePath) Responding to DHCP response in /var/run/vdsm/sourceRoutes/1400670796 sourceRoute::INFO::2014-05-21 13:13:16,526::sourceRouteThread::37::root::(process_IN_CLOSE_WRITE_filePath) interface em1.172 is not a libvirt interface
This race is as old as vdsm networking, no reason for it to block rhev-3.4.0.
verified VDSM version 4.16.0-1.el6_5
Removing clone request, as the issue is already merged to upstream stable branch http://gerrit.ovirt.org/#/c/28438/
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-0159.html