Bug 961184

Summary: iface-start failed to create (start) interface eth0 unspecified error
Product: Red Hat Enterprise Linux 7 Reporter: Jincheng Miao <jmiao>
Component: netcfAssignee: Laine Stump <laine>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.0CC: acathrow, bili, dyuan, gsun, honzhang, mzhan
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: netcf-0.2.3-6.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 11:15:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jincheng Miao 2013-05-09 04:24:08 UTC
Created attachment 745519 [details]
the libvirtd log file

Description of problem:
virsh iface-start eth0 error like this:
error: internal error failed to create (start) interface eth0: unspecified error - interface eth0 failed to become active - possible disconnected cable.

Version-Release number of selected component (if applicable):
libvirt-1.0.5-1.el7.x86_64
qemu-kvm-1.4.0-4.el7.x86_64
kernel-3.9.0-0.55.el7.x86_64

How reproducible:
100%

Steps to Reproduce:

1. stop NetworkManager.
[root@localhost ~]# service NetworkManager stop


2. save the config file of eth0.
[root@localhost ~]# cp /etc/sysconfig/network-scritps/ifcfg-eth0 /tmp


3. prepare ether.xml
<interface type='ethernet' name='eth0'>
  <start mode='onboot'/>
  <protocol family='ipv4'>
    <ip address='10.66.4.54' prefix='23'/>
  </protocol>
</interface>


4. destroy & undefine eth0 and define a new one.

[root@localhost ~]# virsh iface-destroy eth0 
Interface eth0 destroyed

[root@localhost ~]# virsh iface-undefine eth0 
Interface eth0 undefined

[root@localhost ~]# virsh iface-define ether.xml
Interface eth0 defined from ether.xml

[root@localhost ~]# virsh iface-start eth0
error: Failed to start interface eth0
error: internal error failed to create (start) interface eth0: unspecified error - interface eth0 failed to become active - possible disconnected cable.


Expected results:
eth0 start successfully

Actual results:
eth0 start fail

Additional info:
Test it in single ethernet card machine.

libvirtd.log said:
2013-05-09 04:02:40.377+0000: 5129: error : netcfInterfaceCreate:637 : internal error failed to create (start) interface eth0: unspecified error - interface eth0 failed to become active - possible disconnected cable.

Comment 2 Jincheng Miao 2013-05-09 11:37:48 UTC
This bug cannot reproduce in dual ethernet card machine.

Comment 3 Laine Stump 2013-05-14 15:46:46 UTC
libvirt (actually netcf, which is being called by libvirt) is running the command "/sbin/ifup eth0", then checking the interface flags of eth0 to assure that both IFF_UP and IFF_RUNNING are set. If they aren't, the interface isn't seen as "not active", and the interface start is considered to have failed.

Originally netcf only checked for IFF_UP, but the behavior was changed in response to Bug 844578.

Please re-run your test, and when you see the message, get the output of "ifconfig eth0" - either UP or RUNNING will be missing from the output. We need to determine which, why, and whether or not that is proper behavior. (As far as I know, any network interface that is operating properly should have both UP and RUNNING; if the interface is working, but one of those flags is missing, that is likely an error in the kernel).

Comment 6 Laine Stump 2013-05-15 19:08:22 UTC
Can you install the scratch build below and re-test? I've put in the wait loop that I described. Hopefully that will solve the problem:

https://brewweb.devel.redhat.com/taskinfo?taskID=5778893

Comment 7 Jincheng Miao 2013-05-16 02:44:16 UTC
I've test netcf-libs-0.2.3-4.bz961184.el7.x86_64.rpm, and it works well when iface-start. 
So it seems delay before if_is_active check is a good choice. 
But I am not sure about this delay 250ms is suitable for other computers. Maybe some more information should be considered to determinate the time taken by this delay.

Comment 8 Laine Stump 2013-05-16 05:08:23 UTC
It isn't a simple 250msec wait. The status of the flags will be checked *immediately* after return from ifup, and there will be no wait if its already UP+RUNNING - this means that exactly the old behavior is retained in any case that would currently be successful. If the flags aren't both set after this initial check, *then* we wait 250msec and try again. We then repeat those last two steps until we do see UP+RUNNING (up to 10 times maximum). So at worst there will be an extra 2.5 second wait before finally returning an error. But if UP and RUNNING both are set before the full 2.5 seconds, it will exit early.

I've pushed the following patch to upstream netcf, so it will be in the next release:


commit 14af66fa2b119f47a23c9a4043ae8fe2441379fc
Author: Laine Stump <laine>
Date:   Wed May 15 14:13:09 2013 -0400

    wait for IFF_UP and IFF_RUNNING after calling ifup
    
    This fixes https://bugzilla.redhat.com/show_bug.cgi?id=961184
    
    Apparently one or the other of IFF_UP and IFF_RUNNING are not always
    set by the time /sbin/ifup returns control to netcf, so the subsequent
    check to verify that the interface is up may fail. This patch adds a
    loop to re-check the status of the interface every 250msec for up to
    2.5 seconds (or until both flags are set). If timeout is reached, it
    still fails the operation.

Comment 12 Jincheng Miao 2014-01-24 05:22:13 UTC
This fix is verified as below:
# rpm -q netcf
netcf-0.2.3-6.el7.x86_64

# service NetworkManager stop

# cp /etc/sysconfig/network-scritps/ifcfg-eno1 /tmp

prepare ether.xml
<interface type='ethernet' name='eno1'>
  <start mode='onboot'/>
  <protocol family='ipv4'>
    <ip address='10.66.4.54' prefix='23'/>
  </protocol>
</interface>

# virsh iface-destroy eno1 
Interface eno1 destroyed

# virsh iface-undefine eno1 
Interface eno1 undefined

# virsh iface-define ether.xml
Interface eno1 defined from ether.xml

# virsh iface-start eno1
Interface eno1 started

So I change the status to VERIFIED.

Comment 13 Ludek Smid 2014-06-13 11:15:29 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.