Bug 1525523

Summary: 4.14.6-300.fc27 Breaks Ethernet
Product: [Fedora] Fedora Reporter: Marcel Ziswiler <marcel>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 27CC: airlied, ajax, bskeggs, ewk, hdegoede, ichavero, itamar, jarodwilson, jcline, jeremy, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, mchehab, mjg59, sixpack13, steved
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-4.14.8-200.fc26 kernel-4.14.8-300.fc27 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-24 20:55:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marcel Ziswiler 2017-12-13 13:53:08 UTC 
Description of problem:

Update to Linux kernel 4.14.4-200.fc26 broke Ethernet on my Lenovo ThinkPad T470p. If connected to a managed switch like a HP 1810-8G or 1820-8G the link goes down/up every 15 seconds or so without ever being able to establish a stable link. However connecting to a cheapo ZyXEL ES-108A switch seems to work albeit taking a long time to establish the link.

Version-Release number of selected component (if applicable):

Fedore 26 fully up-to-date as of today
kernel-4.14.4-200.fc26

How reproducible:
Plug-in Ethernet cable connecting to managed switch but link won't ever be established.

Steps to Reproduce:
1. Update to kernel-4.14.4-200.fc26
2. Plug-in Ethernet cable
3. Notice no link ever being established

Actual results:
e1000e: enp0s31f6 NIC Link is Down

Expected results:
e1000e: enp0s31f6 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
e.g. as with previous kernel-4.13.16-202.fc26

Additional info:
Dez 13 14:07:53 localhost.localdomain kernel: e1000e: enp0s31f6 NIC Link is Down
Dez 13 14:07:53 localhost.localdomain avahi-daemon[1050]: Leaving mDNS multicast group on interface devnet.IPv4 with address 192.168.10.1.
Dez 13 14:07:53 localhost.localdomain dhcpd[11686]: receive_packet failed on devnet: Network is down
Dez 13 14:07:53 localhost.localdomain avahi-daemon[1050]: Withdrawing address record for 192.168.10.1 on devnet.
Dez 13 14:07:53 localhost.localdomain NetworkManager[1450]: <info>  [1513170473.2111] device (devnet): set-hw-addr: set MAC address to C8:5B:76:DF:3C:48 (vlan-parent)
Dez 13 14:07:53 localhost.localdomain NetworkManager[1450]: <info>  [1513170473.2114] device (enp0s31f6): set-hw-addr: set-cloned MAC address to C8:5B:76:DF:3C:48 (C8:5B:76:DF:3C:48)
Dez 13 14:07:53 localhost.localdomain avahi-daemon[1050]: Joining mDNS multicast group on interface devnet.IPv4 with address 192.168.10.1.
Dez 13 14:07:53 localhost.localdomain kernel: IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
Dez 13 14:07:53 localhost.localdomain kernel: IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
Dez 13 14:07:53 localhost.localdomain avahi-daemon[1050]: New relevant interface devnet.IPv4 for mDNS.
Dez 13 14:07:53 localhost.localdomain avahi-daemon[1050]: Registering new address record for 192.168.10.1 on devnet.IPv4.
Dez 13 14:07:53 localhost.localdomain NetworkManager[1450]: <info>  [1513170473.4380] device (enp0s31f6): state change: prepare -> config (reason 'none', internal state 'managed')
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=filter family=2 entries=134
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=nat family=2 entries=80
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=mangle family=2 entries=52
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=raw family=2 entries=41
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=filter family=10 entries=124
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=nat family=10 entries=74
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=mangle family=10 entries=51
Dez 13 14:07:53 localhost.localdomain audit: NETFILTER_CFG table=raw family=10 entries=42
Dez 13 14:07:53 localhost.localdomain NetworkManager[1450]: <info>  [1513170473.4573] device (enp0s31f6): state change: config -> ip-config (reason 'none', internal state 'managed')
Dez 13 14:07:53 localhost.localdomain NetworkManager[1450]: <info>  [1513170473.4579] dhcp4 (enp0s31f6): activation: beginning transaction (timeout in 45 seconds)
Dez 13 14:07:53 localhost.localdomain NetworkManager[1450]: <info>  [1513170473.4602] dhcp4 (enp0s31f6): dhclient started with pid 11717
Dez 13 14:07:53 localhost.localdomain kernel: IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
Dez 13 14:07:53 localhost.localdomain dhclient[11717]: DHCPREQUEST on enp0s31f6 to 255.255.255.255 port 67 (xid=0xbbb08a2e)
Dez 13 14:07:55 localhost.localdomain audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dez 13 14:07:57 localhost.localdomain NetworkManager[1450]: <info>  [1513170477.3408] device (enp0s31f6): state change: ip-config -> unavailable (reason 'carrier-changed', internal state 'managed')
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=filter family=2 entries=137
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=nat family=2 entries=82
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=mangle family=2 entries=53
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=raw family=2 entries=42
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=filter family=10 entries=127
Dez 13 14:07:57 localhost.localdomain NetworkManager[1450]: <info>  [1513170477.3573] dhcp4 (enp0s31f6): canceled DHCP transaction, DHCP client pid 11717
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=nat family=10 entries=76
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=mangle family=10 entries=52
Dez 13 14:07:57 localhost.localdomain audit: NETFILTER_CFG table=raw family=10 entries=43
Dez 13 14:07:57 localhost.localdomain NetworkManager[1450]: <info>  [1513170477.3573] dhcp4 (enp0s31f6): state changed unknown -> done
Dez 13 14:07:57 localhost.localdomain avahi-daemon[1050]: Interface devnet.IPv4 no longer relevant for mDNS.
...
above sequence repeats every 15 to 20 seconds or so
Comment 1 sixpack13 2017-12-13 20:47:38 UTC 
this e1000e-bug is in kernels since 4.14.3 

https://bugzilla.kernel.org/show_bug.cgi?id=198047

it was discussed on LKML too:
https://marc.info/?t=151242549500005&r=1&w=2

but without changes to kernel-stable so far.

this patch fixed it:
https://marc.info/?l=linux-kernel&m=151272209903675&w=2

it's for kernel 4.4.x but it works for 4.14.x too !
Comment 2 Marcel Ziswiler 2017-12-18 10:56:27 UTC 
Still not fixed in kernel-4.14.5-200.fc26.x86_64 while e.g. arch fixed it a couple days ago. Come on!
Comment 3 Marcel Ziswiler 2017-12-20 08:50:06 UTC 
Another kernel update but issue persists!

[  103.710700] e1000e: enp0s31f6 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
[  103.710784] IPv6: ADDRCONF(NETDEV_CHANGE): enp0s31f6: link becomes ready
[  103.711131] IPv6: ADDRCONF(NETDEV_CHANGE): devnet: link becomes ready
[  103.835433] e1000e: enp0s31f6 NIC Link is Down
[  104.054478] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  104.054619] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  104.060431] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  104.064651] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  104.097883] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  104.110262] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready

[user@host ~]$ uname -a
Linux localhost.localdomain 4.14.6-200.fc26.x86_64 #1 SMP Thu Dec 14 15:32:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Comment 4 Marcel Ziswiler 2017-12-20 13:11:11 UTC 
Upgrade to Fedora 27 did not help neither!

[  315.351538] e1000e: enp0s31f6 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
[  315.351606] IPv6: ADDRCONF(NETDEV_CHANGE): enp0s31f6: link becomes ready
[  315.351955] IPv6: ADDRCONF(NETDEV_CHANGE): devnet: link becomes ready
[  315.473523] e1000e: enp0s31f6 NIC Link is Down
[  315.687285] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  315.687441] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  315.728473] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  315.739121] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  319.725904] e1000e: enp0s31f6 NIC Link is Down
[  319.942916] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  319.943877] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  325.335359] e1000e: enp0s31f6 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
[  325.335434] IPv6: ADDRCONF(NETDEV_CHANGE): enp0s31f6: link becomes ready
[  325.335782] IPv6: ADDRCONF(NETDEV_CHANGE): devnet: link becomes ready
[  325.444692] e1000e: enp0s31f6 NIC Link is Down
[  325.663094] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  325.663384] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  325.707638] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  329.728164] e1000e: enp0s31f6 NIC Link is Down
[  329.942902] IPv6: ADDRCONF(NETDEV_UP): enp0s31f6: link is not ready
[  329.943624] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready
[  329.951287] IPv6: ADDRCONF(NETDEV_UP): devnet: link is not ready

[user@host ~]$ uname -a
Linux localhost.localdomain 4.14.6-300.fc27.x86_64 #1 SMP Thu Dec 14 15:31:24 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Comment 5 Marcel Ziswiler 2017-12-20 14:44:51 UTC 
Issue persists even in rawhide!

[user@host ~]$ uname -a
Linux localhost.localdomain 4.15.0-0.rc4.git1.2.fc28.x86_64 #1 SMP Tue Dec 19 17:24:17 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Looks like it might only exhibit itself on certain Intel Ethernet controller chip variants:

[user@host ~]$ lspci | grep Ethernet
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (5) I219-V (rev 31)
Comment 6 Jeremy Cline 2017-12-20 15:52:46 UTC 
The next builds for the kernel should include the fix, likely v4.14.8 for Fedora 27 and 26. The rawhide kernel should have it after today's build.
Comment 7 Fedora Update System 2017-12-21 13:10:28 UTC 
kernel-4.14.8-300.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-1ebb87e7c0
Comment 8 Fedora Update System 2017-12-21 13:12:06 UTC 
kernel-4.14.8-200.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-7810b7c59f
Comment 9 Fedora Update System 2017-12-21 20:27:08 UTC 
kernel-4.14.8-300.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-1ebb87e7c0
Comment 10 Fedora Update System 2017-12-21 21:11:52 UTC 
kernel-4.14.8-200.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-7810b7c59f
Comment 11 Marcel Ziswiler 2017-12-22 08:48:32 UTC 
Wow, amazing! Rawhide now has it fixed (;-p).

[user@host ~]$ uname -a
Linux localhost.localdomain 4.15.0-0.rc4.git3.2.fc28.x86_64 #1 SMP Thu Dec 21 16:54:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Comment 12 Fedora Update System 2017-12-24 20:55:41 UTC 
kernel-4.14.8-200.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
Comment 13 Fedora Update System 2017-12-24 21:18:02 UTC 
kernel-4.14.8-300.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
Comment 14 sixpack13 2018-01-05 22:08:52 UTC 
since kernel-4.14.8-300.fc27 this bug is fixed (for me).
Thanks !
Comment 15 Marcel Ziswiler 2018-01-06 00:10:11 UTC 
Yes, since kernel-4.14.8-300.fc27 this bug is indeed fixed again. Thanks!