Bug 1654505 - NetworkManager doesn't detect Ethernet connections
Summary: NetworkManager doesn't detect Ethernet connections
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 29
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-28 22:21 UTC by bedny
Modified: 2019-02-21 21:07 UTC (History)
27 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-02-21 21:07:00 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
attached output of - uname -mr ifconfig -a nm-tool route -n cat /var/lib/NetworkManager/NetworkManager.state cat /etc/NetworkManager/NetworkManager.conf cat /etc/network/interfaces dmesg | grep -i dh (7.22 KB, text/plain)
2018-11-28 22:21 UTC, bedny
no flags Details
Journalctl output booting 4.19.4 (245.46 KB, text/plain)
2018-12-01 13:13 UTC, malsworn_deliquesces
no flags Details
Log boot fedora 29 kernel 4.19.6 (324.71 KB, text/plain)
2018-12-09 15:35 UTC, malsworn_deliquesces
no flags Details

Description bedny 2018-11-28 22:21:14 UTC
Created attachment 1509696 [details]
attached output of -  uname -mr ifconfig -a nm-tool route -n cat /var/lib/NetworkManager/NetworkManager.state cat /etc/NetworkManager/NetworkManager.conf cat /etc/network/interfaces dmesg | grep -i dh

After upgrade Fedora 29 to kernel 4.19.2-300.fc29 NetworkManager doesn't see Ethernet connections. After the next upgrade to kernel 4.19.3-300.fc29 the connections are still not detected.

I have to boot with an older kernel version (4.18.17-300.fc29).

attached output of -

uname -mr
ifconfig -a
nm-tool
route -n
cat /var/lib/NetworkManager/NetworkManager.state
cat /etc/NetworkManager/NetworkManager.conf
cat /etc/network/interfaces
dmesg | grep -i dhclient
sudo ethtool eth0

Comment 1 Thomas Haller 2018-11-29 06:39:17 UTC
What means "Ethernet connections" here?

In NetworkManager speak, there are

  - connections (aka. profiles, connection profiles)
  - devices (networking interfaces)

What means "doesn't see" here and how did you get to that conclusion?

What's the output of 

  nmcli connection
  nmcli device
  ip link

?


Thanks.

Comment 2 bedny 2018-11-29 21:40:14 UTC
[Sergey@gerundy Документы]$ uname -a
Linux gerundy 4.19.3-300.fc29.x86_64 #1 SMP Wed Nov 21 15:27:25 UTC 2018 x86_64$
[Sergey@gerundy Документы]$ nmcli connection
NAME    UUID                                  TYPE	DEVICE
virbr0  8f2c91e3-4841-4369-85da-f5c166ea6d6f  bridge    virbr0
enp4s0  f7b394f3-83d8-33aa-8aa0-cc1b03419a4a  ethernet  --
enp5s0  b8733073-7a70-3b1f-a94e-97c5055eca80  ethernet  --
[Sergey@gerundy Документы]$ nmcli device
DEVICE      TYPE      STATE           CONNECTION
virbr0      bridge    подключено      virbr0
enp4s0      ethernet  недоступен      --
enp5s0      ethernet  недоступен      --
lo          loopback  без управления  --
virbr0-nic  tun       без управления  --
[Sergey@gerundy Документы]$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAUL$
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp4s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DO$
    link/ether 00:1f:d0:27:50:99 brd ff:ff:ff:ff:ff:ff
3: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DO$
    link/ether 00:1f:d0:27:50:9b brd ff:ff:ff:ff:ff:ff
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOW$
    link/ether 52:54:00:d3:2f:98 brd ff:ff:ff:ff:ff:ff
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 stat$
    link/ether 52:54:00:d3:2f:98 brd ff:ff:ff:ff:ff:ff
[Sergey@gerundy Документы]$



[Sergey@gerundy ~]$ uname -mr
4.19.4-300.fc29.x86_64 x86_64
[Sergey@gerundy ~]$ nmcli connection
NAME    UUID                                  TYPE	DEVICE
virbr0  d13817be-fb4e-4868-ad18-ffe2ada0b8e0  bridge    virbr0
enp4s0  f7b394f3-83d8-33aa-8aa0-cc1b03419a4a  ethernet  --
enp5s0  b8733073-7a70-3b1f-a94e-97c5055eca80  ethernet  --
[Sergey@gerundy ~]$ nmcli device
DEVICE      TYPE      STATE           CONNECTION
virbr0      bridge    подключено      virbr0
enp4s0      ethernet  недоступен      --
enp5s0      ethernet  недоступен      --
lo          loopback  без управления  --
virbr0-nic  tun       без управления  --
[Sergey@gerundy ~]$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAUL$
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp4s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DO$
    link/ether 00:1f:d0:27:50:99 brd ff:ff:ff:ff:ff:ff
3: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DO$
    link/ether 00:1f:d0:27:50:9b brd ff:ff:ff:ff:ff:ff
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOW$
    link/ether 52:54:00:d3:2f:98 brd ff:ff:ff:ff:ff:ff
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 stat$
    link/ether 52:54:00:d3:2f:98 brd ff:ff:ff:ff:ff:ff

Comment 3 bedny 2018-11-29 21:43:00 UTC
[Sergey@gerundy ~]$ uname -mr
4.19.4-300.fc29.x86_64 x86_64
[Sergey@gerundy ~]$ nmcli connection
NAME    UUID                                  TYPE	DEVICE
virbr0  d13817be-fb4e-4868-ad18-ffe2ada0b8e0  bridge    virbr0
enp4s0  f7b394f3-83d8-33aa-8aa0-cc1b03419a4a  ethernet  --
enp5s0  b8733073-7a70-3b1f-a94e-97c5055eca80  ethernet  --
[Sergey@gerundy ~]$ nmcli device
DEVICE      TYPE      STATE           CONNECTION
virbr0      bridge    подключено      virbr0
enp4s0      ethernet  недоступен      --
enp5s0      ethernet  недоступен      --
lo          loopback  без управления  --
virbr0-nic  tun       без управления  --
[Sergey@gerundy ~]$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAUL$
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp4s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DO$
    link/ether 00:1f:d0:27:50:99 brd ff:ff:ff:ff:ff:ff
3: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DO$
    link/ether 00:1f:d0:27:50:9b brd ff:ff:ff:ff:ff:ff
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOW$
    link/ether 52:54:00:d3:2f:98 brd ff:ff:ff:ff:ff:ff
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 stat$
    link/ether 52:54:00:d3:2f:98 brd ff:ff:ff:ff:ff:ff

Comment 4 Thomas Haller 2018-11-30 07:00:45 UTC
you have two ethernet profiles, f7b394f3-83d8-33aa-8aa0-cc1b03419a4a and b8733073-7a70-3b1f-a94e-97c5055eca80.

Assuming that these profiles are suitably configured for your environment, you need to activate them, for example with

  nmcli connection up b8733073-7a70-3b1f-a94e-97c5055eca80


It's not clear what you think is wrong here.




Profiles only automatically connect, if you configure them as such:

  nmcli connection show b8733073-7a70-3b1f-a94e-97c5055eca80 | grep autoco

  nmcli connection modify b8733073-7a70-3b1f-a94e-97c5055eca80 connection.autoconnect yes

usually that is enabled by default. So, if the profile does not automatically connect, it is probably wrongly configured. Can you activate them manually? (above).

Comment 5 malsworn_deliquesces 2018-11-30 07:18:44 UTC
I have the same problem when a new kernal is installed. When I first booted into 4.19.2 it said "cable not plugged in" and it wouldn't work until I booted into 4.18 first and then when I booted back to 4.19.2 it worked as expected. Today I installed 4.19.4 and the exact same problem occurred. Cable not plugged in and no amount of rebooting with 4.19.4 or 4.19.2 worked. I rebooted back into my latest 4.18 kernel and back to 4.19.4 and then my connection was working as intended. After booting into a 4.18 kernel once, I can reboot as many times with just the 4.19 kernel and it always automatically connects.

My profile always automatically connects, except the first time booting into these new kernels. I checked the setting "connection.autoconnect" and it is set to yes already.

The next time I install a new kernel, it will remove the 4.18 kernel from my boot menu and I won't be able to do this trick anymore to get internet back.

Output from the commands you posted above:
Note: these are after booting into the 4.18 kernel and back into 4.19.4, because otherwise I do not have internet to post to this bug.

~> nmcli connection
NAME    UUID                                  TYPE      DEVICE 
enp2s0  97099a1e-438d-3a71-aec6-ac3e14a38536  ethernet  enp2s0 
virbr0  41cb5aaa-7ac0-4575-aac9-955c136af07d  bridge    virbr0


~> nmcli device
DEVICE      TYPE      STATE      CONNECTION 
enp2s0      ethernet  connected  enp2s0     
virbr0      bridge    connected  virbr0     
lo          loopback  unmanaged  --         
virbr0-nic  tun       unmanaged  --


~> ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 40:61:86:99:87:ae brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:00:b2:c7:46 brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:00:b2:c7:46 brd ff:ff:ff:ff:ff:ff


~> nmcli connection show 97099a1e-438d-3a71-aec6-ac3e14a38536 | grep autoco
connection.autoconnect:                 yes

Comment 6 Thomas Haller 2018-11-30 07:43:02 UTC
in the output of `ip link`:

> 2: enp4s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state
>    link/ether 00:1f:d0:27:50:99 brd ff:ff:ff:ff:ff:ff

I missed that earlier. "NO-CARRIER" means kernel thinks that the cable is not plugged in.

NetworkManager cannot activate a DHCP profile on an interface that has no cable plugged in (or appears that way).

It looks like a kernel/driver issue. Reassigning.

Comment 7 Steve 2018-11-30 14:44:20 UTC
(In reply to Thomas Haller from comment #6)
...
> It looks like a kernel/driver issue. Reassigning.

None of these commands show the actual device driver:

  nmcli connection
  nmcli device
  ip link

In contrast "nmcli" (without arguments) does:

$ nmcli
enp3s0: connected to enp3s0
        "Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller"
        ethernet (r8169), [MAC address removed], hw, mtu 1500
...

Comment 8 Steve 2018-11-30 15:57:12 UTC
(In reply to d.vicesss from comment #5)
...
> The next time I install a new kernel, it will remove the 4.18 kernel from my
> boot menu and I won't be able to do this trick anymore to get internet back.
...

Make this change to /etc/dnf/dnf.conf:

#installonly_limit=3
installonly_limit=0

See "man dnf.conf" for details.

Also, could you post the output from this command, so developers can see the device driver you are using:

$ nmcli | head -3
(Obfuscate the MAC address, if you want.)

Comment 9 Steve 2018-11-30 16:24:14 UTC
(In reply to d.vicesss from comment #5)
...
> expected. Today I installed 4.19.4 and the exact same problem occurred.
> Cable not plugged in and no amount of rebooting with 4.19.4 or 4.19.2
> worked. I rebooted back into my latest 4.18 kernel and back to 4.19.4 and
> then my connection was working as intended. After booting into a 4.18 kernel
> once, I can reboot as many times with just the 4.19 kernel and it always
> automatically connects.
...

See if you can find the journalctl log for the first time you booted 4.19.4. That may take some sleuthing, but start with:

$ last | less

Use this command to get a list of journalctl boot IDs:

$ journalctl --list-boots

Attach the output from:

$ journalctl --no-hostname -b aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

The 32-character hex number is a boot ID from "journalctl --list-boots".

See "man journalctl" for more about the "--list-boots" and "-b" options.

(Review the journalctl log for anything you don't want posted, such as host name, user name, MAC addresses, IP addresses, etc.)

Comment 10 Steve 2018-12-01 00:40:57 UTC
(In reply to Steve from comment #8)
> (In reply to d.vicesss from comment #5)
...
> Also, could you post the output from this command, so developers can see the
> device driver you are using:
> 
> $ nmcli | head -3
> (Obfuscate the MAC address, if you want.)

No need. In Bug 1653006, Comment 2, you posted a link that has it:

enp2s0: connected to enp2s0
        "Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller"
        ethernet (r8169), [censored], hw, mtu 1500
...

See:

Bug 1650984 - RTL8111 NIC not working with kernel 4.19.2

Comment 11 malsworn_deliquesces 2018-12-01 13:13:37 UTC
Created attachment 1510387 [details]
Journalctl output booting 4.19.4

Sadly, "journalctl --list-boots" only lists boots until 16 november for some reason. I tracked the date and time with "last | less" and dove into journalctl myself and I think I found the right logs. It is a bit confusing because of timeskips, so I hope you can make sense of it.

Comment 12 Steve 2018-12-04 15:26:55 UTC
(In reply to d.vicesss from comment #11)
> Created attachment 1510387 [details]
> Journalctl output booting 4.19.4
> 
> Sadly, "journalctl --list-boots" only lists boots until 16 november for some
> reason. I tracked the date and time with "last | less" and dove into
> journalctl myself and I think I found the right logs. It is a bit confusing
> because of timeskips, so I hope you can make sense of it.

Your log shows:

... kernel: r8169 0000:02:00.0 eth0: RTL8168d/8111d, ..., XID 281000c0, IRQ 29

kernel-4.19.6-300.fc29 is available. Update with:

# dnf update kernel --refresh --enablerepo=updates-testing

See Bug 1650984, Comment 23.


BTW, the "timeskips" are normal. Initially the system clock is set according to the "real time clock" (RTC). Later, it is set according to your time zone setting. From your log:

... systemd[1]: RTC configured in localtime, applying delta of 60 minutes to system time.

To see your current time and date settings:

$ timedatectl

See the man pages for "timedatectl", "hwclock", and "chronyd".

Comment 13 malsworn_deliquesces 2018-12-05 07:12:22 UTC
Installing 4.19.6-200.fc28.x86_64 from the testing repo fixed the issue for me. I left some karma on bodhi too. Thank you for your help.

Partly off-topic question: will I now continuously get kernel updates from testing or only when I add --enablerepo=updates-testo? I would prefer stable kernels.

Comment 14 Steve 2018-12-05 16:20:33 UTC
(In reply to d.vicesss from comment #13)
> Installing 4.19.6-200.fc28.x86_64 from the testing repo fixed the issue for
> me. I left some karma on bodhi too. Thank you for your help.

Thanks for testing and for leaving karma.

> Partly off-topic question: will I now continuously get kernel updates from
> testing or only when I add --enablerepo=updates-testo? I would prefer stable
> kernels.

The "--enablerepo" option temporarily enables the repo. ("man dnf" doesn't make the clear.)

You can see all repos with:

# dnf repolist --all

The repo config files are in this now misnamed directory:

$ ls /etc/yum.repos.d/

The repo config files come from this package:

$ rpm -qf /etc/yum.repos.d/
fedora-repos-28-5.noarch

Comment 15 malsworn_deliquesces 2018-12-09 15:35:45 UTC
Created attachment 1512852 [details]
Log boot fedora 29 kernel 4.19.6

Sadly, I must bring some bad news. The 4.19.6 testing kernel I installed in F28 appeared to have fixed the issue for me, but I upgraded to F29 yesterday and now internet does not work anymore. It says the same thing again "cable not plugged in". Neither kernel 4.19.6.fc29 or 4.19.6.fc28 let me keep internet and sadly 4.18 was removed from the list since I only kept 5 kernels, so I cannot fix it by booting with that one.

I attached the boot log of F29 with this kernel in the hopes there is a way to patch this. If you need further details please let me know.

Comment 16 Steve 2018-12-10 06:29:27 UTC
(In reply to d.vicesss from comment #15)
...

Thanks for your update.

...
> ... 4.18 was removed from the list since I only kept 5
> kernels, so I cannot fix it by booting with that one.
...

To stop "dnf" from removing kernels, make this change to /etc/dnf/dnf.conf until this bug is resolved:

#installonly_limit=3
installonly_limit=0 # stops "dnf" from removing kernels

See "man dnf.conf" for details.

To find the F29 kernels in all repos:

# dnf repoquery kernel

To reinstall the F29-release kernel:

# dnf install kernel-0:4.18.16-300.fc29.x86_64

To manually remove a specific kernel:

# dnf list kernel # get a list of kernels
# dnf remove kernel\*-4.19.2-301.fc29 # copy&paste the version from the list

Comment 17 Steve 2018-12-10 06:51:18 UTC
(In reply to Steve from comment #16)
...
> To find the F29 kernels in all repos:
> 
> # dnf repoquery kernel

Correction. Kernel 4.19.7 is available in updates-testing:

# dnf -q repoquery kernel --enablerepo=updates-testing
kernel-0:4.18.16-300.fc29.x86_64
kernel-0:4.19.6-300.fc29.x86_64
kernel-0:4.19.7-300.fc29.x86_64

Please test 4.19.7.

Comment 18 Steve 2018-12-10 08:38:58 UTC
(In reply to d.vicesss from comment #15)
...
> 4.18 was removed from the list since I only kept 5
> kernels, so I cannot fix it by booting with that one.

Here are two variants of a possible workaround to get networking started:

Marc Dionne (Bug 1650984, Comment 3):

Once the machine is booted, the network can be enabled by loading realtek and reloading the r8169 module:

$ modprobe realtek
$ rmmod r8169
$ modprobe r8169

Michael Wiktowy (Bug 1650984, Comment 6):

As a more permanent temporary work-around:
'echo realtek > /etc/modules-load.d/realtek.conf'
causes systemd to load the module on boot.

Comment 19 malsworn_deliquesces 2018-12-10 12:41:35 UTC
The modprobe, rmmod and modprobe workaround did the trick, but it persists between reboots so I sadly cannot test 4.19.7 for you. I enabled internet with the workaround in order to download kernel 4.19.7. When I first rebooted into 4.19.6 internet was already working, so of course it was also working when I booted into 4.19.7 afterwards.

If there is a way for me to test 4.19.7 please let me know. I am not very good at this technology stuff.

I would like to thank all of you for your time and patience to get this working again for me and others.

Comment 20 Steve 2018-12-10 14:21:06 UTC
(In reply to d.vicesss from comment #19)
...
> If there is a way for me to test 4.19.7 please let me know. I am not very
> good at this technology stuff.

It sounds like 4.19.7 is working as expected. As a further test, try power cycling the computer (use "Power Off" from the Gnome menu -- don't suspend).

If you have a router or switch, power cycle it too.

I would suggest using the system the way you normally would and reporting back if networking fails to start again.

> I would like to thank all of you for your time and patience to get this
> working again for me and others.

Thanks for testing and reporting.

BTW, the Gnome Desktop comes with a graphical app for managing software (called "Software"), but it isn't very convenient for kernel testing:

https://wiki.gnome.org/Apps/Software

Comment 21 Justin M. Forbes 2019-01-29 16:13:31 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 29 kernel bugs.

Fedora 29 has now been rebased to 4.20.5-200.fc29.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.

Comment 22 Justin M. Forbes 2019-02-21 21:07:00 UTC
*********** MASS BUG UPDATE **************
This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 3 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.


Note You need to log in before you can comment on or make changes to this bug.