Bug 1419347 - lost ALL WEB connectivity for quite a while
Summary: lost ALL WEB connectivity for quite a while
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: NetworkManager
Version: 24
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Lubomir Rintel
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-05 17:07 UTC by Ray Holme
Modified: 2017-02-17 08:16 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-02-17 08:16:00 UTC
Type: Bug


Attachments (Terms of Use)
Before and after VPNC brought up (3.32 KB, text/plain)
2017-02-08 17:35 UTC, Ray Holme
no flags Details
this is after vpnc is invoked and all works fine (4.20 KB, text/plain)
2017-02-08 17:37 UTC, Ray Holme
no flags Details

Description Ray Holme 2017-02-05 17:07:01 UTC
Description of problem: After upgrading to new release, could do NOTHING with web and Firefox - all name searches fail.


Version-Release number of selected component (if applicable): I upgrade weekly and just got 4.9.6 installed over 4.9.5. Many sub-components also upgraded (around 70 updates).


How reproducible: 100% for about 30 minutes, not sure as not thrilled with booting again if the problem went away permanently or will come back after booting.


Steps to Reproduce:
1. reboot
2. login
3. bring up firefox and try to connect to ANY known (saved) address

Actual results: address unreachable - .... name not found


Expected results: normally this works fine


Additional info:
 1) upgraded two systems today
     my wife's 2-cpu has NO wifi ability and worked fine
     my        4-cpu has wireless AND hard wire connect ability
      IT FAILED BADLY

  2) I went back to 4.9.5 and that did not help

  3) so back to 4.9.6
     a) turned off wifi (should have been off by default)
         that did not help
     b) turn wired connection off and on a couple times
         that did not help - but showed a new connection as expected
     c) able to ssh to one site NO problem
        able to connect to my local router using Firefox no problem
     d) brought up VPN and connected to another site using ssh

    shortly after all of the above, firefox started working right

I will have to reboot in the next day or so (another reported bug causes me to NOT return from sleep mode 50% of the time), so I will note if the fix just noted is permanent or the problem recurrs when I reboot. I will note this ASAP in this bug report.

Apologies if I chose the wrong component. I have no idea how to determine which component is the real problem. But it sure has to do with the Network and Managing it.

Comment 1 Ray Holme 2017-02-05 23:47:03 UTC
I came back from sleep mode and again Firefox is dead in water.
SSH worked fine. Turning off and on the wired connection did nothing.

STARTING VPN KICKS SOMETHING.

As soon as I started VPN, Firefox started working.

This is nasty and not a good fix as many will not have VPN and I would never have guessed that but it was the last thing I tried before.

Comment 2 Ray Holme 2017-02-06 00:02:31 UTC
Yes I rebooted and the same thing happened.

ssh works fine, but firefox cannot find anyone until I start VPN

in short the following command fixes Firefox finding external sites (go figure)
   sudo vpnc

I have been upgrading every week for years and using vpn for many months.
Glad that it fixed, I would never have guessed.

Comment 3 Ray Holme 2017-02-06 00:10:18 UTC
I should have mentioned that BOTH Firefox and Evolution are down until VPN starts, then both act fine. I can connect to any other device in my private network (another computer or the router in Firefox, but NOT externally); and I can ssh to an external box not in a VPN set. Starting VPN makes the computer act normal.

My other computer (wife's) has NO such problem and was updated this morning too.
It does NOT have WIFI hardware (mine is deselected but shows up highlighted in blue on left in "Network Settings", on right says WIFI unavailable). Perhaps this last means nothing, as clicking on wired - makes it look right).

Comment 4 Ray Holme 2017-02-06 15:00:39 UTC
A member of the fedora forum suggested I run ipconfig and route to see what is happening.

I don't see what is wrong but maybe someone in red hat will:

Before I run "vpnc" - we have (ifconfig; rout) > /tmp/somefile (somefile below)

enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.101.101  netmask 255.255.255.0  broadcast 192.168.101.255
        inet6 fe80::d37:ca3f:2ee7:98c5  prefixlen 64  scopeid 0x20<link>
        ether 78:45:c4:30:16:3a  txqueuelen 1000  (Ethernet)
        RX packets 48584  bytes 36756153 (35.0 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 48884  bytes 27510237 (26.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 27126  bytes 11206471 (10.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 27126  bytes 11206471 (10.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 00:00:00:00:00:00  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         barricade-route 0.0.0.0         UG    100    0        0 enp4s0
192.168.101.0   0.0.0.0         255.255.255.0   U     100    0        0 enp4s0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

---- after running vpnc and all working fine - we get

enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.101.101  netmask 255.255.255.0  broadcast 192.168.101.255
        inet6 fe80::d37:ca3f:2ee7:98c5  prefixlen 64  scopeid 0x20<link>
        ether 78:45:c4:30:16:3a  txqueuelen 1000  (Ethernet)
        RX packets 46313  bytes 36194013 (34.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 45758  bytes 26981741 (25.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 22771  bytes 10846215 (10.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 22771  bytes 10846215 (10.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1412
        inet 10.4.19.173  netmask 255.255.255.255  destination 10.4.19.173
        inet6 fe80::62ad:ab3a:73c1:e6f7  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 1532  bytes 294164 (287.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1554  bytes 100496 (98.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 00:00:00:00:00:00  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         barricade-route 0.0.0.0         UG    100    0        0 enp4s0
10.4.19.0       0.0.0.0         255.255.255.0   U     0      0        0 tun0
10.4.19.0       0.0.0.0         255.255.255.0   U     1      0        0 tun0
10.10.10.0      0.0.0.0         255.255.255.0   U     0      0        0 tun0
10.10.10.0      0.0.0.0         255.255.255.0   U     1      0        0 tun0
10.10.11.0      0.0.0.0         255.255.255.0   U     0      0        0 tun0
10.10.11.0      0.0.0.0         255.255.255.0   U     1      0        0 tun0
10.10.12.0      0.0.0.0         255.255.255.0   U     0      0        0 tun0
10.10.12.0      0.0.0.0         255.255.255.0   U     1      0        0 tun0
10.110.0.0      0.0.0.0         255.255.0.0     U     0      0        0 tun0
10.110.0.0      0.0.0.0         255.255.0.0     U     1      0        0 tun0
iccbad1.iccb.or 0.0.0.0         255.255.255.255 UH    0      0        0 tun0
iccbad2.iccb.or 0.0.0.0         255.255.255.255 UH    0      0        0 tun0
gateiccb        barricade-route 255.255.255.255 UGH   0      0        0 enp4s0
192.168.101.0   0.0.0.0         255.255.255.0   U     100    0        0 enp4s0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

Needless to say all the 10. adrreses and gateiccb were added by vpnc.

Good luck and ouch - this is a real PITA.
If I did not have a use for VPNC (and lucky I needed it yesterday), I would be dead in the water.
Please remember that this machine has WIFI ability which is turned OFF as I use a direct wire. My other machine (wife's) does not and is just fine using the wire for internet.

Comment 5 Ray Holme 2017-02-08 13:28:28 UTC
I do not know if this is related, but I never got this error before using vpnc.

Bringing VPNC up now is the ONLY way I can talk to the outside network.
I can talk to the router but not through it till vpnc comes up.

I get this error (never before) when bring vpnc up

/etc/vpnc/vpnc-script: line 376: /var/run/vpnc/resolv.conf-backup: No such file or directory

Comment 6 Thomas Haller 2017-02-08 13:36:40 UTC
This looks like a configuration error, but the configuration is not provided, so who knows.

Anyway, NetworkManager doesn't seem involved. Reassigning to vpnc.

Comment 7 Thomas Haller 2017-02-08 13:48:50 UTC
Thinking about it again. This bug hasn't enough information. It is unclear what is happening.

The first comment doesn't talk about vpnc. If you have a problem without VPN (comment 0), that should be debugged without adding vpnc into the game.

The bug report should better describe the current state.
It should describe precisely what you did before.

E.g. one can only assume you are using NetworkManager to configure your networking. But no configuration of NetworkManager is shown. Using NetworkManager together with external vpnc is of course possible, but you have to configure your network properly (what that means depends on your environment).


If you are in a state where it doesn't work, look at the output of

  ip route
  ip addr
  ip link
  cat /etc/resolv.conf
  ping -c 3 8.8.8.8
  nmcli device
  nmcli connection

Comment 8 Ray Holme 2017-02-08 17:35:19 UTC
Created attachment 1248649 [details]
Before and after VPNC brought up

This one is the dead version (returned from sleep before bringing up vpnc).

Comment 9 Ray Holme 2017-02-08 17:37:35 UTC
Created attachment 1248651 [details]
this is after vpnc is invoked and all works fine

Noted that there is NO resolve.conf and the unchanged vpnc script complains about this. upon startup - have no idea if this is related.

/etc/vpnc/vpnc-script: line 376: /var/run/vpnc/resolv.conf-backup: No such file or directory
VPNC started in background (pid: 10890)...

Comment 10 Thomas Haller 2017-02-08 17:42:08 UTC
you see that you can ping the internet, but name-resolution is not working because /etc/resolv.conf is in an invalid state (it was apparently written by vpnc, and left in an invalid state).

Fix your resolv.conf file, e.g. by putting there googles name-server ("nameserver 8.8.8.8").

If you use vpnc, configure it in a way to handle your resolv.conf correctly. Consult the documentation for how to do that.

Alternatively, you may consider using the NetworkManager-vpnc-gnome plugin.

Comment 11 Ray Holme 2017-02-08 18:14:34 UTC
OK, I did not know what to attach. I have been using the same scripts (standard) and vpnc for over 6 months with no errors. I update the OS and other software weekly and have survived all prior updates without an error.

This problem occurred last week-end immediately after a
   dnf update
   reboot

I only luckily stumbled upon VPNC coming up FIXED the problem. I have no idea what it kicks, but obviously something happens.

I did notice that the resolv.conf has some VPNC stuff at the beginning which I have no knowledge of. The last two lines were all I needed before (nameserver lines). I also note that this file is the same in both attached snapshots.

Perhaps the warning when I bring up vpnc (not prior to last update) is related to the problem and that resolve.conf might also be related.

Sorry but I have no idea what is going on. ALL WAS WORKING FULLY BEFORE 2/4/2017 OS UPDATE.

Here is the script I ran - as requested:
(echo route; ip route; echo addr; ip addr; echo link; ip link; \
  echo resolv.conf; cat /etc/resolv.conf; echo pinging 8s; ping -c 3 8.8.8.8; \
  echo device; nmcli device; echo connection; nmcli connection) > /tmp/bug_info

Comment 12 Ray Holme 2017-02-08 18:19:14 UTC
OK, problem resolved. Feel free to close.

Evidently the last upgrade I did changed something about how resolv.conf was being fixed. I put a script change into my script in (kills ssh/sftp sessions) 
 /usr/lib/systemd/system-sleep

I now restore my original resolv.conf upon termination.
   VPNC had pillaged this with it's own!
I guess the latest release got too agressive with resolv.conf
 or the prior release was putting things back somehow.


THIS HAS WORKED FOR A LONG TIME.

Thanks.

Over and out.


Note You need to log in before you can comment on or make changes to this bug.