Bug 1275645

Summary: NetworkManager(-dispatcher) does not automount cifs share after boot/login
Product: [Fedora] Fedora Reporter: grobi <grobi>
Component: NetworkManagerAssignee: Lubomir Rintel <lkundrak>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 22CC: dcbw, grobi, lkundrak, psimerda
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-19 18:21:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
last boot log
none
bootlog with NM related messages only none

Description grobi 2015-10-27 12:19:46 UTC
Created attachment 1086836 [details]
last boot log

Hello,

Description of problem: 
Can't automount cifs share after NetworkManager becomes 'up'


Version-Release number of selected component (if applicable):
NetworkManager-1.0.6-6.fc22.x86_64
NetworkManager-1.0.6-7.fc22.x86_64 (updates-testing)

How reproducible:
Always


Steps to Reproduce:
MY_USER_ID=$UID
SERVER_IP=192.168.15.191
sudo bash -c "cat >> /etc/fstab <<EOF
//$SERVER_IP/operational /mnt/srv-files cifs credentials=/home/mgrobelin/.smb/srv-files.credentials.conf,iocharset=utf8,noauto,rw,nounix,sec=ntlmssp,uid=$MY_USER_ID,ip=$SERVER_IP 0 0
EOF"
 
# create credentials files (providing user, password & domain) as you do not want to save your password inside /etc/fstab
mkdir ~/.smb
cat >> ~/.smb/srv-files.credentials.conf <<EOF
username=mgrobelin
password=XXX
domain=mycomp
EOF
chmod 600 ~/.smb/srv-files.credentials.conf
 
# create mountpoint
sudo mkdir /mnt/srv-files
 

MY_INTERFACE="enp0s25"
sudo bash -c "cat >> /etc/NetworkManager/dispatcher.d/90-cifs-mounts <<EOF
#!/bin/bash
 
IF=\\\$1
STATUS=\\\$2
 
if [ \"\\\$IF\" == \"$MY_INTERFACE\" ]
then
    case \"\\\$2\" in
        up)
        logger \"NM Dispatcher Script up triggered: mounting\"
        mount /mnt/srv-files
        ;;
        pre-down)
        logger \"NM Dispatcher Script pre-down triggered: unmounting\"
        umount /mnt/srv-files
        ;;
        *)
	logger "NM Dispatcher Script $STATUS triggered"
        ;;
    esac
fi
EOF"
sudo chmod +x /etc/NetworkManager/dispatcher.d/90-cifs-mounts
sudo ln -s /etc/NetworkManager/dispatcher.d/90-cifs-mounts /etc/NetworkManager/dispatcher.d/pre-down.d/90-cifs-mounts
sudo systemctl enable NetworkManager-wait-online


Actual results:
- NetworkManager is connecting with success to LAN after login, but share is not mounted

- sudo mount /mnt/srv-files/ and sudo umount /mnt/srv-files/ works without problem => share get mounted/unmounted 

- I can execute NetworkManager dispatcher script without problem: sudo /etc/NetworkManager/dispatcher.d/90-cifs-mounts enp0s25 pre-down and sudo /etc/NetworkManager/dispatcher.d/90-cifs-mounts enp0s25 up mounts/unmounts share

- As soon as logged in and received a connection to LAN I can reconnect by clicking "Wired > Turn Off" and "Wired > My LAN" and mounting/unmounting behaves as expected


Expected results:
NetworkManager is connecting with success to LAN after login and share is mounted.


Additional info:
- I do not know how NetworkManager-wait-online influences this procedure. I do not know what that service does at all.

- Bootlog has some lines of interest:
Oct 27 12:40:35 myhost nm-dispatcher[1486]: Dispatching action 'pre-up' for enp0s25
Oct 27 12:40:35 myhost nm-dispatcher[1486]: Dispatching action 'up' for enp0s25
Oct 27 12:40:35 myhost root[1520]: NM Dispatcher Script up triggered: mounting cifs
Oct 27 12:40:40 myhost NetworkManager[1092]: <info>  startup complete
Oct 27 12:40:40 myhost systemd[1]: Started Network Manager Wait Online.
Oct 27 12:40:40 myhost systemd[1]: Reached target Network.
Oct 27 12:40:40 myhost systemd[1]: Starting Network.
[...]

Oct 27 12:40:41 myhost kernel: CIFS VFS: Error connecting to socket. Aborting operation.
Oct 27 12:40:41 myhost kernel: CIFS VFS: cifs_mount failed w/return code = -113
Oct 27 12:40:41 myhost nm-dispatcher[1486]: Unable to find suitable address.
Oct 27 12:40:41 myhost nm-dispatcher[1486]: Script '/etc/NetworkManager/dispatcher.d/90-cifs-mounts' exited with error status 32.
Oct 27 12:40:41 myhost NetworkManager[1092]: <warn>  (2) 90-cifs-mounts failed (failed): Script '/etc/NetworkManager/dispatcher.d/90-cifs-mounts' exited with error status 32.

- My interpretation is that NetworkManager fires 'up'-event before network is actually available. This is where i thought of NetworkManager-wait-online steps it. But it seems not to do the things i expect.

Comment 1 Dan Williams 2015-10-27 14:48:26 UTC
NM fires the "up" event when the first one of IPv4 or IPv6 is available, if they are both enabled.  And by default, they are.  So could you check with 'ip addr' to see what the IP configuration is, and whether enp0s25 has a valid IPv6 address (eg, one that doesn't start with FE80 which is link-local)?

Comment 2 grobi 2015-10-27 15:32:58 UTC
I currently have a loopback, a wired interface (enp0s25), a wifi interface (wlo1) and some VirtualBox interfaces configured.

Here is the output from having "ip addr" executed just before the mount within my dispatcher script:

Oct 27 16:14:35 myhost nm-dispatcher[1468]: Dispatching action 'pre-up' for enp0s25
Oct 27 16:14:35 myhost NetworkManager[1072]: <info>  (enp0s25): device state change: ip-check -> secondaries (reason 'none') [80 90 0]
Oct 27 16:14:35 myhost NetworkManager[1072]: <info>  (enp0s25): device state change: secondaries -> activated (reason 'none') [90 100 0]
Oct 27 16:14:35 myhost NetworkManager[1072]: <info>  NetworkManager state is now CONNECTED_LOCAL
Oct 27 16:14:35 myhost NetworkManager[1072]: <info>  NetworkManager state is now CONNECTED_SITE
Oct 27 16:14:35 myhost NetworkManager[1072]: <info>  Policy set 'mycomp LAN' (enp0s25) as default for IPv4 routing and DNS.
Oct 27 16:14:35 myhost NetworkManager[1072]: <info>  (enp0s25): Activation: successful, device activated.
Oct 27 16:14:35 myhost nm-dispatcher[1468]: Dispatching action 'up' for enp0s25
Oct 27 16:14:35 myhost root[1502]: NM Dispatcher Script up triggered: mounting cifs
Oct 27 16:14:35 myhost nm-dispatcher[1468]: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
Oct 27 16:14:35 myhost nm-dispatcher[1468]: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
Oct 27 16:14:35 myhost nm-dispatcher[1468]: inet 127.0.0.1/8 scope host lo
Oct 27 16:14:35 myhost nm-dispatcher[1468]: valid_lft forever preferred_lft forever
Oct 27 16:14:35 myhost nm-dispatcher[1468]: inet6 ::1/128 scope host
Oct 27 16:14:35 myhost nm-dispatcher[1468]: valid_lft forever preferred_lft forever
Oct 27 16:14:35 myhost nm-dispatcher[1468]: 2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
Oct 27 16:14:35 myhost nm-dispatcher[1468]: link/ether 58:20:b1:73:8f:12 brd ff:ff:ff:ff:ff:ff
Oct 27 16:14:35 myhost nm-dispatcher[1468]: inet 192.168.15.100/24 brd 192.168.15.255 scope global enp0s25
Oct 27 16:14:35 myhost nm-dispatcher[1468]: valid_lft forever preferred_lft forever
Oct 27 16:14:35 myhost nm-dispatcher[1468]: inet6 fe80::5a20:b1ff:fe73:8f12/64 scope link tentative
Oct 27 16:14:35 myhost nm-dispatcher[1468]: valid_lft forever preferred_lft forever
Oct 27 16:14:35 myhost nm-dispatcher[1468]: 3: wlo1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
Oct 27 16:14:35 myhost nm-dispatcher[1468]: link/ether 5c:e0:c5:54:70:54 brd ff:ff:ff:ff:ff:ff


[...]


Oct 27 16:14:41 myhost nm-dispatcher[1468]: Unable to find suitable address.
Oct 27 16:14:41 myhost nm-dispatcher[1468]: Script '/etc/NetworkManager/dispatcher.d/90-cifs-mounts' exited with error status 32.
Oct 27 16:14:41 myhost nm-dispatcher[1468]: Dispatching action 'pre-up' for virbr0

For better readability I have extracted the NetworkManager parts of my latest boot log with egrep '(nm-dispatcher|NetworkManager)' /tmp/bootlog >> /tmp/nm.log

Comment 3 grobi 2015-10-27 15:33:29 UTC
Created attachment 1086905 [details]
bootlog with NM related messages only

Comment 4 Dan Williams 2015-10-27 18:50:47 UTC
Interesting; so the script shows that we have an address.  Can you add 'ip route' to the script so we can see what routes there are?

Comment 5 grobi 2015-10-28 08:22:26 UTC
It prints this:

Oct 28 09:17:10 myhost nm-dispatcher[1467]: default via 192.168.15.254 dev enp0s25  proto static  metric 100
Oct 28 09:17:10 myhost nm-dispatcher[1467]: 192.168.15.0/24 dev enp0s25  proto kernel  scope link  src 192.168.15.100  metric 100


ip route when logged in results to this:

default via 192.168.15.254 dev enp0s25  proto static  metric 100 
192.168.15.0/24 dev enp0s25  proto kernel  scope link  src 192.168.15.100  metric 100 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1


So the route for virbr0 (VirtualBox bridged interface I guess) is not available to the dispatcher. May this be related to my issue?

Comment 6 Dan Williams 2015-10-28 18:31:43 UTC
(In reply to grobi from comment #5)
> It prints this:
> 
> Oct 28 09:17:10 myhost nm-dispatcher[1467]: default via 192.168.15.254 dev
> enp0s25  proto static  metric 100
> Oct 28 09:17:10 myhost nm-dispatcher[1467]: 192.168.15.0/24 dev enp0s25 
> proto kernel  scope link  src 192.168.15.100  metric 100
> 
> 
> ip route when logged in results to this:
> 
> default via 192.168.15.254 dev enp0s25  proto static  metric 100 
> 192.168.15.0/24 dev enp0s25  proto kernel  scope link  src 192.168.15.100 
> metric 100 
> 192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1
> 
> 
> So the route for virbr0 (VirtualBox bridged interface I guess) is not
> available to the dispatcher. May this be related to my issue?

No, its unrelated since the CIFS server is at 192.168.15.91 for which the route is already added by the time the server runs.

Can you add a "ping -D -c 10 -i 1 192.168.15.91" to the dispatcher script before the mount happens to see if we can actually ping the server or not?

Comment 7 grobi 2015-10-30 13:52:31 UTC
Oct 30 14:42:11 myhost nm-dispatcher[1481]: [1446212531.983726] From 192.168.15.100 icmp_seq=9 Destination Host Unreachable
Oct 30 14:42:11 myhost nm-dispatcher[1481]: [1446212531.984099] From 192.168.15.100 icmp_seq=10 Destination Host Unreachable
Oct 30 14:42:11 myhost nm-dispatcher[1481]: --- 192.168.15.91 ping statistics ---
Oct 30 14:42:11 myhost nm-dispatcher[1481]: 10 packets transmitted, 0 received, +10 errors, 100% packet loss, time 8998ms

Oh interesting ping doesn't work within dispatcher, but perfectly fine when running it from a shell after login.

Do you have an idea why it behaves like that?

Maybe this is caused by our corporate firewall? It requires some kind of windows domain authentication. That is why I used to create an "enterprise login" instead of regular user account, so machine connects with our Domain Controller on login.

Comment 8 grobi 2015-11-18 09:27:20 UTC
Problem disappeared somehow last week even with the mentioned configuration and previously installed version 1.0.6-7 of NetworkManager.

Meanwhile I received 1.0.6-8 and it behaves as desired, too.

Comment 9 Fedora End Of Life 2016-07-19 18:21:06 UTC
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.