Bug 1175005
| Summary: | Cannot mount NFS v3 partition until after reboot | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Andrew Beekhof <abeekhof> |
| Component: | nfs-utils | Assignee: | Steve Dickson <steved> |
| Status: | CLOSED WONTFIX | QA Contact: | Filesystem QE <fs-qe> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.1 | CC: | bb, chris, email.ahmedkamal, fmhstar, lersek, mathieu.peltier, me, xzhou, yoyang |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-12-15 07:32:25 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Andrew Beekhof
2014-12-16 23:22:06 UTC
(In reply to Andrew Beekhof from comment #0) > Additional info: > > [root@rhos5-swift1 ~]# systemctl status rpc-statd.service > rpc-statd.service - NFS status monitor for NFSv2/3 locking. > Loaded: loaded (/usr/lib/systemd/system/rpc-statd.service; static) > Active: failed (Result: exit-code) since Wed 2014-12-17 15:14:24 AEDT; > 3min 27s ago > Process: 25551 ExecStart=/usr/sbin/rpc.statd --no-notify $STATDARG > (code=exited, status=1/FAILURE) > > Dec 17 15:14:24 rhos5-swift1 systemd[1]: Starting NFS status monitor for > NFSv2/3 locking.... > Dec 17 15:14:24 rhos5-swift1 rpc.statd[25552]: Version 1.3.0 starting > Dec 17 15:14:24 rhos5-swift1 rpc.statd[25552]: Flags: TI-RPC > Dec 17 15:14:24 rhos5-swift1 rpc.statd[25552]: Initializing NSM state > Dec 17 15:14:24 rhos5-swift1 systemd[1]: rpc-statd.service: control process > exited, code=exited status=1 > Dec 17 15:14:24 rhos5-swift1 systemd[1]: Failed to start NFS status monitor > for NFSv2/3 locking.. > Dec 17 15:14:24 rhos5-swift1 systemd[1]: Unit rpc-statd.service entered > failed state. Was there any more details in /var/log/messages as to why rpc.statd did not startup up after the install? nada I checked again and there was a little more. Hope it helps: Jan 10 07:31:40 localhost systemd: Starting NFS status monitor for NFSv2/3 locking.... Jan 10 07:31:40 localhost rpc.statd[24956]: Version 1.3.0 starting Jan 10 07:31:40 localhost rpc.statd[24956]: Flags: TI-RPC Jan 10 07:31:40 localhost rpc.statd[24956]: failed to create RPC listeners, exiting Jan 10 07:31:40 localhost systemd: rpc-statd.service: control process exited, code=exited status=1 Jan 10 07:31:40 localhost systemd: Failed to start NFS status monitor for NFSv2/3 locking.. Jan 10 07:31:40 localhost systemd: Unit rpc-statd.service entered failed state. Jan 10 07:31:40 localhost rpc.statd[24958]: Version 1.3.0 starting Jan 10 07:31:40 localhost rpc.statd[24958]: Flags: TI-RPC Jan 10 07:31:40 localhost rpc.statd[24958]: failed to create RPC listeners, exiting (In reply to Andrew Beekhof from comment #4) > I checked again and there was a little more. Hope it helps: > > Jan 10 07:31:40 localhost systemd: Starting NFS status monitor for NFSv2/3 > locking.... > Jan 10 07:31:40 localhost rpc.statd[24956]: Version 1.3.0 starting > Jan 10 07:31:40 localhost rpc.statd[24956]: Flags: TI-RPC > Jan 10 07:31:40 localhost rpc.statd[24956]: failed to create RPC listeners, > exiting > Jan 10 07:31:40 localhost systemd: rpc-statd.service: control process > exited, code=exited status=1 > Jan 10 07:31:40 localhost systemd: Failed to start NFS status monitor for > NFSv2/3 locking.. > Jan 10 07:31:40 localhost systemd: Unit rpc-statd.service entered failed > state. > Jan 10 07:31:40 localhost rpc.statd[24958]: Version 1.3.0 starting > Jan 10 07:31:40 localhost rpc.statd[24958]: Flags: TI-RPC > Jan 10 07:31:40 localhost rpc.statd[24958]: failed to create RPC listeners, > exiting This means statd could not create a UDP or TCP socket or it could not register with rpcbind. Looking at rpc.statd systemd service file following After= exists After=network.target nss-lookup.target rpcbind.target I wonder if this After= should have rpcbind.service instead After=network.target nss-lookup.target rpcbind.service Doesn't seem to help.
I experimented by starting things by hand:
[root@rhos6-cinder1 ~]# systemctl start rpcbind.service
[root@rhos6-cinder1 ~]# systemctl status rpcbind.service
rpcbind.service - RPC bind service
Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; static)
Active: active (running) since Tue 2015-01-13 00:25:13 AEDT; 23s ago
Process: 2673 ExecStart=/sbin/rpcbind -w ${RPCBIND_ARGS} (code=exited, status=0/SUCCESS)
Main PID: 2674 (rpcbind)
CGroup: /system.slice/rpcbind.service
└─2674 /sbin/rpcbind -w
Jan 13 00:25:13 rhos6-cinder1.vmnet.lab.bos.redhat.com systemd[1]: Started RPC bind service.
Jan 13 00:25:13 rhos6-cinder1.vmnet.lab.bos.redhat.com rpcbind[2674]: Cannot open '/var/lib/rpcbind/rpcbind.xdr' file for reading, errno 2 (No such file or directory)
Jan 13 00:25:13 rhos6-cinder1.vmnet.lab.bos.redhat.com rpcbind[2674]: Cannot open '/var/lib/rpcbind/portmap.xdr' file for reading, errno 2 (No such file or directory)
this looks suspicious, yet still allows rpc-statd to start
[root@rhos6-cinder1 ~]# systemctl start rpc-statd.service
[root@rhos6-cinder1 ~]# systemctl status rpc-statd.service
rpc-statd.service - NFS status monitor for NFSv2/3 locking.
Loaded: loaded (/usr/lib/systemd/system/rpc-statd.service; static)
Active: active (running) since Tue 2015-01-13 00:25:26 AEDT; 26s ago
Process: 2677 ExecStart=/usr/sbin/rpc.statd --no-notify $STATDARGS (code=exited, status=0/SUCCESS)
Main PID: 2678 (rpc.statd)
CGroup: /system.slice/rpc-statd.service
└─2678 /usr/sbin/rpc.statd --no-notify
Jan 13 00:25:26 rhos6-cinder1.vmnet.lab.bos.redhat.com rpc.statd[2678]: Version 1.3.0 starting
Jan 13 00:25:26 rhos6-cinder1.vmnet.lab.bos.redhat.com rpc.statd[2678]: Flags: TI-RPC
Jan 13 00:25:26 rhos6-cinder1.vmnet.lab.bos.redhat.com systemd[1]: Started NFS status monitor for NFSv2/3 locking..
and now magically it works (or fails but only due to a config error on my part):
2015-01-13 00:30:40.294 1861 ERROR cinder.volume.drivers.remotefs [-] Exception during mounting NFS mount failed for share 192.168.124.1:/srv/rhos-6.0/cinder. Error - {'nfs': u"Unexpected error while running command.\nCommand: sudo cinder-rootwrap /etc/cinder/rootwrap.conf mount -t nfs -o v3 192.168.124.1:/srv/rhos-6.0/cinder /var/lib/cinder/mnt/61f52c57a53dbf069df27650ab88800c\nExit code: 32\nStdout: u''\nStderr: u'mount.nfs: mounting 192.168.124.1:/srv/rhos-6.0/cinder failed, reason given by server: No such file or directory\\n'"}
I wonder if `systemctl daemon-reload` wasn't enough for the new option to take effect
Could you please retest with the latest nfs-utils bits? The systemd scripts have been reworked so I'm thinking this is not longer a problem. Isn't that what reproducers are for? Faced with the same problem. But the work, if after # systemctl start rpcbind.servise do # systemctl restart rpcbind.servise Then mounting takes place without problems. Forgot indicate that faced with this problem on Fedora 22 x86_64 nfs-utils-1.3.2-9.fc22.x86_64 I'm seeing this is still a problem on CentOS 7.4 as of today. I've spun up a fresh VM and the problem is consistent. It only goes away after reboot. Manually starting the services in order, RPC-Statd always fails even with RPCBIND started, and after restarting it as well. For some reason only a full system reboot fixes the issue. After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened. |