Bug 1178720 - nfs fails to start on boot
Summary: nfs fails to start on boot
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: nfs-utils
Version: 21
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Steve Dickson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1194000 (view as bug list)
Depends On:
Blocks: 1284738
TreeView+ depends on / blocked
 
Reported: 2015-01-05 11:50 UTC by Brian J. Murrell
Modified: 2019-09-12 08:09 UTC (History)
9 users (show)

Fixed In Version: nfs-utils-1.3.1-6.4.fc21
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1284738 (view as bug list)
Environment:
Last Closed: 2015-07-10 19:13:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Brian J. Murrell 2015-01-05 11:50:01 UTC
Description of problem:
On every boot nfs-server fails to start.

Version-Release number of selected component (if applicable):
nfs-utils-1.3.1-2.0.fc21.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Enable nfs-server
2. Reboot

Actual results:
No NFS server is started and clients complain

Expected results:
NFS server should be started and clients should be happy

Additional info:

After boot it is dead:
# systemctl status nfs-server
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled)
   Active: inactive (dead)

Yet starts with no problem:
# systemctl start nfs-server
# systemctl status nfs-server
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled)
   Active: active (exited) since Mon 2015-01-05 06:47:11 EST; 1s ago
  Process: 4451 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)
  Process: 4449 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
 Main PID: 4451 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/nfs-server.service

From journalctl:

Jan 05 06:29:34 pc.interlinx.bc.ca systemd[1]: Cannot add dependency job for unit nfs.target, ignoring: Unit nfs.target failed to load: No such file or directory.

Not sure what else to include.  Happy to gather further info upon request.

Comment 1 Steve Dickson 2015-01-24 13:57:14 UTC
This should be fixed in the latest release nfs-utils-1.3.1-5.0.fc21

Comment 2 Brian J. Murrell 2015-03-23 15:51:54 UTC
Doesn't seem to be:

# systemctl status nfs
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled)
   Active: inactive (dead)
# rpm -q nfs-utils
nfs-utils-1.3.1-6.1.fc21.x86_64
# systemctl start nfs
# systemctl status nfs
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled)
   Active: active (exited) since Mon 2015-03-23 11:51:13 EDT; 4s ago
  Process: 5556 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)
  Process: 5554 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
 Main PID: 5556 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/nfs-server.service

Any further information I can provide?

Comment 3 Rubén 2015-04-04 23:21:47 UTC
I've got the same problem. nfs-utils-1.3.1-6.2.fc21 and not solved yet. Besides, when I try to force its enabling (although nfs.service is already enabled) I get the same message:

$ sudo systemctl enable nfs.service
Failed to execute operation: No such file or directory

Comment 4 David Jones 2015-04-06 14:51:50 UTC
I'm experiencing the same thing on RHEL 7, with nfs-utils 1.3.0-0.8 

I'm guessing that it's related to the nfs.service symlink to nfs-server,service. Systemd doesn't always work as you'd expect with symlinks. You need to run "systemctl enable nfs-server.service". 

Unfortunately, this is not well documented, either by RH or the Fedora project. 

It was working before on our RHEL server, but it broke after updating from 1.3.0.

nfs.target is probably using a symlink, and systemd doesn't like it.

Comment 5 David Jones 2015-04-06 17:24:52 UTC
See #1209213 for a fix.

Comment 6 David Jones 2015-04-06 17:25:41 UTC
Sorry, here's the link: https://bugzilla.redhat.com/show_bug.cgi?id=1209213.

Comment 7 Gavin Burris 2015-05-21 13:59:32 UTC
Just bitten by this bug at 8AM this morning.  It is present in RHEL 7 with: nfs-utils-1.3.0-0.8.el7.x86_64

Comment 8 Gavin Burris 2015-05-21 14:04:37 UTC
This took down our production cluster.  How has the bug gone four months and not been squashed!?

Comment 9 David Jones 2015-05-21 17:02:24 UTC
I don't think there is a fix for it. Once it happens, it has to be fixed manually.

Comment 10 Gavin Burris 2015-05-21 17:41:29 UTC
There is no manually; This is systems programming.  The "manual" piece should go into the pre or post script of the RPM.

Comment 11 Gavin Burris 2015-05-22 13:48:58 UTC
Is anyone there?  This is a LANDMINE for the next person that does a RHEL 7.1 maintenance window.  This needs to be fixed ASAP.  Cheers.

Comment 12 Gavin Burris 2015-05-22 13:53:56 UTC
Can the PRE script of the RPM be updated to correct the old link on update?

ALSO Take note that this recent change breaks configuration management, too, because the nfsd service name has changed, from nfs to nfs-server.  Can the service name be aliased in systemd?

Comment 13 David Jones 2015-05-22 15:07:04 UTC
The service name didn't actually change. nfs.service is just a link to nfs-server.service. It never should have been there, because linked unit files don't work properly with systemd and it's confusing. Someone who didn't really understand systemd probably thought it would be convenient to have. See bug 1192501 that I submitted for this. 

The NFS dependency tree has been a bit of a mess ever since Fedora migrated to systemd. It's always been pretty confusing exactly what services you need to enable, particularly if you're using secure NFS. I think that may finally be resolved though. 

Note that this issue is actually tracked by bug 1203765 for RHEL 7, which also provides a KB reference.

Comment 14 Gavin Burris 2015-05-22 18:24:30 UTC
Hrm.  "You are not authorized to access bug #1203765"  Looks like this is internal-only.  I found the knowledgebase article, but the question still stands...

Why isn't there a POST script that checks for this and corrects?  This took down our research cluster.  It'a a landmine, and it is lying in wait in the current repos.

Cheers.

Comment 15 Brian J. Murrell 2015-06-06 22:15:50 UTC
This bug is still present in F21 fully updated as of today:

# systemctl status nfs-server
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled)
   Active: inactive (dead)

(In reply to David Jones from comment #13)
> The service name didn't actually change. nfs.service is just a link to
> nfs-server.service.

But notice that I, as the reporter on this ticket have never used the "nfs" symlink but have always used the nfs-server unit file, so the nfs symlink is a red herring to this ticket.

So now that this is confirmed as still present in F21, can we have it looked at again, please?

Comment 16 Rubén 2015-06-07 22:59:52 UTC
Still present in Fedora 22.

Comment 17 Steve Dickson 2015-06-08 14:16:32 UTC
(In reply to Rubén Lledó from comment #16)
> Still present in Fedora 22.

Is is there any type of logging in /var/log/messages 
as to why systemd is not starting the service?

Comment 18 Brian J. Murrell 2015-06-08 14:30:57 UTC
@Steve: I don't have anything in my /var/log/messages about it.  :-(

Comment 19 Rubén 2015-06-12 16:48:40 UTC
(In reply to Steve Dickson from comment #17)
> (In reply to Rubén Lledó from comment #16)
> > Still present in Fedora 22.
> 
> Is is there any type of logging in /var/log/messages 
> as to why systemd is not starting the service?

This is all I could find (output of grep -wi nfs /var/log/messages):

Jun 12 17:50:10 localhost systemd: Cannot add dependency job for unit nfs.target, ignoring: Unit nfs.target failed to load: No such file or directory.
Jun 12 17:50:20 localhost systemd: Starting Preprocess NFS configuration...
Jun 12 17:50:20 localhost systemd: Started Preprocess NFS configuration.
Jun 12 17:50:20 localhost audit: <audit-1130> pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=nfs-config comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jun 12 17:50:20 localhost kernel: audit: type=1130 audit(1434124220.672:77): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=nfs-config comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jun 12 17:51:41 localhost systemd: Started RPC security service for NFS client and server.
Jun 12 17:51:41 localhost systemd: Started RPC security service for NFS server.
Jun 12 17:51:41 localhost systemd: Reached target NFS client services.
Jun 12 17:51:41 localhost systemd: Starting NFS client services.
Jun 12 17:52:05 localhost systemd: Starting Notify NFS peers of a restart...
Jun 12 17:52:05 localhost systemd: Started Notify NFS peers of a restart.

Comment 20 Steve Dickson 2015-06-26 13:43:44 UTC
I'm thinking this is the fix for this problem

commit b98f2af15fea1d14a4b6ab1ab2dcdfec8db5dd1b
Author: Steve Dickson <steved>
Date:   Thu Jun 25 11:09:38 2015 -0400

    nfs-server: Use rpcbind.service instead of rpbind.target
    
    To trigger the systemd socket activation support
    in rpcbind, nfs-service needs to Requires/After
    rpcbind.service instead of rpbind.target
    
    Signed-off-by: Steve Dickson <steved>

Comment 21 Fedora Update System 2015-06-26 15:17:04 UTC
nfs-utils-1.3.1-6.4.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/nfs-utils-1.3.1-6.4.fc21

Comment 22 Fedora Update System 2015-06-27 12:43:08 UTC
Package nfs-utils-1.3.1-6.4.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing nfs-utils-1.3.1-6.4.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-10853/nfs-utils-1.3.1-6.4.fc21
then log in and leave karma (feedback).

Comment 23 Rubén 2015-07-01 09:55:11 UTC
(In reply to Fedora Update System from comment #22)
> Package nfs-utils-1.3.1-6.4.fc21:
> * should fix your issue,
> * was pushed to the Fedora 21 testing repository,
> * should be available at your local mirror within two days.
> Update it with:
> # su -c 'yum update --enablerepo=updates-testing nfs-utils-1.3.1-6.4.fc21'
> as soon as you are able to.
> Please go to the following url:
> https://admin.fedoraproject.org/updates/FEDORA-2015-10853/nfs-utils-1.3.1-6.
> 4.fc21
> then log in and leave karma (feedback).

I don't understand. If I want the bug fixed, should I downgrade nfs-utils-1.3.2-8.fc22 to nfs-utils-1.3.1-6.4.fc21? Why aren't fixes applied to all versions after the buggy one? I don't like to have a package unsynchronized with the whole system.

Comment 24 Steve Dickson 2015-07-01 14:04:49 UTC
(In reply to Rubén Lledó from comment #23)
> (In reply to Fedora Update System from comment #22)
> > Package nfs-utils-1.3.1-6.4.fc21:
> > * should fix your issue,
> > * was pushed to the Fedora 21 testing repository,
> > * should be available at your local mirror within two days.
> > Update it with:
> > # su -c 'yum update --enablerepo=updates-testing nfs-utils-1.3.1-6.4.fc21'
> > as soon as you are able to.
> > Please go to the following url:
> > https://admin.fedoraproject.org/updates/FEDORA-2015-10853/nfs-utils-1.3.1-6.
> > 4.fc21
> > then log in and leave karma (feedback).
> 
> I don't understand. If I want the bug fixed, should I downgrade
> nfs-utils-1.3.2-8.fc22 to nfs-utils-1.3.1-6.4.fc21? Why aren't fixes applied
> to all versions after the buggy one? I don't like to have a package
> unsynchronized with the whole system.

No... upgrade to nfs-utils-1.3.2-9.fc22 

See https://bugzilla.redhat.com/show_bug.cgi?id=1233005 that
is the f22 bz for this problem.

Comment 25 Steve Dickson 2015-07-01 19:40:16 UTC
*** Bug 1194000 has been marked as a duplicate of this bug. ***

Comment 26 Fedora Update System 2015-07-10 19:13:23 UTC
nfs-utils-1.3.1-6.4.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 27 karlp 2015-08-17 12:16:25 UTC
For me, on fc21, with nfs-utils up to date, NFS still doesn't start automatically.  I still have the the broken symlink, as described here: https://bugzilla.redhat.com/show_bug.cgi?id=1209213 and mentioned above.


nfs-utils version:
Version     : 1.3.1
Release     : 6.4.fc21

Comment 28 David Jones 2015-08-17 16:18:55 UTC
I don't think the update fixes the problem. It only prevents it. If you've already applied the bad update, You'll probably have to fix it manually, using the method I described.

I actually experienced this issue on RHEL 7, not Fedora. I installed a couple systems using the initial install disk. They worked great at first, but the first big update broke all kinds of things on both of them, and I'm probably just going to do a reinstall using the latest release.

From what I've heard, the same thing happened with the initial release of RHEL 6.

I guess the moral is, don't ever use the first release of anything.


Note You need to log in before you can comment on or make changes to this bug.