Bug 1401561

I'm guessing this has something to do with this:
      https://bugzilla.redhat.com/show_bug.cgi?id=1402083

(In reply to Steve Dickson from comment #1)
> I'm guessing this has something to do with this:
>       https://bugzilla.redhat.com/show_bug.cgi?id=1402083

I have build selinux-policy-3.13.1-225.2.fc24 from https://kojipkgs.fedoraproject.org/packages/selinux-policy/3.13.1/225.2.fc25/src/selinux-policy-3.13.1-225.2.fc25.src.rpm and installed it on a Fedora 24 test system (selinux-policy, selinux-policy-targeted).

But this update does not solve the problem. rpcbind still fails with /var/run/rpcbind.lock: No such file or directory.

I have also tried booting with "semanage dontaudit off", but "grep rpcbind /var/log/audit/audit.log" still shows only sucsess lines.

I also have done a "restorecon -rv /usr /etc" after updates of selinux-policy-targeted

(I had to remove packages docker docker-selinux container-selinux because of dependency problems with selinux-policy-3.13.1-225.2.fc24, but that's another problem.)

Downgrading to rpcbind-0.2.3-10.rc1.fc24.x86_64 still solves the problem, even with selinux-policy-3.13.1-225.2.fc24 installed.

I also have tried rpcbind-0.2.3-13.rc2.fc24.x86_64 and rpcbind-0.2.4-0.fc24.x86_64, both build with mock from src rpm from koji (rpcbind-0.2.4-0.fc25.src.rpm rpcbind-0.2.3-13.rc2.fc25.src.rpm).

Both newer versions doesn't solve the problem, rpcbind still fails with /var/run/rpcbind.lock: No such file or directory .

This problem occurs because systemd can (depending on timing conditions at boot) start up rpcbind.service before starting up systemd-tmpfiles-setup.service. The latter is what creates the /run directory. So, when rpcbind starts, there might be no /run directory yet and hence it fails.

Brute force fix is to add

Wants=systemd-tmpfiles-setup.service
After=systemd-tmpfiles-setup.service

to /usr/lib/systemd/system/rpcbind.service

(A better fix would probably be something like "RequiresMountFor /run" but I don't think this works because there us no /run mount unit because it is not a regular mount, so the brute force fix is all I can think of at the moment).

Or, better yet, perhaps one can remove the DefaultDependencies=no  line ... that's what's preventing systemd from automatically adding a dependency that ensures basic system functionality (like the /run directory) is present.

However, the packagers probably had some reason for removing the default dependencies, even though its removal caused this bug.

This is the upstream fix

commit ee569be4d6189a68b38d2af162af00ff475b48e2
Author: Yann Leprince <yann.leprince>
Date:   Wed Dec 21 14:32:54 2016 -0500

    Fix boot dependency in systemd service file
    
    From: Yann Leprince <yann.leprince>
    
    This fix ensures that a separate /var partition will be mounted before
    rpcbind tries to write its status to /var/run.
    
    Acked-by: NeilBrown <neilb>
    Signed-off-by: Yann Leprince <yann.leprince>
    Signed-off-by: Steve Dickson <steved>

diff --git a/systemd/rpcbind.service.in b/systemd/rpcbind.service.in
index e7e86da..03a9e0b 100644
--- a/systemd/rpcbind.service.in
+++ b/systemd/rpcbind.service.in
@@ -2,6 +2,7 @@
 Description=RPC Bind
 Documentation=man:rpcbind(8)
 DefaultDependencies=no
+RequiresMountsFor=@statedir@
 
 # Make sure we use the IP addresses listed for
 # rpcbind.socket, no matter how this unit is started.

I'm not sure that upstream fix is sufficient (if @statedir@ expands to /var as the comment suggests).

It will ensure that /var is mounted before rpcbind starts, but it will not ensure that /run is mounted too -- both /var AND /run have to be mounted before rpcbind starts, since it complains about not being able to access /var/run/rpcbind.lock, and /var/run is a symlink to /run.

Even if you add RequiresMountsFor=@statedir@, it would still be possible for rpcbind to be started after /var was mounted but before the /run tmpfs was created, and then rpcbind would still fail.

If @statedir@ expands to /var/run, then perhaps the fix would work, but I'm skeptical because I'm not sure systemd actually has the notion of a /var/run (or even /run) mount point:

# systemctl status /var
    var.mount - /var
    ...
    Active: active (mounted)

BUT

# systemctl status /var/run
Unit var-run.mount could not be found.
# systemctl status /run
Unit run.mount could not be found.

rpcbind-0.2.4-1.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-26a132511e

rpcbind-0.2.3-14.rc2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-b1bd13d611

(In reply to Philip Spencer from comment #6)
> I'm not sure that upstream fix is sufficient (if @statedir@ expands to /var
> as the comment suggests).
> 
> It will ensure that /var is mounted before rpcbind starts, but it will not
> ensure that /run is mounted too -- both /var AND /run have to be mounted
> before rpcbind starts, since it complains about not being able to access
> /var/run/rpcbind.lock, and /var/run is a symlink to /run.
> 
> Even if you add RequiresMountsFor=@statedir@, it would still be possible for
> rpcbind to be started after /var was mounted but before the /run tmpfs was
> created, and then rpcbind would still fail.
> 
> If @statedir@ expands to /var/run, then perhaps the fix would work, but I'm
> skeptical because I'm not sure systemd actually has the notion of a /var/run
> (or even /run) mount point:
I set the statedir to be --with-statedir=/run/rpcbind in the spec file.

In my testing... the right thing was happen.

rpcbind-0.2.3-14.rc2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-b1bd13d611

rpcbind-0.2.4-1.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-26a132511e

I've just installed rpcbind-0.2.4-1.fc25.x86_64

and I'm still getting:

Jan 06 13:00:19 hopper.une.edu.au rpcbind[592]: rpcbind: /var/run/rpcbind.lock: No such file or directory

My /var filesystem in mounted via fstab

(In reply to Norman Gaywood from comment #12)
> I've just installed rpcbind-0.2.4-1.fc25.x86_64
> 
> and I'm still getting:
> 
> Jan 06 13:00:19 hopper.une.edu.au rpcbind[592]: rpcbind:
> /var/run/rpcbind.lock: No such file or directory
> 
> My /var filesystem in mounted via fstab

Hmm... could you please cat out 

/usr/lib/systemd/system/rpcbind.service

My /usr/lib/systemd/system/rpcbind.service is:
kojack ~ # cat /usr/lib/systemd/system/rpcbind.service
[Unit]
Description=RPC Bind
Documentation=man:rpcbind(8)
DefaultDependencies=no
RequiresMountsFor=/run/rpcbind

# Make sure we use the IP addresses listed for
# rpcbind.socket, no matter how this unit is started.
Wants=rpcbind.socket
After=rpcbind.socket

[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/rpcbind
ExecStart=/usr/bin/rpcbind $RPCBIND_OPTIONS -w -f

[Install]
WantedBy=multi-user.target
kojack ~ # 

Which is what comes with the rpm. rpm -qV rpcbind-0.2.4-1.fc25.x86_64 produces no output.

Interestingly the rpcbind fails to start the first time:
Jan 09 11:11:28 kojack.une.edu.au rpcbind[519]: rpcbind: /var/run/rpcbind.lock: No such file or directory

But later (perhaps a dependency on rpc.statd?) rpcbind starts:

Jan 09 11:11:29 kojack.une.edu.au rpc.statd[856]: Version 1.3.3 starting
Jan 09 11:11:29 kojack.une.edu.au audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=rpcbind comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

So after a reboot we have:
kojack ~ # systemctl status rpcbind.service
● rpcbind.service - RPC Bind
   Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2017-01-09 11:11:29 AEDT; 10min ago
     Docs: man:rpcbind(8)
 Main PID: 799 (rpcbind)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/rpcbind.service
           └─799 /usr/bin/rpcbind -w -f

Jan 09 11:11:29 kojack.une.edu.au systemd[1]: Starting RPC Bind...
Jan 09 11:11:29 kojack.une.edu.au systemd[1]: Started RPC Bind.

But the first startup of rpcbind (at 11:11:28) failed. This caused ypbind to fail and my system is only partially running.

I "fix" it by making /etc/systemd/system/rpcbind.service that contains (as suggested in comment #4):

[Unit]
Description=RPC Bind
Documentation=man:rpcbind(8)
DefaultDependencies=no
RequiresMountsFor=/run/rpcbind

# Make sure we use the IP addresses listed for
# rpcbind.socket, no matter how this unit is started.
Wants=rpcbind.socket
After=rpcbind.socket

Wants=systemd-tmpfiles-setup.service
After=systemd-tmpfiles-setup.service

[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/rpcbind
ExecStart=/usr/bin/rpcbind $RPCBIND_OPTIONS -w -f

[Install]
WantedBy=multi-user.target

Another thing I note:

kojack ~ # strings /usr/sbin/rpcbind | grep '/run/'
/var/run/rpcbind.sock
/var/run/rpcbind.lock
/run/rpcbind
/run/rpcbind/rpcbind.xdr
/run/rpcbind/portmap.xdr
kojack ~ # 

So there is still a /var/run path in the binary. Perhaps there where more places that "statedir" where set?

rpcbind-0.2.3-14.rc2.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.

Created attachment 1238566 [details]
strace of rpcbind start

rpcbind-0.2.4-1.fc25.x86_64

In the service file I changed the ExecStart line to:

ExecStart=/usr/bin/strace -D /usr/bin/rpcbind $RPCBIND_OPTIONS -w -f

The attached file is the output of:

journalctl -b -l| egrep 'strace|rpcbind'

shortly after rebooting the system.

As you can see from the strace, rpcbind tries to start once and fails to open:
Jan 09 15:45:57 kojack.une.edu.au strace[538]: open("/var/run/rpcbind.lock", O_RDONLY|O_CREAT, 0444) = -1 ENOENT (No such file or directory)

Then seconds later, rpcbind starts again and succeeds:

Jan 09 15:47:16 kojack.une.edu.au strace[1852]: open("/var/run/rpcbind.lock", O_RDONLY|O_CREAT, 0444) = 8

This is still opening /var/run/rpcbind.lock

(In reply to Norman Gaywood from comment #15)
> Another thing I note:
> 
> kojack ~ # strings /usr/sbin/rpcbind | grep '/run/'
> /var/run/rpcbind.sock
> /var/run/rpcbind.lock
> /run/rpcbind
> /run/rpcbind/rpcbind.xdr
> /run/rpcbind/portmap.xdr
> kojack ~ # 
> 
> So there is still a /var/run path in the binary. Perhaps there where more
> places that "statedir" where set?

Yes! Would you mind opening up another bz saying not
all paths are using the "statedir" variable 

tia!

Created attachment 1238904 [details]
Output of "journalctl -b | grep rpcbind": rcpbind still fails with error access to /run/...

Independent of /var/run in rpcbind binary the service rpcbind still fails at boot with access error on files in directory /run. See the attached extract of "journalctl -b | grep rpcbind"

I have tested with rpcbind-0.2.4-1.fc25.x86_64 . It doesn't solve the problem.

Created attachment 1238908 [details]
/etc/systemd/system/rpcbind.service.d/tmpfiles-setup.conf

I have created a file /etc/systemd/system/rpcbind.service.d/tmpfiles-setup.conf containing the dependencies submitted in comment #4:

Wants=systemd-tmpfiles-setup.service
After=systemd-tmpfiles-setup.service

Now rpcbind is running after booting has finished. Previously rpcbind didn't run after booting.

Since some important network services depend on a running rpcbind process, I suggest to build a new rpcbind package with the lines above included in /usr/lib/systemd/system/rpcbind.service, at least as a workaround until a better solution is found. This should be done for Fedora 24 and 25.

Created attachment 1238910 [details]
rcpbind error access to /run/... before switching root

With file /etc/systemd/system/rpcbind.service.d/tmpfiles-setup.conf the output of rpcbind to the journal is shorter. I have attached a new extract of the output of journalctl -b.

There are still four messages about access error to /run/..., but they are before the kernel is switching root. After that there is only a sucsess message that rpcbind has started. Previously there was other error message lines instead.

So dependencies in file /etc/systemd/system/rpcbind.service.d/tmpfiles-setup.conf does not solve all problems, but at least rpcbind is running.

I have checked for files /run/rpcbind/rpcbind.xdr and /run/rpcbind/portmap.xdr:
They don't exist on my running system, but the directory /run/rpcbind exists.
The error messages tells us that rpcbind "cannot save any registration", so it may be important to also solve the reason for these error messages.

Has anyone an idea?

(In reply to Steve Dickson from comment #18)
> > So there is still a /var/run path in the binary. Perhaps there where more
> > places that "statedir" where set?
> 
> Yes! Would you mind opening up another bz saying not
> all paths are using the "statedir" variable 

I can do that if you want, but why not use this bug?

I also see that this bug has been re-opened.

So I won't start another bug unless you ask me again :-)

Created attachment 1238928 [details]
Patch which replaces /var/run by /run in rpcbind sources

I have downloaded rpcbind-0.2.4-1.fc25.src.rpm and extracted the sources.
Then I have searched the sources for /var/run and replaced it by /run.
Here is a patch which does this.

Please can someone apply it to the sources (or at least apply it to a new package build)?

Thanks!

rpcbind-0.2.4-1.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

(In reply to Edgar Hoch from comment #23)
> Created attachment 1238928 [details]
> Patch which replaces /var/run by /run in rpcbind sources
> 
> I have downloaded rpcbind-0.2.4-1.fc25.src.rpm and extracted the sources.
> Then I have searched the sources for /var/run and replaced it by /run.
> Here is a patch which does this.
> 
> Please can someone apply it to the sources (or at least apply it to a new
> package build)?
> 
> Thanks!
A couple things... first thanks! Second this patch has to
go to upstream To: libtirpc-devel.net,
cc-ing: linux-nfs.org in the appropriate 
patch format and finally
  
Instead of hard code the path for the rpcbind.lock

--- rpcbind-0.2.4.orig/src/rpcbind.c	2016-11-28 21:47:28.000000000 +0100
+++ rpcbind-0.2.4.orig/src/rpcbind.c	2017-01-10 01:04:02.081414774 +0100
@@ -106,7 +106,7 @@ 
 /* who to suid to if -s is given */
 #define RUN_AS  "daemon"
 
-#define RPCBINDDLOCK "/var/run/rpcbind.lock"
+#define RPCBINDDLOCK "/run/rpcbind.lock"
 rpcbind.lock
I would rather make it dynamic like RPCBFILE, so the
above would change to

#define RPCBINDDLOCK RPCBIND_STATEDIR "/rpcbind.lock"

(In reply to Steve Dickson from comment #25)
> (In reply to Edgar Hoch from comment #23)
> > Created attachment 1238928 [details]
> > Patch which replaces /var/run by /run in rpcbind sources
> > 
> > I have downloaded rpcbind-0.2.4-1.fc25.src.rpm and extracted the sources.
> > Then I have searched the sources for /var/run and replaced it by /run.
> > Here is a patch which does this.
> > 
> > Please can someone apply it to the sources (or at least apply it to a new
> > package build)?
> > 
> > Thanks!
> A couple things... first thanks! Second this patch has to
> go to upstream To: libtirpc-devel.net,
> cc-ing: linux-nfs.org in the appropriate 
> patch format and finally

This is not true... since the upstream does no use systemd-tmpfiles
to make sure the directory exits. 

>   
> Instead of hard code the path for the rpcbind.lock
> 
> --- rpcbind-0.2.4.orig/src/rpcbind.c	2016-11-28 21:47:28.000000000 +0100
> +++ rpcbind-0.2.4.orig/src/rpcbind.c	2017-01-10 01:04:02.081414774 +0100
> @@ -106,7 +106,7 @@ 
>  /* who to suid to if -s is given */
>  #define RUN_AS  "daemon"
>  
> -#define RPCBINDDLOCK "/var/run/rpcbind.lock"
> +#define RPCBINDDLOCK "/run/rpcbind.lock"
>  rpcbind.lock
> I would rather make it dynamic like RPCBFILE, so the
> above would change to
> 
> #define RPCBINDDLOCK RPCBIND_STATEDIR "/rpcbind.lock"
I am going to make this change to keep everything in
one spot.

rpcbind-0.2.4-2.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-516e3fbb90

rpcbind-0.2.3-15.rc2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-ae8bb3b08f

rpcbind-0.2.3-15.rc2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-ae8bb3b08f

rpcbind-0.2.4-2.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-516e3fbb90

rpcbind-0.2.4-2.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

rpcbind-0.2.3-15.rc2.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.

Thanks for patching and updating rpcbind. But the new packages don't solve the problem. rpcbind still fails to start at boot time as described in this bug description, but now fails with /run/rpcbind/rpcbind.lock instead of /var/run/rpcbind.lock:

rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory

The workaround of comment #20 is still neccessary to sucessfully start rpcbind at boot time.

Package tested: rpcbind-0.2.4-2.fc25.x86_64


By the way: /var/run is still used in package binaries:

# for f in $( rpm -ql rpcbind) ; do strings - -f $f |grep /var ; done
/usr/bin/rpcbind: /var/runA
/usr/bin/rpcbind: /var/run/rpcbind.sock
/usr/bin/rpcinfo: /var/run
/usr/sbin/rpcbind: /var/runA
/usr/sbin/rpcbind: /var/run/rpcbind.sock
/usr/sbin/rpcinfo: /var/run

I can also confirm that rpcbind-0.2.4-2.fc25.x86_64 does not fix the problem.

I still have to add these lines to the service file (as per above comments):

Wants=systemd-tmpfiles-setup.service
After=systemd-tmpfiles-setup.service

As before rpcbind fails to start:

Jan 22 09:43:43 myhost rpcbind[696]: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
Jan 22 09:43:43 myhost systemd[1]: rpcbind.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 09:43:43 myhost systemd[1]: Failed to start RPC Bind.
Jan 22 09:43:43 myhost systemd[1]: Dependency failed for NIS/YP (Network Information Service) Server.
Jan 22 09:43:43 myhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=rpcbind comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Jan 22 09:43:43 myhost systemd[1]: ypserv.service: Job ypserv.service/start failed with result 'dependency'.
Jan 22 09:43:43 myhost systemd[1]: Dependency failed for NIS/YP (Network Information Service) Clients to NIS Domain Binder.
Jan 22 09:43:43 myhost systemd[1]: ypbind.service: Job ypbind.service/start failed with result 'dependency'.
Jan 22 09:43:43 myhost systemd[1]: rpcbind.service: Unit entered failed state.
Jan 22 09:43:43 myhost systemd[1]: rpcbind.service: Failed with result 'exit-code'.

Then, a second or two later, starts OK:

Jan 22 09:43:45 myhost systemd[1]: Started RPC Bind.
Jan 22 09:43:45 myhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=rpcbind comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 22 09:43:45 myhost rpc.statd[1318]: Version 1.3.3 starting
Jan 22 09:43:45 myhost rpc.statd[1318]: Flags: TI-RPC

Except that starting then does not fix the ypserv and ypbind that failed on the first attempt.

Same for me.

The update from rpcbind 0.2.3-11.rc1.fc25 to 0.2.4-2.fc25 on my FC25 systems have introduced this bug.

rpcbind does seem to be running after boot but ypbind isn't so there is no NIS services available for login.

Jan 22 07:56:10 study rpcbind: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory

Downgrading to rpcbind-0.2.3-11.rc1.fc25 works around the problem, for now, for me.

https://bugzilla.redhat.com/show_bug.cgi?id=1415496
(In reply to Terry Barnaby from comment #35)
> The update from rpcbind 0.2.3-11.rc1.fc25 to 0.2.4-2.fc25 on my FC25 systems
> have introduced this bug.
Me top. For me, 0.2.4-2.fc25 introduced this bug, which miserably broke yp/nis:

cf.: https://bugzilla.redhat.com/show_bug.cgi?id=1415496

This problem did not occur for me on F24 until the upgrade to rpcbind-0.2.3-15.rc2.fc24.x86_64 this morning.  The service now fails to start because there is no /run/rpcbind/ directory.  The lock file is rather located at just "/run/rpcbind.lock".

Running "mkdir /run/rpcbind ; restorecon -v /run/rpcbind" corrects the problem and allows the service to start.

If we're changing the path, the new RPM needs to create the new directory.

(In reply to Paul Stauffer from comment #37)
> Running "mkdir /run/rpcbind ; restorecon -v /run/rpcbind" corrects the
> problem and allows the service to start.

I think the right methode to create /run/rpcbind would be

systemd-tmpfiles --create /usr/lib/tmpfiles.d/rpcbind.conf


Otherwise you have to ensure that owner, group, mode and fcontext are right (this is what systemd-tmpfiles do):

install -v -d -o rpc -g rpc -m u+rwx,go= /run/rpcbind ; restorecon /run/rpcbind

Same here with:

ypbind-1.38-7.fc25.x86_64
rpcbind-0.2.4-2.fc25.x86_64

ypbind didn't start after a reboot today:

# grep -1 -e rpcbind -e ypbind /var/log/messages
Jan 23 09:01:37 localhost kernel: Installing knfsd (copyright (C) 1996 okir.de).
Jan 23 09:01:37 localhost rpcbind: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
Jan 23 09:01:37 localhost systemd: rpcbind.service: Main process exited, code=exited, status=1/FAILURE
Jan 23 09:01:37 localhost systemd: Failed to start RPC Bind.
Jan 23 09:01:37 localhost systemd: Dependency failed for NIS/YP (Network Information Service) Clients to NIS Domain Binder.
Jan 23 09:01:37 localhost systemd: ypbind.service: Job ypbind.service/start failed with result 'dependency'.
Jan 23 09:01:37 localhost systemd: rpcbind.service: Unit entered failed state.
Jan 23 09:01:37 localhost systemd: rpcbind.service: Failed with result 'exit-code'.
Jan 23 09:01:37 localhost systemd: Reached target User and Group Name Lookups.

[I beg to differ with the last line...]

Some minutes later after noticing the ypbind problem, 'systemctl restart ypbind' worked.  Looks like something else required rpcbind and it started 20 seconds later at 9:01:57:

systemctl status rpcbind
● rpcbind.service - RPC Bind
   Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2017-01-23 09:01:57 EST; 41min ago
     Docs: man:rpcbind(8)
 Main PID: 2720 (rpcbind)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/rpcbind.service
           └─2720 /usr/bin/rpcbind -w -f


unfortuntely this didn't save the day:

cat /etc/systemd/system/ypbind.service.d/systemd-restart=always.conf
[Service]
Restart=always

Yep, rpcbind-0.2.4-2.fc25.x86_64 and rpcbind-0.2.3-15.rc2.fc24.x86_64 breaks ypbind, the problem is the path of the rpcbind.socket. Btw. there is also a

/usr/lib/systemd/system/rpcbind.socket

file with

[Socket]
ListenStream=/run/rpcbind.sock

So if you change the path of the rpcbind socket you need to update the /usr/lib/systemd/system/rpcbind.socket file as well.

Hmm... forget my last comment, my rpcbind starts via rpcbind.socket and I thought the path of the socket changed too and so my rpcbind fails to start but the socket path is still /var/run/rpcbind.sock only the lock file changed to /run/rpcbind/rpcbind.lock. So my problems is fixed with the workaround from comment #20

Even with the workaround from comment #20 I had the problem that ypbind fails to start. So I compared the new /usr/lib/systemd/system/rpcbind.service file with the old one and saw that the new file has

[Unit]
DefaultDependencies=no

in it.

I changed DefaultDependencies=no to DefaultDependencies=yes (which is the default) and now ypbind works again after a reboot.


So the workaround for ypbind which works now perfectly :-) for me is to create a the file

/etc/systemd/system/rpcbind.service.d/DefaultDependencies-fix.conf 

with

[Unit]
DefaultDependencies=yes

in it. I don't need the fix from comment #20 anymore.

rpcbind-0.2.3-16.rc2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-a07577f460

rpcbind-0.2.4-3.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-c43a81dc30

I create the same conf files in comment #42 for rpcbind.service and also for rpcbind.socket. With these conf files, NIS and NFS works fine for me.

From comment #4
> However, the packagers probably had some reason for removing the default
> dependencies, even though its removal caused this bug.

Are there any reason?

Just tried rpcbind-0.2.4-3.fc25 from Koji. This did not work for me.

Jan 24 05:54:48 study rpcbind: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
Jan 24 05:54:48 study systemd: rpcbind.service: Main process exited, code=exited, status=1/FAILURE
Jan 24 05:54:48 study systemd: Failed to start RPC Bind.

I also experience the same problems, after upgrading to rpcbind-0.2.4-2.fc25.i686,
I will now try with rpcbind-0.2.4-3.fc25

The new rpcbind-0.2.4-3.fc25 does NOT solve the problem.  I first resorted to
the workaround of comment #20 above, but I got

[/etc/systemd/system/rpcbind.service.d/tmpfiles-setup.conf:6] Assignment outside of section. Ignoring.
[/etc/systemd/system/rpcbind.service.d/tmpfiles-setup.conf:7] Assignment outside of section. Ignoring.
rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
rpcbind.service: Main process exited, code=exited, status=1/FAILURE
Failed to start RPC Bind.

Perhaps there is a missing "[Unit]" in that file.  Finally I solved by copying
/lib/systemd/system/rpcbind.service in /etc/systemd/system/ and adding the two
new lines of comment #4

(In reply to Maurizio Paolini from comment #48)
> The new rpcbind-0.2.4-3.fc25 does NOT solve the problem.  I first resorted to
> the workaround of comment #20 above, but I got
> ...
> Perhaps there is a missing "[Unit]" in that file.

Sorry, yes, of course, I didn't mention this in comment #20:
The directives "Wants" and "After" are only allowed in the unit section.

$ cat /etc/systemd/system/rpcbind.service.d/tmpfiles-setup.conf 
[Unit]
Wants=systemd-tmpfiles-setup.service
After=systemd-tmpfiles-setup.service

rpcbind-0.2.3-16.rc2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-a07577f460

Steve, AFAIS, you seem to have missed an important detail:

Your RPM doesn't use YOUR rpmbind.socket and rpmbind.service files. It uses the versions from rpcbind-0.2.4/systemd/* inside of patched tarball.

I.e. if you want to us to use your version as in git, you either need to modify your spec or to update the patches.

rpcbind-0.2.3-17.rc2.fc24 rpcbind-0.2.3-17.rc2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-44ec8c7a0e

rpcbind-0.2.3-17.rc2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-44ec8c7a0e

Any idea when rpcbind-0.2.4-4.fc25 is going to hit updates-testing?

rpcbind-0.2.4-4.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-ad932714f2

(In reply to Joe Doss from comment #54)
> Any idea when rpcbind-0.2.4-4.fc25 is going to hit updates-testing?

Sorry about that... I thought I did a few days ago.

No problem Steve. Thanks for that! This bug has plagued our dev boxes that use Vagrant with a shared NFS mount to the host machine. 

I left a comment on https://bodhi.fedoraproject.org/updates/FEDORA-2017-ad932714f2 that might cause issues for some other folks. Without the most recent selinux-policy, rpcbind-0.2.4-4.fc25 will fail to start still due to permissions issues on rpcbind.socket and SELinux. 

An example of this edge case happens when Vagrant auto installs the packages needed for setting up an NFS shared folder between the VM and the host OS. Vagrant is smart enough to install the missing NFS packages, but it doesn't know to update selinux-policy, so it fails when using a fresh Fedora 25 cloud-base image. It will probably go away once the cloud base images for Vagrant gets updated.

Let's hope we can get some votes soon to push this fix to stable! :)

Many thanks, rpcbind-0.2.4-4.fc25 works for ne to.

(In reply to Steve Dickson from comment #56)
> Sorry about that... I thought I did a few days ago.
AFAICT, there was as bug/defect/hickup in bodhi a couple of days ago, which did not split multiple updates correctly :-(

BTW: rpcbind-0.2.4-4.fc25 works for me, too.

Can more people test that the F24 package works properly?  As with Joe, this has been severely impacting the Vagrant workflow for my team

https://bodhi.fedoraproject.org/updates/FEDORA-2017-44ec8c7a0e

0.2.3-17.rc2.fc24 fixed my problem with ypbind.

Strangely, bodhi.fedoraproject.org does not let me login using "konqueror".
I had to login (and add karma) using firefox
(I know this is not the right place to report this, sorry)

rpcbind-0.2.4-4.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-ad932714f2

rpcbind-0.2.4-4.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

rpcbind-0.2.3-17.rc2.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.

I don't know if this is exactly the same issue, but I still see some error during boot regarding rpcbind:

Mar  7 09:41:25 localhost audit: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=iscsid comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar  7 09:41:25 localhost kernel: RPC: Registered named UNIX socket transport module.
Mar  7 09:41:25 localhost kernel: RPC: Registered udp transport module.
Mar  7 09:41:25 localhost kernel: RPC: Registered tcp transport module.
Mar  7 09:41:25 localhost kernel: RPC: Registered tcp NFSv4.1 backchannel transport module.
Mar  7 09:41:25 localhost dracut-pre-udev: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
Mar  7 09:41:25 localhost rpc.statd[372]: Version 2.1.1 starting
Mar  7 09:41:25 localhost rpc.statd[372]: Initializing NSM state
Mar  7 09:41:25 localhost rpc.statd[372]: Running as root.  chown /var/lib/nfs/statd to choose different user
Mar  7 09:41:25 localhost rpc.statd[372]: Failed to register (statd, 1, udp): svc_reg() err: RPC: Remote system error - Connection refused
Mar  7 09:41:25 localhost rpc.statd[372]: Failed to register (statd, 1, tcp): svc_reg() err: RPC: Remote system error - Connection refused
Mar  7 09:41:25 localhost rpc.statd[372]: Failed to register (statd, 1, udp6): svc_reg() err: RPC: Remote system error - Connection refused
Mar  7 09:41:25 localhost rpc.statd[372]: Failed to register (statd, 1, tcp6): svc_reg() err: RPC: Remote system error - Connection refused
Mar  7 09:41:25 localhost rpc.statd[372]: failed to create RPC listeners, exiting
Mar  7 09:41:25 localhost systemd: Started dracut pre-udev hook.

I have rpcbind-0.2.4-4.fc25.x86_64

I open up https://bugzilla.redhat.com/show_bug.cgi?id=1430915
because dracut-pre-udev is not starting rpcbind via systemd
which is expected.

With a freshly installed F25 I still need to manually create the temporary files as described in Comment 38:

systemd-tmpfiles --create /usr/lib/tmpfiles.d/rpcbind.conf


With that a

systemctl start rpcbind

succeeds.

Without, it failed with:

Mar 12 17:30:45 example.org systemd[1]: Starting RPC Bind...
Mar 12 17:30:45 example.org rpcbind[1783]: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
Mar 12 17:30:45 example.org systemd[1]: rpcbind.service: Main process exited, code=exited, status=1/FAILURE
Mar 12 17:30:45 example.org systemd[1]: Failed to start RPC Bind.


$ rpm -q rpcbind 
rpcbind-0.2.4-4.fc25.x86_64

this is still broken after installing fedora25 with latest updates

rpcbind-0.2.4-5.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-66c84b1ba8

rpcbind-0.2.4-6.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-5c5fcde1a0

rpcbind-0.2.4-6.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-5c5fcde1a0

rpcbind-0.2.4-5.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-66c84b1ba8

rpcbind-0.2.4-5.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

rpcbind-0.2.4-6.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

*** Bug 1424940 has been marked as a duplicate of this bug. ***

Issue is still here with rpcbind-0.2.4-5.fc25

(In reply to Maël Lavault from comment #77)
> Issue is still here with rpcbind-0.2.4-5.fc25

This must be a different problem then...

With rpcbind-0.2.4-5.rc1.fc25.x86_64 after reboot ypbind was failing and showing these errors:
May 15 10:56:35 ourserver systemd: Starting RPC Bind...
May 15 10:56:35 ourserver rpcbind: rpcbind: pmap_prot.c:50: xdr_pmap: Assertion `regs != NULL' failed.
May 15 10:56:35 ourserver systemd: Started RPC Bind.
May 15 10:56:35 ourserver systemd: rpcbind.service: Main process exited, code=killed, status=6/ABRT
May 15 10:56:35 ourserver systemd: rpcbind.service: Unit entered failed state.
May 15 10:56:35 ourserver systemd: rpcbind.service: Failed with result 'signal'.

Oddly rpcbind status was showing ok just a failure on ypbind:
-- Unit rpcbind.service has finished starting up.
-- 
-- The start-up result is done.
May 15 10:58:15 ourserver systemd[1]: rpcbind.service: Main process exited, code=killed, status=6/ABRT
May 15 10:58:15 ourserver systemd[1]: rpcbind.service: Unit entered failed state.
May 15 10:58:15 ourserver systemd[1]: rpcbind.service: Failed with result 'signal'.
May 15 10:58:17 ourserver ypbind[2631]: Binding took 46 seconds
May 15 10:58:17 ourserver ypbind[2633]: NIS server for domain divscimath is not responding.
May 15 10:58:17 ourserver ypbind[2634]: Killing ypbind with PID 2290.
May 15 10:58:17 ourserver ypbind[2290]: 2292: Signal (15) for quitting program arrived.
May 15 10:58:17 ourserver systemd[1]: Starting RPC Bind...
-- Subject: Unit rpcbind.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit rpcbind.service has begun starting up.
May 15 10:58:17 ourserver ypbind[2635]: Try increase NISTIMEOUT in /etc/sysconfig/ypbind
May 15 10:58:17 ourserver systemd[1]: ypbind.service: Control process exited, code=exited status=1
May 15 10:58:17 ourserver systemd[1]: Started RPC Bind.
-- Subject: Unit rpcbind.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit rpcbind.service has finished starting up.
-- 
-- The start-up result is done.
May 15 10:58:18 ourserver systemd[1]: Failed to start NIS/YP (Network Information Service) Clients to NIS Domain Binder.
-- Subject: Unit ypbind.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit ypbind.service has failed.
-- 
-- The result is failed.
May 15 10:58:18 ourserver systemd[1]: ypbind.service: Unit entered failed state.
May 15 10:58:18 ourserver systemd[1]: ypbind.service: Failed with result 'exit-code'.


Downgraded to rpcbind-0.2.3-11.rc1.fc25.x86_64 and both ypbind and rpcbind start fine.

(In reply to RobbieTheK from comment #79)
> With rpcbind-0.2.4-5.rc1.fc25.x86_64 after reboot ypbind was failing and
> showing these errors:
> ...
> Downgraded to rpcbind-0.2.3-11.rc1.fc25.x86_64 and both ypbind and rpcbind
> start fine.

Why do you try the older rpcbind-0.2.4-5.rc1.fc25.x86_64 from repo updates-testing instead of current rpcbind-0.2.4-5.fc25.x86_64 from repo updates?

(In reply to Edgar Hoch from comment #80)
> (In reply to RobbieTheK from comment #79)
> > With rpcbind-0.2.4-5.rc1.fc25.x86_64 after reboot ypbind was failing and
> > showing these errors:
> > ...
> > Downgraded to rpcbind-0.2.3-11.rc1.fc25.x86_64 and both ypbind and rpcbind
> > start fine.
> 
> Why do you try the older rpcbind-0.2.4-5.rc1.fc25.x86_64 from repo
> updates-testing instead of current rpcbind-0.2.4-5.fc25.x86_64 from repo
> updates?

I guess because rpcbind-0.2.4-5.rc1.fc25 is the current rpm from the repo "updates" and not "updates-testing".

<rant>
FWIW: Today's update to rpcbind-0.2.4-5.fc25.x86_64 had disasterous effects on my network - Don't you guys test what you're shipping?
</rant>

(In reply to JM from comment #81)
> I guess because rpcbind-0.2.4-5.rc1.fc25 is the current rpm from the repo
> "updates" and not "updates-testing".

Sorry, my repo mirror had a bit delay. Now dnf wants to upgrade to rpcbind-0.2.4-5.rc1.fc25 too...

And yes, it is a desaster... Maintainers, please replace rpcbind-0.2.4-5.rc1.fc25 with an newer working version asap - perhaps with previous rpcbind-0.2.4-5.fc25 but renamed to a name newer than rpcbind-0.2.4-5.rc1.fc25 so dnf upgrade works...!

A hint for those admins / users who have not upgraded yet:

Prevent updating of rpcbind by using dnf plugin versionlock.
You may need to install dnf plugin versionlock first.

# dnf install python-dnf-plugins-extras-versionlock
# dnf versionlock add rpcbind-0.2.4-5.fc25.x86_64


If you have already upgraded, then you can downgrade with the following command (and then use versionlock).

dnf downgrade https://kojipkgs.fedoraproject.org//packages/rpcbind/0.2.4/5.fc25/x86_64/rpcbind-0.2.4-5.fc25.x86_64.rpm

Thanks for the hint, Edgar. Much appreciated. Same issues here. Perhaps you should re-open this bug report?

(In reply to Michael J. Chudobiak from comment #85)
> Thanks for the hint, Edgar. Much appreciated. Same issues here. Perhaps you
> should re-open this bug report?

There exists already a new bug report for this problem: bug 1450765

Steve has already build a new packages which solves the problem (see bug 1450765).

Hint for those admins / users who will not wait for the new package to appear in the repo:

dnf upgrade https://kojipkgs.fedoraproject.org//packages/rpcbind/0.2.4/6.rc1.fc25/x86_64/rpcbind-0.2.4-6.rc1.fc25.x86_64.rpm

(In reply to Edgar Hoch from comment #84)
> # dnf versionlock add rpcbind-0.2.4-5.fc25.x86_64

Sorry, I forgot that there is still a bug in dnf-automatic on Fedora 25 - versionlock is ignored. See bug 1402631.

If you are running dnf-automatic, use the dnf command in comment 87.

rpcbind-0:0.2.4-5.rc1.fc25.x86_64 broke my systems again as well :(
Thanks for the work arounds, but really this should not have occurred, again ...