RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1454876 - rpcbind crash on start
Summary: rpcbind crash on start
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: rpcbind
Version: 7.3
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Steve Dickson
QA Contact: Yongcheng Yang
URL:
Whiteboard:
: 1451651 1457963 (view as bug list)
Depends On:
Blocks: 1455142 1456511 1457172
TreeView+ depends on / blocked
 
Reported: 2017-05-23 16:16 UTC by Konstantin Olchanski
Modified: 2020-12-14 08:44 UTC (History)
39 users (show)

Fixed In Version: rpcbind-0.2.0-42.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1455142 1457172 (view as bug list)
Environment:
Last Closed: 2017-08-01 18:36:03 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3053461 0 None None None 2017-05-24 22:03:08 UTC
Red Hat Product Errata RHBA-2017:1992 0 normal SHIPPED_LIVE rpcbind bug fix update 2017-08-01 18:10:39 UTC

Description Konstantin Olchanski 2017-05-23 16:16:49 UTC 
The latest rpcbind update is defective. rpcbind does not run at all, crashes soon after starting.

This is the update:
https://blog.linuxadmins.org/rhsa-20171262-1-important-rpcbind-security-update/
[root@iris01 ~]# rpm -q rpcbind
rpcbind-0.2.0-38.el7_3.x86_64

This is maybe the same bug in fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=1450765

This is the secret bug tracking rpcbind updates:
https://bugzilla.redhat.com/show_bug.cgi?id=1449462

This is what I see when I start rpcbind: (the crash is right after connection from an el6 machine)

[root@iris01 ~]# rpcbind -d -w -f
libtirpc: debug level 1
rpcbind: local: 0 lookup routines :

rpcbind: rpcbind : my address is (null)

rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 3 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 4 to the rpcbind list
rpcbind: check binding for local

rpcbind: udp: 0 lookup routines :

rpcbind: rpcbind : my address is 0.0.0.0.0.111

rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 2 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 3 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 4 to the rpcbind list
rpcbind: check binding for udp

rpcbind: rmtcall fd for udp is 7

rpcbind: tcp: 0 lookup routines :

rpcbind: udp6: 0 lookup routines :

rpcbind: rpcbind : my address is ::.0.111

rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 3 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 4 to the rpcbind list
rpcbind: check binding for udp6

rpcbind: rmtcall fd for udp6 is 11

rpcbind: tcp6: 0 lookup routines :

rpcbind: debugging enabled.
rpcbind: using '/run/rpcbind/rpcbind.xdr' startup file
rpcbind: will start from scratch
rpcbind: using '/run/rpcbind/portmap.xdr' startup file
rpcbind: will start from scratch
rpcbind: pmap_rmtcall callit req for (100004, 2, 2, udp) from 142.90.103.115.236.60 : 
rpcbind: not found

*** Error in `rpcbind': free(): invalid pointer: 0x00007fff48459d90 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7c503)[0x7f78f142a503]
/lib64/libtirpc.so.1(xdr_bytes+0x8b)[0x7f78f1bb20ab]
rpcbind(+0x672b)[0x7f78f1fe972b]
rpcbind(+0x422d)[0x7f78f1fe722d]
/lib64/libtirpc.so.1(svc_getreq_common+0x251)[0x7f78f1bac511]
/lib64/libtirpc.so.1(svc_getreq_poll+0x8b)[0x7f78f1bac6ab]
rpcbind(+0x7832)[0x7f78f1fea832]
rpcbind(+0x3600)[0x7f78f1fe6600]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f78f13cfb35]
rpcbind(+0x3800)[0x7f78f1fe6800]
======= Memory map: ========
7f78e8000000-7f78e8021000 rw-p 00000000 00:00 0 

K.O.
Comment 2 Konstantin Olchanski 2017-05-23 16:20:03 UTC 
I confirm that "yum downgrade" to rpcbind-0.2.0-38.el7.x86_64 resolves the problem, rpcbind runs without crash. K.O.
Comment 3 Konstantin Olchanski 2017-05-23 16:43:55 UTC 
Oops, posted a bogus link to the rpcbind update notice, correct link is this:
http://rhn.redhat.com/errata/RHSA-2017-1262.html
K.O.
Comment 4 Steve Dickson 2017-05-23 17:25:25 UTC 
I'm not able to reproduce this... 

What command are you using to create that call to rpcbind?
Comment 5 Konstantin Olchanski 2017-05-23 18:04:35 UTC 
on my side, rpcbind crashes right out from systemd startup scripts, I tried "-d -w -f" to look at the crash.

on the remote side, I do not know now to read the 6-number "from" reported by rpcbind. the first 4 digits look like a valid IP address of a local machine running el6. the el6 machine afait, does not issue any special rpcbind calls other than the usual NFS and NIS stuff. At each invocation, the IP address is different, but in my 3 tries it was always another el6 machine.

my guess? it is an NIS broadcast, the 100004 printed by rpcbind before the crash is ypserv.

the good news? afaik, impossible to firewall rpcbind to selectively block nis broadcasts...

the bad news? I think I need rpcbind for nfsv3 mounts to work. (yes, confirmed, on machines with dead rpcbind, nfsv3 mounts hang, do not work).

K.O.
Comment 6 Michiel De Wilde 2017-05-24 08:24:33 UTC 
I'm having the rpcbind crash too here, both on RHEL6 and RHEL7 after that security update to rpcbind and libtirpc. The rpcbind service crashes when ypbind tries to bind to the NIS domain.
Comment 7 Marcus Camen 2017-05-24 08:54:29 UTC 
Just to confirm the issue:
At our customers site nearly every NFS server is affected.
The backtrace is the same and a rollback to rpcbind-0.2.0-38.el7.x86_64 does resolve the problem.
Comment 8 Volker Schäfer 2017-05-24 10:26:51 UTC 
Same here, rpcbind dies in combination with yp after the security update to rpcbind-0.2.0-38.el7_3
Comment 9 Konstantin Olchanski 2017-05-24 14:44:34 UTC 
Finally found the tracker for the original problem that the update was supposed to fix:
https://bugzilla.redhat.com/show_bug.cgi?id=1448124
K.O.
Comment 13 Clifford Perry 2017-05-24 19:38:38 UTC 
Hi all, 
This is a reminder that bugzilla is not a support tool. IF you have not yet opened a support case to allow Red Hat to correctly track this for you, please do open a ticket by logging into access.redhat.com and opening a new case with a pointer to this bugzilla and CVE-2017-8779. 

Regards,
Comment 17 Murphy Zhou 2017-05-25 07:56:31 UTC 
This crash was first seen on May 16th
https://bugzilla.redhat.com/show_bug.cgi?id=1450765#c11
Comment 18 Murphy Zhou 2017-05-25 09:07:00 UTC 
Note to reproduce:

Terminal 0: # rpcbind -d -w -f

Terminal 1: # rpcinfo -b 100004 2
Comment 23 Steve Dickson 2017-05-25 11:24:52 UTC 
Yes, I did realize what as happening... But I was playing
around with letting the xdr routines do the allocation
but there seems to a problem there too... 

I'll test the patch out today... thanks!
Comment 24 VUIIS SysAdmin 2017-05-25 18:38:56 UTC 
I am having the same issue with ypbind crashing on both clients and NIS servers. I see this in both RHEL6 and RHEL7. Downgrading to previous versions of rpcbind and libtirpc
do not solve the problem.
Comment 25 andrew2.hart 2017-05-26 10:52:52 UTC 
@ VUIIS SysAdmin 
yum downgrade rpcbind 
+ restarting rpcbind and ypbind should work.
I didn't need to downgrade libtirpc.
Maybe you could run:
rpcbind -df
to see if it is the same problem you have.
Comment 26 VUIIS SysAdmin 2017-05-26 14:50:43 UTC 
@ (In reply to andrew2.hart from comment #25)
> @ VUIIS SysAdmin 
> yum downgrade rpcbind 
> + restarting rpcbind and ypbind should work.
> I didn't need to downgrade libtirpc.
> Maybe you could run:
> rpcbind -df
> to see if it is the same problem you have.

Not sure what I am looking for but this is what I got:

# rpcbind -df
libtirpc: debug level 1
rpcbind: local: 0 lookup routines :

rpcbind: rpcbind : my address is (null)

rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 3 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 4 to the rpcbind list
rpcbind: check binding for local

rpcbind: udp: 0 lookup routines :

rpcbind: rpcbind : my address is 0.0.0.0.0.111

rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 2 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 3 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 4 to the rpcbind list
rpcbind: check binding for udp

rpcbind: rmtcall fd for udp is 7

rpcbind: tcp: 0 lookup routines :

rpcbind: udp6: 0 lookup routines :

rpcbind: rpcbind : my address is ::.0.111

rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 3 to the rpcbind list
rpcbind: FUNCTION rbllist_add
rpcbind: Add the prog 100000 vers 4 to the rpcbind list
rpcbind: check binding for udp6

rpcbind: rmtcall fd for udp6 is 11

rpcbind: tcp6: 0 lookup routines :

rpcbind: debugging enabled.
rpcbind: PMAP_GETPORT req for (100004, 2, udp) from 10.140.19.237.2.255 :
rpcbind: port = 0

rpcbind: PMAP_GETPORT req for (100004, 2, udp) from 10.140.19.211.3.57 :
rpcbind: port = 0
Comment 27 Frank Sorenson 2017-05-26 18:58:03 UTC 
@VUIIS SysAdmin 

Just a reminder that bugzilla is not a support tool.  If you continue having problems, please open a support ticket at access.redhat.com

Frank
Comment 32 Yongcheng Yang 2017-05-31 13:22:43 UTC 
Moving to VERIFIED according to the test logs of Comment #31.
Comment 33 Aymeric 2017-06-07 15:15:18 UTC 
This issue also affect RHEL6.9 (at least)
If needed we have captured an UDP packet that case rcpbind to crash (just ask me).
Regards,
Aymeric
Comment 34 Frank Sorenson 2017-06-07 20:29:31 UTC 
(In reply to Aymeric from comment #33)
> This issue also affect RHEL6.9 (at least)
> If needed we have captured an UDP packet that case rcpbind to crash (just
> ask me).

The reproducer is well understood.  The bugzilla tracking this issue for 6.9 is bz1458240
Comment 35 J. Bruce Fields 2017-06-09 14:14:12 UTC 
*** Bug 1451651 has been marked as a duplicate of this bug. ***
Comment 36 Steve Dickson 2017-06-15 15:58:07 UTC 
*** Bug 1457963 has been marked as a duplicate of this bug. ***
Comment 37 jeremiah 2017-06-20 01:15:15 UTC 
Looks like only x86 was fixed but not also ARM platform. Do I need to make a new bug report for that?
Comment 38 Yongcheng Yang 2017-06-21 04:24:12 UTC 
(In reply to jeremiah from comment #37)
> Looks like only x86 was fixed but not also ARM platform. Do I need to make a
> new bug report for that?

I have checked this issue fixed in rpcbind-0.2.0-42.el7.aarch64.

Would you please have a test within your ARM platform?

------------------------------------------------------
[21:18:45 root@ ~~]# service_rpcbind restart
Redirecting to /bin/systemctl restart rpcbind.service
[21:18:45 root@ ~~]# ps aux | grep [r]pcbind
rpc      10956  4.0  0.0  11712  7232 ?        Ss   21:18   0:00 /sbin/rpcbind -w
[21:18:45 root@ ~~]# rpcinfo -b 100004 2
10.12.0.163.2.150	pluto.lab.eng.rdu.redhat.com
10.12.0.161.2.250	eagle.lab.eng.rdu.redhat.com
10.12.0.158.2.124	bsod-bdc.lab.eng.rdu.redhat.com
10.12.0.159.2.139	bsod2.lab.eng.rdu.redhat.com
10.12.0.162.3.33	longhaul.lab.eng.rdu.redhat.com
10.12.0.157.3.60	bsod.lab.eng.rdu.redhat.com
10.12.0.159.2.139	bsod2.lab.eng.rdu.redhat.com
10.12.0.163.2.150	pluto.lab.eng.rdu.redhat.com
10.12.0.161.2.250	eagle.lab.eng.rdu.redhat.com
10.12.0.158.2.124	bsod-bdc.lab.eng.rdu.redhat.com
10.12.0.162.3.33	longhaul.lab.eng.rdu.redhat.com
10.12.0.157.3.60	bsod.lab.eng.rdu.redhat.com
[21:18:58 root@ ~~]# ps aux | grep [r]pcbind
rpc      10956  0.2  0.0  11712  7232 ?        Ss   21:18   0:00 /sbin/rpcbind -w
Comment 40 jeremiah 2017-06-27 22:18:56 UTC 
> I have checked this issue fixed in rpcbind-0.2.0-42.el7.aarch64.
>
> Would you please have a test within your ARM platform?

Ah. Apparently this was never processed downstream to any other distribution (namely CentOS). That is, of course, not RedHat's responsibility so pardon my noise & thanks for the reply!
Comment 41 errata-xmlrpc 2017-08-01 18:36:03 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1992
Note You need to log in before you can comment on or make changes to this bug.