Bug 1312452 - Erlang Port Mapper crashes at startup
Erlang Port Mapper crashes at startup
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: erlang (Show other bugs)
8.0 (Liberty)
x86_64 Linux
unspecified Severity unspecified
: ga
: 8.0 (Liberty)
Assigned To: John Eckersberg
Leonid Natapov
:
Depends On: 1310808
Blocks: 1313511
  Show dependency treegraph
 
Reported: 2016-02-26 12:39 EST by John Eckersberg
Modified: 2016-04-07 17:31 EDT (History)
12 users (show)

See Also:
Fixed In Version: erlang-R16B-03.10min.8.el7ost
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1310808
: 1313511 (view as bug list)
Environment:
Last Closed: 2016-04-07 17:31:33 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description John Eckersberg 2016-02-26 12:39:53 EST
+++ This bug was initially created as a clone of Bug #1310808 +++

Description of problem:

After upgrading from erlang-R16B-03.11 to erlang-R16B-03.13 epmd has started crashing at start up in some of our machines.

Version-Release number of selected component (if applicable):

erlang-R16B-03.13

How reproducible:

Start epmd daemon in a machine without ipv6 address

Steps to Reproduce:
[root@myhost ~]# epmd -debug
epmd: Wed Feb 17 08:27:23 2016: epmd running - daemon = 0
*** buffer overflow detected ***: epmd terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f01fe269ac7]
/lib64/libc.so.6(+0x10bc80)[0x7f01fe267c80]
/lib64/libc.so.6(+0x10da37)[0x7f01fe269a37]
epmd[0x403a96]
epmd[0x401915]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f01fe17db15]
epmd[0x401d0d]
======= Memory map: ========
00400000-0040c000 r-xp 00000000 fd:00 34108967                           /usr/lib64/erlang/erts-5.10.4/bin/epmd
0060b000-0060c000 r--p 0000b000 fd:00 34108967                           /usr/lib64/erlang/erts-5.10.4/bin/epmd
0060c000-0060d000 rw-p 0000c000 fd:00 34108967                           /usr/lib64/erlang/erts-5.10.4/bin/epmd
0149f000-014c0000 rw-p 00000000 00:00 0                                  [heap]
7f01fcc70000-7f01fcc85000 r-xp 00000000 fd:00 50663                      /usr/lib64/libz.so.1.2.7
7f01fcc85000-7f01fce84000 ---p 00015000 fd:00 50663                      /usr/lib64/libz.so.1.2.7
7f01fce84000-7f01fce85000 r--p 00014000 fd:00 50663                      /usr/lib64/libz.so.1.2.7
7f01fce85000-7f01fce86000 rw-p 00015000 fd:00 50663                      /usr/lib64/libz.so.1.2.7
7f01fce86000-7f01fce95000 r-xp 00000000 fd:00 66728                      /usr/lib64/libbz2.so.1.0.6
7f01fce95000-7f01fd094000 ---p 0000f000 fd:00 66728                      /usr/lib64/libbz2.so.1.0.6
7f01fd094000-7f01fd095000 r--p 0000e000 fd:00 66728                      /usr/lib64/libbz2.so.1.0.6
7f01fd095000-7f01fd096000 rw-p 0000f000 fd:00 66728                      /usr/lib64/libbz2.so.1.0.6
7f01fd096000-7f01fd0ba000 r-xp 00000000 fd:00 16885                      /usr/lib64/liblzma.so.5.0.99
7f01fd0ba000-7f01fd2b9000 ---p 00024000 fd:00 16885                      /usr/lib64/liblzma.so.5.0.99
7f01fd2b9000-7f01fd2ba000 r--p 00023000 fd:00 16885                      /usr/lib64/liblzma.so.5.0.99
7f01fd2ba000-7f01fd2bb000 rw-p 00024000 fd:00 16885                      /usr/lib64/liblzma.so.5.0.99
7f01fd2bb000-7f01fd2d0000 r-xp 00000000 fd:00 66743                      /usr/lib64/libelf-0.163.so
7f01fd2d0000-7f01fd4cf000 ---p 00015000 fd:00 66743                      /usr/lib64/libelf-0.163.so
7f01fd4cf000-7f01fd4d0000 r--p 00014000 fd:00 66743                      /usr/lib64/libelf-0.163.so
7f01fd4d0000-7f01fd4d1000 rw-p 00015000 fd:00 66743                      /usr/lib64/libelf-0.163.so
7f01fd4d1000-7f01fd4d5000 r-xp 00000000 fd:00 66753                      /usr/lib64/libattr.so.1.1.0
7f01fd4d5000-7f01fd6d4000 ---p 00004000 fd:00 66753                      /usr/lib64/libattr.so.1.1.0
7f01fd6d4000-7f01fd6d5000 r--p 00003000 fd:00 66753                      /usr/lib64/libattr.so.1.1.0
7f01fd6d5000-7f01fd6d6000 rw-p 00004000 fd:00 66753                      /usr/lib64/libattr.so.1.1.0
7f01fd6d6000-7f01fd6ec000 r-xp 00000000 fd:00 16866                      /usr/lib64/libpthread-2.17.so
7f01fd6ec000-7f01fd8ec000 ---p 00016000 fd:00 16866                      /usr/lib64/libpthread-2.17.so
7f01fd8ec000-7f01fd8ed000 r--p 00016000 fd:00 16866                      /usr/lib64/libpthread-2.17.so
7f01fd8ed000-7f01fd8ee000 rw-p 00017000 fd:00 16866                      /usr/lib64/libpthread-2.17.so
7f01fd8ee000-7f01fd8f2000 rw-p 00000000 00:00 0 
7f01fd8f2000-7f01fd907000 r-xp 00000000 fd:00 142                        /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7f01fd907000-7f01fdb06000 ---p 00015000 fd:00 142                        /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7f01fdb06000-7f01fdb07000 r--p 00014000 fd:00 142                        /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7f01fdb07000-7f01fdb08000 rw-p 00015000 fd:00 142                        /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7f01fdb08000-7f01fdb4d000 r-xp 00000000 fd:00 82543                      /usr/lib64/libdw-0.163.so
7f01fdb4d000-7f01fdd4c000 ---p 00045000 fd:00 82543                      /usr/lib64/libdw-0.163.so
7f01fdd4c000-7f01fdd4e000 r--p 00044000 fd:00 82543                      /usr/lib64/libdw-0.163.so
7f01fdd4e000-7f01fdd4f000 rw-p 00046000 fd:00 82543                      /usr/lib64/libdw-0.163.so
7f01fdd4f000-7f01fdd53000 r-xp 00000000 fd:00 66757                      /usr/lib64/libcap.so.2.22
7f01fdd53000-7f01fdf52000 ---p 00004000 fd:00 66757                      /usr/lib64/libcap.so.2.22
7f01fdf52000-7f01fdf53000 r--p 00003000 fd:00 66757                      /usr/lib64/libcap.so.2.22
7f01fdf53000-7f01fdf54000 rw-p 00004000 fd:00 66757                      /usr/lib64/libcap.so.2.22
7f01fdf54000-7f01fdf5b000 r-xp 00000000 fd:00 16870                      /usr/lib64/librt-2.17.so
7f01fdf5b000-7f01fe15a000 ---p 00007000 fd:00 16870                      /usr/lib64/librt-2.17.so
7f01fe15a000-7f01fe15b000 r--p 00006000 fd:00 16870                      /usr/lib64/librt-2.17.so
7f01fe15b000-7f01fe15c000 rw-p 00007000 fd:00 16870                      /usr/lib64/librt-2.17.so
7f01fe15c000-7f01fe312000 r-xp 00000000 fd:00 16840                      /usr/lib64/libc-2.17.so
7f01fe312000-7f01fe512000 ---p 001b6000 fd:00 16840                      /usr/lib64/libc-2.17.so
7f01fe512000-7f01fe516000 r--p 001b6000 fd:00 16840                      /usr/lib64/libc-2.17.so
7f01fe516000-7f01fe518000 rw-p 001ba000 fd:00 16840                      /usr/lib64/libc-2.17.so
7f01fe518000-7f01fe51d000 rw-p 00000000 00:00 0 
7f01fe51d000-7f01fe61e000 r-xp 00000000 fd:00 16848                      /usr/lib64/libm-2.17.so
7f01fe61e000-7f01fe81d000 ---p 00101000 fd:00 16848                      /usr/lib64/libm-2.17.so
7f01fe81d000-7f01fe81e000 r--p 00100000 fd:00 16848                      /usr/lib64/libm-2.17.so
7f01fe81e000-7f01fe81f000 rw-p 00101000 fd:00 16848                      /usr/lib64/libm-2.17.so
7f01fe81f000-7f01fe822000 r-xp 00000000 fd:00 16846                      /usr/lib64/libdl-2.17.so
7f01fe822000-7f01fea21000 ---p 00003000 fd:00 16846                      /usr/lib64/libdl-2.17.so
7f01fea21000-7f01fea22000 r--p 00002000 fd:00 16846                      /usr/lib64/libdl-2.17.so
7f01fea22000-7f01fea23000 rw-p 00003000 fd:00 16846                      /usr/lib64/libdl-2.17.so
7f01fea23000-7f01fea25000 r-xp 00000000 fd:00 16874                      /usr/lib64/libutil-2.17.so
7f01fea25000-7f01fec24000 ---p 00002000 fd:00 16874                      /usr/lib64/libutil-2.17.so
7f01fec24000-7f01fec25000 r--p 00001000 fd:00 16874                      /usr/lib64/libutil-2.17.soAborted (core dumped)


Actual results:

epmd crashes

Expected results:

epmd daemon starts

Additional info:

strace shows that before the crash:
bind(3, {sa_family=AF_INET, sin_port=htons(4369), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
listen(3, 128)                          = 0
socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = -1 EAFNOSUPPORT (Address family not supported by protocol)
select(4, [3], NULL, NULL, {5, 0})      = 0 (Timeout)
open("/dev/tty", O_RDWR|O_NOCTTY|O_NONBLOCK) = 4
writev(4, [{"*** ", 4}, {"buffer overflow detected", 24}, {" ***: ", 6}, {"epmd", 4}, {" terminated\n", 12}], 5*** buffer overflow detected ***: epmd terminated
) = 50

systemctl start epmd@0.0.0.0 

works fine.

--- Additional comment from John Eckersberg on 2016-02-22 14:32:37 EST ---

Can you attach the core dump?  Merely starting epmd on a host with no IPv6 addresses doesn't seem to be enough to trigger the bug, I'm unable to reproduce:

[root@centos7 ~]# rpm -q erlang-erts
erlang-erts-R16B-03.13.el7.x86_64
[root@centos7 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:87:2d:f5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.147/24 brd 192.168.122.255 scope global dynamic eth0
       valid_lft 2850sec preferred_lft 2850sec
[root@centos7 ~]# epmd -debug
epmd: Mon Feb 22 14:27:20 2016: epmd running - daemon = 0


(no crash for me)

--- Additional comment from Raúl Andrés on 2016-02-23 01:44 EST ---



--- Additional comment from Raúl Andrés on 2016-02-23 02:59:59 EST ---

I have IPv6 support disabled:

[root@myhost tmp]#  cat /proc/net/if_inet6                                
cat: /proc/net/if_inet6: No such file or directory
[root@myhost tmp]#  modprobe -c| grep ipv6
options ipv6 disable=1
Comment 1 John Eckersberg 2016-02-26 12:44:37 EST
This also reproduces on OSP:

[root@rhel7 ~]# rpm -q erlang-erts
erlang-erts-R16B-03.10min.5.el7ost.x86_64
[root@rhel7 ~]# epmd -d 2>&1 | head
epmd: Fri Feb 26 12:38:16 2016: epmd running - daemon = 0
*** buffer overflow detected ***: epmd terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7fc11d6f4b37]
/lib64/libc.so.6(+0x10bcf0)[0x7fc11d6f2cf0]
/lib64/libc.so.6(+0x10daa7)[0x7fc11d6f4aa7]
epmd[0x403a96]
epmd[0x401915]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fc11d608b15]
epmd[0x401d0d]
...

Disable IPv6 first as per https://access.redhat.com/solutions/8709?tour=6#rhel7disable in order to reproduce.
Comment 4 Leonid Natapov 2016-04-05 08:18:50 EDT
erlang-R16B-03.10min.8.el7ost

disabled ipv6 according to https://access.redhat.com/solutions/8709#rhel7disable

no epmd  crash occurs.
Comment 5 errata-xmlrpc 2016-04-07 17:31:33 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0603.html

Note You need to log in before you can comment on or make changes to this bug.