Bug 1461493

Summary:	inet:gethostbyname returns IPv4 results when IPv6 is configured [OSP-10]
Product:	Red Hat OpenStack	Reporter:	John Eckersberg <jeckersb>
Component:	erlang	Assignee:	Peter Lemenkov <plemenko>
Status:	CLOSED ERRATA	QA Contact:	Udi Shkalim <ushkalim>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	10.0 (Newton)	CC:	achernet, apevec, jeckersb, jschluet, lhh, michele, plemenko, ushkalim
Target Milestone:	z4	Keywords:	Triaged, ZStream
Target Release:	10.0 (Newton)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	erlang-18.3.4.5-3.el7ost.1	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1461190	Environment:
Last Closed:	2017-09-06 17:06:29 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description John Eckersberg 2017-06-14 15:10:56 UTC

+++ This bug was initially created as a clone of Bug #1461190 +++

Description of problem:
If IPv6 is configured via proto_dist and/or inetrc, inet:gethostbyname will still prefer IPv4 addresses.  This breaks epmd lookups if epmd is forced to listen only on IPv6.

Version-Release number of selected component (if applicable):
erlang-erts-18.3.4.4-3.el7ost.x86_64

How reproducible:
Always

Steps to Reproduce:
[root@rhel7 ~]# env | grep ERL                                                                                                                                                                                       
ERL_INET_GETHOST_DEBUG=5
ERL_INETRC=/root/inetrc
ERL_EPMD_ADDRESS=fd00::1
[root@rhel7 ~]# cat /root/inetrc
{inet6, true}.
[root@rhel7 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:13:f6:0e brd ff:ff:ff:ff:ff:ff
    inet 192.168.200.200/24 brd 192.168.200.255 scope global dynamic eth0
       valid_lft 2881sec preferred_lft 2881sec
    inet6 fd00::1/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe13:f60e/64 scope link 
       valid_lft forever preferred_lft forever
[root@rhel7 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
fd00::1     rhel7


Actual results:
[root@rhel7 ~]# erl -sname foo -proto_dist inet6_tcp
Erlang/OTP 18 [erts-7.3.1.2] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.3.1.2  (abort with ^G)
(foo@rhel7)1> inet:gethostbyname("rhel7").
inet_gethost[7615] (DEBUG):Saved domainname .
inet_gethost[7615] (DEBUG):Saved domainname .
inet_gethost[7616] (DEBUG):Worker got request, op = 1, proto = 1, data = rhel7.
inet_gethost[7616] (DEBUG):Starting gethostbyname(rhel7)
inet_gethost[7616] (DEBUG):gethostbyname OK
{ok,{hostent,"rhel7.example.org",[],inet,4,
             [{192,168,200,200}]}}


Expected results:
[root@rhel7 ~]# erl -sname foo -proto_dist inet6_tcp
Erlang/OTP 18 [erts-7.3.1.2] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.3.1.2  (abort with ^G)
(foo@rhel7)1> inet:gethostbyname("rhel7").
inet_gethost[7694] (DEBUG):Saved domainname .
inet_gethost[7694] (DEBUG):Created worker[7695] with fd 3
inet_gethost[7694] (DEBUG):Saved domainname .
inet_gethost[7695] (DEBUG):Worker got request, op = 1, proto = 2, data = rhel7.
inet_gethost[7695] (DEBUG):Starting getaddrinfo(rhel7, ...)
inet_gethost[7695] (DEBUG):getaddrinfo returned 0
{ok,{hostent,"rhel7",[],inet6,16,[{64768,0,0,0,0,0,0,1}]}}


Additional info:
Upstream fix on OTP 19 - https://github.com/erlang/otp/commit/5d7dcfc2e0de7e93b29d01f07a2f970720d62f9d
Backport for OTP 18.3.4.4 - https://github.com/jeckersb/otp/commit/5ceafb8276b2d920abb3724450e87d8724d21f8e

Comment 1 Peter Lemenkov 2017-07-14 15:20:17 UTC

Different build (against RHEL 7.3) - erlang-18.3.4.5-3.el7ost.1

Comment 4 Udi Shkalim 2017-08-20 10:13:58 UTC

[root@controller-0 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:44:73:bc brd ff:ff:ff:ff:ff:ff
    inet 192.168.24.11/24 brd 192.168.24.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.24.8/32 brd 192.168.24.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe44:73bc/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP qlen 1000
    link/ether 52:54:00:23:f2:a0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fe23:f2a0/64 scope link 
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP qlen 1000
    link/ether 52:54:00:19:ae:76 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fe19:ae76/64 scope link 
       valid_lft forever preferred_lft forever
5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 62:86:39:dd:81:d1 brd ff:ff:ff:ff:ff:ff
6: br-isolated: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 52:54:00:23:f2:a0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fe23:f2a0/64 scope link 
       valid_lft forever preferred_lft forever
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 52:54:00:19:ae:76 brd ff:ff:ff:ff:ff:ff
    inet6 2620:52:0:13b8:5054:ff:fe3e:a/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe19:ae76/64 scope link 
       valid_lft forever preferred_lft forever
8: vlan50: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether e2:d9:b1:b1:8d:f0 brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.8/24 brd 172.16.0.255 scope global vlan50
       valid_lft forever preferred_lft forever
    inet6 fe80::e0d9:b1ff:feb1:8df0/64 scope link 
       valid_lft forever preferred_lft forever
9: vlan20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 5e:d4:cc:98:d5:1e brd ff:ff:ff:ff:ff:ff
    inet6 fd00:fd00:fd00:2000::12/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fd00:fd00:fd00:2000::10/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::5cd4:ccff:fe98:d51e/64 scope link 
       valid_lft forever preferred_lft forever
10: vlan30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 22:e3:c0:3d:51:b0 brd ff:ff:ff:ff:ff:ff
    inet6 fd00:fd00:fd00:3000::13/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::20e3:c0ff:fe3d:51b0/64 scope link 
       valid_lft forever preferred_lft forever
11: vlan40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether fa:31:10:57:05:3a brd ff:ff:ff:ff:ff:ff
    inet6 fd00:fd00:fd00:4000::1b/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::f831:10ff:fe57:53a/64 scope link 
       valid_lft forever preferred_lft forever
12: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 3e:b5:35:6e:a6:4a brd ff:ff:ff:ff:ff:ff
13: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 1e:11:de:02:27:4d brd ff:ff:ff:ff:ff:ff
14: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65470 qdisc noqueue master ovs-system state UNKNOWN qlen 1000
    link/ether 0e:ae:6f:74:7e:d7 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::cae:6fff:fe74:7ed7/64 scope link 
       valid_lft forever preferred_lft forever
[root@controller-0 ~]#  erl -sname foo -proto_dist inet6_tcp
Erlang/OTP 18 [erts-7.3.1.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.3.1.3  (abort with ^G)
(foo@controller-0)1> erl -sname foo -proto_dist inet6_tcp
(foo@controller-0)1> erl -sname foo -proto_dist inet6_tcp.
* 1: syntax error before: foo
(foo@controller-0)1>  inet:gethostbyname("rhel7").
{error,nxdomain}
(foo@controller-0)2>  inet:gethostbyname("controller-0").
{ok,{hostent,"controller-0",[],inet,4,
             [{192,168,24,8},{192,168,24,11},{172,16,0,8}]}}

Comment 5 John Eckersberg 2017-08-24 16:50:59 UTC

I think you're missing a few things from the original comment required to make this work:

- set environment ERL_INETRC=/root/inetrc , just needs to be some readable path

- set content of above ERL_INETRC file to "{inet6, true}."

- set environment ERL_EPMD_ADDRESS=fd00::1 , replace with some valid ipv6 addr for your host or just add that ipv6 addr to some interface like I did

- ensure /etc/hosts resolves the short hostname correctly to the ipv6 address chosen, such as:

  [root@rhel7 ~]# cat /etc/hosts
  127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
  ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
  fd00::1     rhel7

- (optional) set ERL_INET_GETHOST_DEBUG=5 to get some extra debug output like in my original comment.

Before running the `erl` command, make sure there is no existing epmd process running.  If there is, kill it.  This will ensure epmd is respawned and binds to the correct address provided by ERL_EPMD_ADDRESS.

Comment 6 Udi Shkalim 2017-08-29 15:53:35 UTC

erlang-kernel-18.3.4.5-3.el7ost.1.x86_64


[root@controller-2 ~]# env | grep ERL
ERL_INET_GETHOST_DEBUG=5
ERL_INETRC=/root/inetrc
ERL_EPMD_ADDRESS=fd00:fd00:fd00:2000::12


[root@controller-2 ~]# cat /etc/hosts
# HEADER: This file was autogenerated at 2017-08-29 10:41:55 -0400
# HEADER: by puppet.  While it can still be managed manually, it
# HEADER: is definitely not recommended.
127.0.0.1       localhost       localhost.localdomain localhost4 localhost4.localdomain4
::1     localhost       localhost.localdomain localhost6 localhost6.localdomain6
fd00:fd00:fd00:2000::12 controller-2.localdomain controller-2


[root@controller-2 ~]# ifconfig vlan20
vlan20: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fd00:fd00:fd00:2000::18  prefixlen 64  scopeid 0x0<global>
        inet6 fd00:fd00:fd00:2000::12  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::cc9f:82ff:fe23:a387  prefixlen 64  scopeid 0x20<link>
        ether ce:9f:82:23:a3:87  txqueuelen 1000  (Ethernet)
        RX packets 1617054  bytes 285988454 (272.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1637853  bytes 359321348 (342.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


I stopped the pacemaker cluster just to make sure that we don't have any rabbit process that blocks us.

[root@controller-2 ~]# erl -sname foo -proto_dist inet6_tcp
inet_gethost[356516] (DEBUG):Saved domainname .
inet_gethost[356516] (DEBUG):Created worker[356517] with fd 3
inet_gethost[356516] (DEBUG):Saved domainname .
inet_gethost[356517] (DEBUG):Worker got request, op = 1, proto = 1, data = controller-2.
inet_gethost[356517] (DEBUG):Starting gethostbyname(controller-2)
inet_gethost[356517] (DEBUG):gethostbyname OK
inet_gethost[356516] (DEBUG):End of file while reading from pipe.
inet_gethost[356516] (DEBUG):Erlang has closed.
Erlang/OTP 18 [erts-7.3.1.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.3.1.3  (abort with ^G)
(foo@controller-2)1> inet:gethostbyname("controller-2").
inet_gethost[357622] (DEBUG):Saved domainname .
inet_gethost[357622] (DEBUG):Created worker[357623] with fd 3
inet_gethost[357622] (DEBUG):Saved domainname .
inet_gethost[357623] (DEBUG):Worker got request, op = 1, proto = 2, data = controller-2.
inet_gethost[357623] (DEBUG):Starting getaddrinfo(controller-2, ...)
inet_gethost[357623] (DEBUG):getaddrinfo returned 0
{ok,{hostent,"controller-2.localdomain",[],inet6,16,
             [{64768,64768,64768,8192,0,0,0,18}]}}
(foo@controller-2)2>

Comment 8 errata-xmlrpc 2017-09-06 17:06:29 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2653