Description of problem:

We did upgrade from 3.7.x to 3.11.x. However, we've found out that glusterfs prefers/uses IPv6 when IPv6 lookup of "localhost" is found in /etc/hosts.

> 1.2.3.4 gfs01.example.com gfs01
> xxx:xxx:xx:::x gfs01.example.com gfs01

Unfortunately, when IPv6 is disabled like so:

> net.ipv6.conf.all.disable_ipv6 = 1

glusterfs won't be able to connect, because IP address cannot and won't be assigned. However, note that IPv6 is configured at eth interface. This leads to errors in log from which, unless running with debug, you cannot determine what the problem is.

> [2017-08-22 10:42:17.099153] D [MSGID: 0] [common-utils.c:327:gf_resolve_ip6] 0-resolver: returning ip-xxx:xxx:xx:::x (port-24007) for hostname: gfs01.example.com and port: 24007
> [2017-08-22 10:42:17.099202] E [socket.c:3287:socket_connect] 0-backup_gluster-client-3: connection attempt on 1.2.3.4:24007 failed, (Cannot assign requested address)
> [2017-08-22 10:42:17.099219] D [socket.c:2449:socket_event_handler] 0-transport: connect failed with some other error than EINPROGRESS or ENOENT, so nothing more to do; disconnecting socket
> [2017-08-22 10:42:17.099228] D [socket.c:686:__socket_shutdown] 0-backup_gluster-client-3: shutdown() returned -1. Transport endpoint is not connected
> [2017-08-22 10:42:17.099233] D [socket.c:733:__socket_disconnect] 0-backup_gluster-client-3: __socket_teardown_connection () failed: Transport endpoint is not connected
> [2017-08-22 10:42:17.099236] D [socket.c:2474:socket_event_handler] 0-transport: EPOLLERR - disconnecting now

This issue wasn't present in 3.7.x. Also, this shouldn't be happening, although I don't have universal solution for all platforms. One of the solutions could be to try to connect all, IPv4 and IPv6, addresses returned from the lookup. Also, notice that log messages do contain IPv6 and IPv4 addresses of the host which can be misleading.


Version-Release number of selected component (if applicable):

3.11.3-1 at Debian Jessie


How reproducible:


Steps to Reproduce:
0. Note - we were doing upgrade from 3.7.x and already had cluster with volumes
1. Install Debian Jessie
2. Have IPv4 and IPv6 entries in /etc/hosts for the server as written above
3. install and start glusterd
4. create GFS volumes
5. glusterfs shouldn't be able to connect to glusterd
6. try to restart glusterd and all gluster processes, if glusterfs was able to connect

Actual results:

glusterfs is unable to connect to glusterd, because IPv6 is disabled in the system.

Expected results:

glusterfs is able to connect to glusterd regardless whether lookup of gfs01.example.com returns IPv6 as well or not, resp. as it used to work in 3.7.x.

Additional info:

Workaround is to comment out IPv6 entry from /etc/hosts for "localhost".