(In reply to bitchecker from comment #0)
> Created attachment 1130160 [details]
> /var/log/glusterfs/mnt-.log
> 
> hi,
> i've a glusterfs storage with two nodes that have a replicated volume.
> 
> server side it works properly:
> 
> # gluster volume heal volume info
> Brick gluster01:/data/brick/volume
> Number of entries: 0
> 
> Brick gluster02:/data/brick/volume
> Number of entries: 0
> 
> # gluster volume info
>  
> Volume Name: volume
> Type: Replicate
> Volume ID: 3c6c72bf-f85d-4c4c-a536-19a75e2ae71b
> Status: Started
> Number of Bricks: 1 x 2 = 2
> Transport-type: tcp
> Bricks:
> Brick1: gluster01:/data/brick/volume
> Brick2: gluster02:/data/brick/volume
> Options Reconfigured:
> auth.allow: {list of ip}
> performance.readdir-ahead: on
> 
> 
> # gluster volume status                                                     
> 
> Status of volume: volume
> Gluster process                             TCP Port  RDMA Port  Online  Pid
> -----------------------------------------------------------------------------
> -
> Brick gluster01:/data/brick/volume      49152     0          Y       2116 
> Brick gluster02:/data/brick/volume      49152     0          Y       1356 
> NFS Server on localhost                     2049      0          Y      
> 2254 
> Self-heal Daemon on localhost               N/A       N/A        Y      
> 2262 
> NFS Server on gluster02                     2049      0          Y      
> 1296 
> Self-heal Daemon on gluster02               N/A       N/A        Y      
> 1316 
>  
> Task Status of Volume volume
> -----------------------------------------------------------------------------
> -
> There are no active volume tasks
> 
> 
> on clients i've the problem because i can't mount the volume:
> 
> # mount.glusterfs glusterfs02:/volume /mnt/
shouldn't the address be gluster02?

> Mount failed. Please check the log file for more details.
> 
> the only log that i've is: /var/log/glusterfs/mnt-.log
> 
> i've attached the log.

From the log it looks like all sub-volumes are down so mount failed,

[MSGID: 108006] [afr-common.c:4015:afr_notify] 0-volume-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.

(In reply to Atin Mukherjee from comment #2)
> shouldn't the address be gluster02?

addresses are fully resolvable.

(In reply to RajeshReddy from comment #3)
> From the log it looks like all sub-volumes are down so mount failed,
> 
> [MSGID: 108006] [afr-common.c:4015:afr_notify] 0-volume-replicate-0: All
> subvolumes are down. Going offline until atleast one of them comes back up.

i've seen that, but with STATUS and INFO command i don't see any problem so i don't know how to solve, i'm searching online but nothing.
you know how solve this?

(In reply to bitchecker from comment #5)
> (In reply to RajeshReddy from comment #3)
> > From the log it looks like all sub-volumes are down so mount failed,
> > 
> > [MSGID: 108006] [afr-common.c:4015:afr_notify] 0-volume-replicate-0: All
> > subvolumes are down. Going offline until atleast one of them comes back up.
> 
> i've seen that, but with STATUS and INFO command i don't see any problem so
> i don't know how to solve, i'm searching online but nothing.
> you know how solve this?

this problem is only on clients, servers can mount volume properly!

[2016-02-23 16:33:23.282375] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-0: DNS resolution failed on host gluster01

[2016-02-23 16:33:23.431491] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-1: DNS resolution failed on host gluster02

Certainly they are not resolvable, atleast the log indicates the same.

(In reply to Atin Mukherjee from comment #7)
> [2016-02-23 16:33:23.282375] E
> [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-0: DNS
> resolution failed on host gluster01
> 
> [2016-02-23 16:33:23.431491] E
> [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-1: DNS
> resolution failed on host gluster02
> 
> Certainly they are not resolvable, atleast the log indicates the same.

they are specified on hosts file, but if i try with ip address i can't even mount volume.

i solved the problem!

the mistake was on name of server, glusterfs servers are known as gluster01 and gluster02 and client need to know them with the same name.


i don't know if is normal that if i use ip address to try mount it doesn't works properly.

after a reboot, clients and servers are unable to mount the volume.

i've two mnt log:
/var/log/glusterfs/mnt.log:

[2016-02-24 15:08:38.101109] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.8 (args: /usr/sbin/glusterfs --volfile-server=gluster01 --volfile-id=/volume /mnt)
[2016-02-24 15:08:38.241023] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-02-24 15:08:38.430844] E [MSGID: 127002] [quick-read.c:835:check_cache_size_ok] 0-volume-quick-read: Cache size 1073741824 is greater than the max size of 1016430592
[2016-02-24 15:08:38.430890] E [MSGID: 101019] [xlator.c:433:xlator_init] 0-volume-quick-read: Initialization of volume 'volume-quick-read' failed, review your volfile again
[2016-02-24 15:08:38.430907] E [graph.c:322:glusterfs_graph_init] 0-volume-quick-read: initializing translator failed
[2016-02-24 15:08:38.430914] E [graph.c:661:glusterfs_graph_activate] 0-graph: init failed
pending frames:
frame : type(0) op(0)
patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash: 
2016-02-24 15:08:38
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.7.8
[2016-02-24 15:08:38.443995] W [glusterfsd.c:1236:cleanup_and_exit] (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x331) [0x7f17b2ab3801] -->/usr/sbin/glusterfs(glusterfs_process_volfp+0x126) [0x7f17b2aae1a6] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7f17b2aad789] ) 0-: received signum (0), shutting down
[2016-02-24 15:08:38.444110] I [fuse-bridge.c:5685:fini] 0-fuse: Unmounting '/mnt'.


/var/log/glusterfs/mnt-.log:

[2016-02-23 15:58:22.732798] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.8 (args: /usr/sbin/glusterfs --volfile-server=gluster02 --volfile-id=/volume /mnt/)
[2016-02-23 15:58:22.758758] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-02-23 15:58:22.773890] I [MSGID: 114020] [client.c:2106:notify] 0-volume-client-0: parent translators are ready, attempting connect on transport
[2016-02-23 15:58:22.774685] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-02-23 15:58:22.901192] E [MSGID: 101075] [common-utils.c:306:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Nome o servizio sconosciuto)
[2016-02-23 15:58:22.901258] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-0: DNS resolution failed on host gluster01
[2016-02-23 15:58:22.901318] I [MSGID: 114020] [client.c:2106:notify] 0-volume-client-1: parent translators are ready, attempting connect on transport
[2016-02-23 15:58:23.044273] E [MSGID: 101075] [common-utils.c:306:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Nome o servizio sconosciuto)
[2016-02-23 15:58:23.044317] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-1: DNS resolution failed on host gluster02
Final graph:
+------------------------------------------------------------------------------+
  1: volume volume-client-0
  2:     type protocol/client
  3:     option ping-timeout 42
  4:     option remote-host gluster01
  5:     option remote-subvolume /data/brick/volume
  6:     option transport-type socket
  7:     option send-gids true
  8: end-volume
  9:  
 10: volume volume-client-1
 11:     type protocol/client
 12:     option ping-timeout 42
 13:     option remote-host gluster02
 14:     option remote-subvolume /data/brick/volume
 15:     option transport-type socket
 16:     option send-gids true
 17: end-volume
 18:  
 19: volume volume-replicate-0
 20:     type cluster/replicate
 21:     subvolumes volume-client-0 volume-client-1
 22: end-volume
 23:  
 24: volume volume-dht
 25:     type cluster/distribute
 26:     subvolumes volume-replicate-0
 27: end-volume
 28:  
 29: volume volume-write-behind
 30:     type performance/write-behind
 31:     subvolumes volume-dht
 32: end-volume
 33:  
 34: volume volume-read-ahead
 35:     type performance/read-ahead
 36:     subvolumes volume-write-behind
 37: end-volume
 38:  
 39: volume volume-readdir-ahead
 40:     type performance/readdir-ahead
 41:     subvolumes volume-read-ahead
 42: end-volume
 43:  
 44: volume volume-io-cache
 45:     type performance/io-cache
 46:     subvolumes volume-readdir-ahead
 47: end-volume
 48:  
 49: volume volume-quick-read
 50:     type performance/quick-read
 51:     subvolumes volume-io-cache
 52: end-volume
 53:  
 54: volume volume-open-behind
 55:     type performance/open-behind
 56:     subvolumes volume-quick-read
 57: end-volume
 58:  
 59: volume volume-md-cache
 60:     type performance/md-cache
 61:     subvolumes volume-open-behind
 62: end-volume
 63:  
 64: volume volume
 65:     type debug/io-stats
 66:     option latency-measurement off
 67:     option count-fop-hits off
 68:     subvolumes volume-md-cache
 69: end-volume
 70:  
 71: volume meta-autoload
 72:     type meta
 73:     subvolumes volume
 74: end-volume
 75:  
+------------------------------------------------------------------------------+
[2016-02-23 15:58:23.048949] E [MSGID: 108006] [afr-common.c:4015:afr_notify] 0-volume-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2016-02-23 15:58:23.069369] I [fuse-bridge.c:5139:fuse_graph_setup] 0-fuse: switched to graph 0
[2016-02-23 15:58:23.077380] I [fuse-bridge.c:4060:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.22
[2016-02-23 15:58:23.077637] I [MSGID: 108006] [afr-common.c:4143:afr_local_init] 0-volume-replicate-0: no subvolumes up
[2016-02-23 15:58:23.077925] W [fuse-bridge.c:758:fuse_attr_cbk] 0-glusterfs-fuse: 2: LOOKUP() / => -1 (Il socket di destinazione non è connesso)
[2016-02-23 15:58:23.092700] I [fuse-bridge.c:4986:fuse_thread_proc] 0-fuse: unmounting /mnt/
The message "I [MSGID: 108006] [afr-common.c:4143:afr_local_init] 0-volume-replicate-0: no subvolumes up" repeated 2 times between [2016-02-23 15:58:23.077637] and [2016-02-23 15:58:23.091271]
[2016-02-23 15:58:23.093014] W [glusterfsd.c:1236:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7dc5) [0x7f82ca93edc5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7f82cbfa9905] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7f82cbfa9789] ) 0-: received signum (15), shutting down
[2016-02-23 15:58:23.093029] I [fuse-bridge.c:5685:fini] 0-fuse: Unmounting '/mnt/'.
[2016-02-23 16:16:52.150695] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.8 (args: /usr/sbin/glusterfs --volfile-server=gluster02 --volfile-id=/volume /mnt/)
[2016-02-23 16:16:52.181144] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-02-23 16:16:52.199373] I [MSGID: 114020] [client.c:2106:notify] 0-volume-client-0: parent translators are ready, attempting connect on transport
[2016-02-23 16:16:52.200228] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-02-23 16:16:52.320766] E [MSGID: 101075] [common-utils.c:306:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Nome o servizio sconosciuto)
[2016-02-23 16:16:52.320809] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-0: DNS resolution failed on host gluster01
[2016-02-23 16:16:52.320888] I [MSGID: 114020] [client.c:2106:notify] 0-volume-client-1: parent translators are ready, attempting connect on transport
[2016-02-23 16:16:52.437595] E [MSGID: 101075] [common-utils.c:306:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Nome o servizio sconosciuto)
[2016-02-23 16:16:52.437644] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-1: DNS resolution failed on host gluster02
Final graph:
+------------------------------------------------------------------------------+
  1: volume volume-client-0
  2:     type protocol/client
  3:     option ping-timeout 42
  4:     option remote-host gluster01
  5:     option remote-subvolume /data/brick/volume
  6:     option transport-type socket
  7:     option send-gids true
  8: end-volume
  9:  
 10: volume volume-client-1
 11:     type protocol/client
 12:     option ping-timeout 42
 13:     option remote-host gluster02
 14:     option remote-subvolume /data/brick/volume
 15:     option transport-type socket
 16:     option send-gids true
 17: end-volume
 18:  
 19: volume volume-replicate-0
 20:     type cluster/replicate
 21:     subvolumes volume-client-0 volume-client-1
 22: end-volume
 23:  
 24: volume volume-dht
 25:     type cluster/distribute
 26:     subvolumes volume-replicate-0
 27: end-volume
 28:  
 29: volume volume-write-behind
 30:     type performance/write-behind
 31:     subvolumes volume-dht
 32: end-volume
 33:  
 34: volume volume-read-ahead
 35:     type performance/read-ahead
 36:     subvolumes volume-write-behind
 37: end-volume
 38:  
 39: volume volume-readdir-ahead
 40:     type performance/readdir-ahead
 41:     subvolumes volume-read-ahead
 42: end-volume
 43:  
 44: volume volume-io-cache
 45:     type performance/io-cache
 46:     subvolumes volume-readdir-ahead
 47: end-volume
 48:  
 49: volume volume-quick-read
 50:     type performance/quick-read
 51:     subvolumes volume-io-cache
 52: end-volume
 53:  
 54: volume volume-open-behind
 55:     type performance/open-behind
 56:     subvolumes volume-quick-read
 57: end-volume
 58:  
 59: volume volume-md-cache
 60:     type performance/md-cache
 61:     subvolumes volume-open-behind
 62: end-volume
 63:  
 64: volume volume
 65:     type debug/io-stats
 66:     option latency-measurement off
 67:     option count-fop-hits off
 68:     subvolumes volume-md-cache
 69: end-volume
 70:  
 71: volume meta-autoload
 72:     type meta
 73:     subvolumes volume
 74: end-volume
 75:  
+------------------------------------------------------------------------------+
[2016-02-23 16:16:52.441886] E [MSGID: 108006] [afr-common.c:4015:afr_notify] 0-volume-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2016-02-23 16:16:52.460512] I [fuse-bridge.c:5139:fuse_graph_setup] 0-fuse: switched to graph 0
[2016-02-23 16:16:52.460789] I [fuse-bridge.c:4060:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.22
[2016-02-23 16:16:52.460979] I [MSGID: 108006] [afr-common.c:4143:afr_local_init] 0-volume-replicate-0: no subvolumes up
[2016-02-23 16:16:52.461257] W [fuse-bridge.c:758:fuse_attr_cbk] 0-glusterfs-fuse: 2: LOOKUP() / => -1 (Il socket di destinazione non è connesso)
[2016-02-23 16:16:52.468348] I [fuse-bridge.c:4986:fuse_thread_proc] 0-fuse: unmounting /mnt/
The message "I [MSGID: 108006] [afr-common.c:4143:afr_local_init] 0-volume-replicate-0: no subvolumes up" repeated 2 times between [2016-02-23 16:16:52.460979] and [2016-02-23 16:16:52.466340]
[2016-02-23 16:16:52.468730] W [glusterfsd.c:1236:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7dc5) [0x7fefc4d73dc5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7fefc63de905] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7fefc63de789] ) 0-: received signum (15), shutting down
[2016-02-23 16:16:52.468748] I [fuse-bridge.c:5685:fini] 0-fuse: Unmounting '/mnt/'.
[2016-02-23 16:28:45.534853] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.8 (args: /usr/sbin/glusterfs --volfile-server=glusterfs01 --volfile-id=/volume /mnt/)
[2016-02-23 16:28:45.564469] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-02-23 16:28:45.582692] I [MSGID: 114020] [client.c:2106:notify] 0-volume-client-0: parent translators are ready, attempting connect on transport
[2016-02-23 16:28:45.583405] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-02-23 16:28:45.763697] E [MSGID: 101075] [common-utils.c:306:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Nome o servizio sconosciuto)
[2016-02-23 16:28:45.763738] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-0: DNS resolution failed on host gluster01
[2016-02-23 16:28:45.763812] I [MSGID: 114020] [client.c:2106:notify] 0-volume-client-1: parent translators are ready, attempting connect on transport
[2016-02-23 16:28:45.940000] E [MSGID: 101075] [common-utils.c:306:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Nome o servizio sconosciuto)
[2016-02-23 16:28:45.940040] E [name.c:247:af_inet_client_get_remote_sockaddr] 0-volume-client-1: DNS resolution failed on host gluster02
Final graph:
+------------------------------------------------------------------------------+
  1: volume volume-client-0
  2:     type protocol/client
  3:     option ping-timeout 42
  4:     option remote-host gluster01
  5:     option remote-subvolume /data/brick/volume
  6:     option transport-type socket
  7:     option send-gids true
  8: end-volume
  9:  
 10: volume volume-client-1
 11:     type protocol/client
 12:     option ping-timeout 42
 13:     option remote-host gluster02
 14:     option remote-subvolume /data/brick/volume
 15:     option transport-type socket
 16:     option send-gids true
 17: end-volume
 18:  
 19: volume volume-replicate-0
 20:     type cluster/replicate
 21:     subvolumes volume-client-0 volume-client-1
 22: end-volume
 23:  
 24: volume volume-dht
 25:     type cluster/distribute
 26:     subvolumes volume-replicate-0
 27: end-volume
 28:  
 29: volume volume-write-behind
 30:     type performance/write-behind
 31:     subvolumes volume-dht
 32: end-volume
 33:  
 34: volume volume-read-ahead
 35:     type performance/read-ahead
 36:     subvolumes volume-write-behind
 37: end-volume
 38:  
 39: volume volume-readdir-ahead
 40:     type performance/readdir-ahead
 41:     subvolumes volume-read-ahead
 42: end-volume
 43:  
 44: volume volume-io-cache
 45:     type performance/io-cache
 46:     subvolumes volume-readdir-ahead
 47: end-volume
 48:  
 49: volume volume-quick-read
 50:     type performance/quick-read
 51:     subvolumes volume-io-cache
 52: end-volume
 53:  
 54: volume volume-open-behind
 55:     type performance/open-behind
 56:     subvolumes volume-quick-read
 57: end-volume
 58:  
 59: volume volume-md-cache
 60:     type performance/md-cache
 61:     subvolumes volume-open-behind
 62: end-volume
 63:  
 64: volume volume
 65:     type debug/io-stats
 66:     option latency-measurement off
 67:     option count-fop-hits off
 68:     subvolumes volume-md-cache
 69: end-volume
 70:  
 71: volume meta-autoload
 72:     type meta
 73:     subvolumes volume
 74: end-volume
 75:  
+------------------------------------------------------------------------------+
[2016-02-23 16:28:45.951727] E [MSGID: 108006] [afr-common.c:4015:afr_notify] 0-volume-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2016-02-23 16:28:45.978554] I [fuse-bridge.c:5139:fuse_graph_setup] 0-fuse: switched to graph 0
[2016-02-23 16:28:45.979002] I [fuse-bridge.c:4060:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.22
[2016-02-23 16:28:45.979204] I [MSGID: 108006] [afr-common.c:4143:afr_local_init] 0-volume-replicate-0: no subvolumes up
[2016-02-23 16:28:45.979502] W [fuse-bridge.c:758:fuse_attr_cbk] 0-glusterfs-fuse: 2: LOOKUP() / => -1 (Il socket di destinazione non è connesso)
[2016-02-23 16:28:45.986834] I [fuse-bridge.c:4986:fuse_thread_proc] 0-fuse: unmounting /mnt/
The message "I [MSGID: 108006] [afr-common.c:4143:afr_local_init] 0-volume-replicate-0: no subvolumes up" repeated 2 times between [2016-02-23 16:28:45.979204] and [2016-02-23 16:28:45.984908]
[2016-02-23 16:28:45.987251] W [glusterfsd.c:1236:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7dc5) [0x7f098a6dbdc5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7f098bd46905] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7f098bd46789] ) 0-: received signum (15), shutting down
[2016-02-23 16:28:45.987269] I [fuse-bridge.c:5685:fini] 0-fuse: Unmounting '/mnt/'.

i've found the problem, is on amount of cache!

Maximum cache will change after reboots and the volume config can be bigger than max cache.

(In reply to bitchecker from comment #11)
> i've found the problem, is on amount of cache!
> 
> Maximum cache will change after reboots and the volume config can be bigger
> than max cache.

Can you please elaborate the problem?

Ignore my previous comment as now I get what you meant!

Also since you are testing upstream bits the bug should be raised with upstream product. I've corrected it.

Following information would be useful to debug this issue further:

1. output of gluster volume info
2. cmd_history.log file of all nodes

# gluster --version
glusterfs 3.7.8 built on Feb  9 2016 06:29:54
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License

# gluster volume info 
 
Volume Name: volume
Type: Replicate
Volume ID: 3c6c72bf-f85d-4c4c-a536-19a75e2ae71b
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: gluster01:/data/brick/volume
Brick2: gluster02:/data/brick/volume
Options Reconfigured:
performance.readdir-ahead: enable
auth.allow: *
cluster.readdir-optimize: on
cluster.lookup-optimize: on
server.allow-insecure: on
features.trash-max-filesize: 1GB
features.trash: on
performance.write-behind-window-size: 10MB
nfs.disable: on
server.root-squash: ON
performance.io-thread-count: 64
performance.io-cache: on
performance.client-io-threads: on
nfs.export-volumes: off
nfs.addr-namelookup: off
performance.cache-size: 500MB


# cat /var/log/glusterfs/cmd_history.log (NODE01)                                                                                                                           
[2016-02-23 16:09:10.183385]  : volume set help : SUCCESS
[2016-02-23 16:09:12.674174]  : volume status : SUCCESS
[2016-02-23 16:09:12.690612]  : volume status : SUCCESS
[2016-02-24 10:28:14.039661]  : volume set volume auth.allow * : SUCCESS
[2016-02-24 10:30:23.813961]  : volume set help : SUCCESS
[2016-02-24 10:30:40.667330]  : volume set volume cluster.readdir-optimize on : SUCCESS
[2016-02-24 10:31:00.538373]  : volume set volume cluster.lookup-optimize on : SUCCESS
[2016-02-24 10:31:09.257735]  : volume set volume server.allow-insecure on : SUCCESS
[2016-02-24 10:31:17.845762]  : volume set volume features.trash-max-filesize 1GB : SUCCESS
[2016-02-24 10:31:27.195250]  : volume set volume features.trash on : SUCCESS
[2016-02-24 10:31:34.849999]  : volume set volume performance.write-behind-window-size 10MB : SUCCESS
[2016-02-24 10:31:53.612553]  : volume set volume nfs.disable on : SUCCESS
[2016-02-24 10:32:01.464254]  : volume set volume server.root-squash ON : SUCCESS
[2016-02-24 10:32:12.944064]  : volume set volume performance.io-thread-count 64 : SUCCESS
[2016-02-24 10:32:21.522073]  : volume set volume performance.io-cache on : SUCCESS
[2016-02-24 10:32:29.544960]  : volume set volume performance.client-io-threads on : SUCCESS
[2016-02-24 10:32:36.873471]  : volume set volume nfs.export-volumes off : SUCCESS
[2016-02-24 10:32:44.317885]  : volume set volume nfs.addr-namelookup off : SUCCESS
[2016-02-24 10:32:51.650657]  : volume set volume performance.readdir-ahead enable : SUCCESS
[2016-02-24 10:33:00.808064]  : volume set volume performance.cache-size 2GB : SUCCESS
[2016-02-24 10:35:57.357085]  : volume set help : SUCCESS
[2016-02-24 10:38:34.864842]  : volume log rotate volume : SUCCESS
[2016-02-24 11:26:22.066994]  : volume set help : SUCCESS
[2016-02-24 11:45:28.202550]  : volume set help : SUCCESS
[2016-02-24 13:47:10.827338]  : volume set help : SUCCESS
[2016-02-24 13:48:02.221481]  : volume set volume performance.cache-size 1GB : SUCCESS
[2016-02-24 14:08:26.806732]  : volume set help : SUCCESS
[2016-02-24 14:09:47.976273]  : volume set help : SUCCESS
[2016-02-24 14:10:51.429318]  : volume status volume : SUCCESS
[2016-02-24 14:28:58.228384]  : volume set help : SUCCESS
[2016-02-24 14:53:51.402021]  : volume set help : SUCCESS
[2016-02-24 15:03:22.970201]  : volume set help : SUCCESS
[2016-02-24 15:11:10.536295]  : volume set help : SUCCESS
[2016-02-24 15:11:21.776935]  : volume status : SUCCESS
[2016-02-24 15:11:21.795461]  : volume status : SUCCESS
[2016-02-24 15:24:25.965889]  : volume set help : SUCCESS
[2016-02-24 15:24:38.642638]  : volume set volume performance.cache-size 0 : FAILED : '0' in 'option cache-size 0' is out of range [4194304 - 34359738368]
[2016-02-24 15:25:21.803863]  : volume set volume performance.cache-size 500MB : SUCCESS
[2016-02-24 15:25:43.293993]  : volume status : SUCCESS
[2016-02-24 15:25:43.299855]  : volume status : SUCCESS
[2016-02-24 15:56:26.525493]  : volume set help : SUCCESS
[2016-02-25 08:27:51.329361]  : volume set help : SUCCESS


# cat /var/log/glusterfs/cmd_history.log (NODE02) 
[2016-02-23 16:10:04.874219]  : volume set help : SUCCESS
[2016-02-23 16:10:17.948524]  : volume status : SUCCESS
[2016-02-23 16:10:17.950331]  : volume status : SUCCESS
[2016-02-23 16:15:53.717774]  : volume set help : SUCCESS
[2016-02-23 16:18:17.108678]  : volume set help : SUCCESS
[2016-02-24 10:34:56.804196]  : volume set help : SUCCESS
[2016-02-24 14:29:03.936580]  : volume set help : SUCCESS
[2016-02-24 14:53:45.494411]  : volume set help : SUCCESS
[2016-02-25 08:30:42.306042]  : volume set help : SUCCESS

Hi Everyone,

I wanted to further detail my findings since it does not appear it has been mentioned yet.

This issue where the IP does not seem to be working for a glusterfs client mount is more related to reverse name resolution.  I tested this on an updated CentOS7 system pulling directly from the gluster 3.8 repo.  The moment I added the servers in the /etc/hosts file, I could mount by hostanme or IP with two client systems.  My logic tells me this is why the servers do not seem to be affected: They already have an updated /etc/hosts file.

I get a similar problem. We only use IP addresses.

[2016-12-13 17:01:15.908466] I [MSGID: 100030] [glusterfsd.c:2454:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.8.6 (args: /usr/sbin/glusterfs --volfile-server=10.90.5.105 --volfile-id=/marketplace_nfs /data)
[2016-12-13 17:01:15.912204] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-12-13 17:01:15.918784] E [MSGID: 127002] [quick-read.c:835:check_cache_size_ok] 0-marketplace_nfs-quick-read: Cache size 1073741824 is greater than the max size of 1040695296
[2016-12-13 17:01:15.918799] E [MSGID: 101019] [xlator.c:433:xlator_init] 0-marketplace_nfs-quick-read: Initialization of volume 'marketplace_nfs-quick-read' failed, review your volfile again
[2016-12-13 17:01:15.918805] E [MSGID: 101066] [graph.c:324:glusterfs_graph_init] 0-marketplace_nfs-quick-read: initializing translator failed
[2016-12-13 17:01:15.918813] E [MSGID: 101176] [graph.c:673:glusterfs_graph_activate] 0-graph: init failed
[2016-12-13 17:01:15.919085] W [glusterfsd.c:1327:cleanup_and_exit] (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x3a1) [0x7f11b21aeae1] -->/usr/sbin/glusterfs(glusterfs_process_volfp+0x14a) [0x7f11b21a94da] -->/usr/sbin/glusterfs(cleanup_and_exit+0x6b) [0x7f11b21a8acb] ) 0-: received signum (1), shutting down
[2016-12-13 17:01:15.919106] I [fuse-bridge.c:5793:fini] 0-fuse: Unmounting '/data'.

# glusterfs --version
glusterfs 3.8.6

Actually, my problem above is most likely caused my the instance memory.

This bug is getting closed because GlusteFS-3.7 has reached its end-of-life.

Note: This bug is being closed using a script. No verification has been performed to check if it still exists on newer releases of GlusterFS.
If this bug still exists in newer GlusterFS releases, please reopen this bug against the newer release.