Bug 763122 (GLUSTER-1390)

Created attachment 288

Created attachment 289

Created attachment 290

Created attachment 291

[2010-08-18 00:13:15] E [rpcsvc.c:478:rpcsvc_conn_peer_check_addr] rpc-service: Failed to get remote addr: Non-re
coverable failure in name resolution
[2010-08-18 00:13:15] E [rpc-socket.c:270:rpcsvc_socket_peername] rpc-socket: Failed to get peer name: Bad file d
escriptor
[2010-08-18 00:13:15] E [rpcsvc.c:1591:rpcsvc_submit_generic] rpc-service: Failed to submit message
[2010-08-18 00:13:15] E [mount3.c:85:mnt3svc_submit_reply] nfs-mount: Reply submission failed
[2010-08-18 00:13:29] E [rpcsvc.c:1230:rpcsvc_program_actor] rpc-service: RPC program not available
[2010-08-18 00:14:18] E [rpcsvc.c:449:rpcsvc_conn_peer_check_name] rpc-service: Failed to get remote addr: Temporary failure in name resolution
[2010-08-18 00:14:38] E [rpcsvc.c:449:rpcsvc_conn_peer_check_name] rpc-service: Failed to get remote addr: Temporary failure in name resolution
[2010-08-18 00:14:38] E [rpc-socket.c:300:rpcsvc_socket_peeraddr] rpc-socket: Failed to get peer addr: Transport endpoint is not connected
[2010-08-18 00:14:38] E [rpcsvc.c:478:rpcsvc_conn_peer_check_addr] rpc-service: Failed to get remote addr: Non-recoverable failure in name resolution
[2010-08-18 00:14:38] E [rpc-socket.c:270:rpcsvc_socket_peername] rpc-socket: Failed to get peer name: Bad file descriptor
[2010-08-18 00:14:38] E [rpcsvc.c:1591:rpcsvc_submit_generic] rpc-service: Failed to submit message
[2010-08-18 00:14:38] E [mount3.c:85:mnt3svc_submit_reply] nfs-mount: Reply submission failed
[2010-08-18 00:15:40] E [rpcsvc.c:449:rpcsvc_conn_peer_check_name] rpc-service: Failed to get remote addr: Temporary failure in name resolution
[2010-08-18 00:16:00] E [rpcsvc.c:449:rpcsvc_conn_peer_check_name] rpc-service: Failed to get remote addr: Temporary failure in name resolution
[2010-08-18 00:16:00] E [rpc-socket.c:300:rpcsvc_socket_peeraddr] rpc-socket: Failed to get peer addr: Transport endpoint is not connected
[2010-08-18 00:16:00] E [rpcsvc.c:478:rpcsvc_conn_peer_check_addr] rpc-service: Failed to get remote addr: Non-recoverable failure in name resolution
[2010-08-18 00:16:00] E [rpc-socket.c:270:rpcsvc_socket_peername] rpc-socket: Failed to get peer name: Bad file descriptor
[2010-08-18 00:16:00] E [rpcsvc.c:1591:rpcsvc_submit_generic] rpc-service: Failed to submit message
[2010-08-18 00:16:00] E [mount3.c:85:mnt3svc_submit_reply] nfs-mount: Reply submission failed
[2010-08-18 00:17:54] E [rpcsvc.c:1230:rpcsvc_program_actor] rpc-service: RPC program not available
[2010-08-18 00:18:46] E [rpcsvc.c:449:rpcsvc_conn_peer_check_name] rpc-service: Failed to get remote addr: Temporary failure in name resolution
[2010-08-18 00:19:06] E [rpcsvc.c:449:rpcsvc_conn_peer_check_name] rpc-service: Failed to get remote addr: Temporary failure in name resolution
[2010-08-18 00:19:06] E [rpc-socket.c:300:rpcsvc_socket_peeraddr] rpc-socket: Failed to get peer addr: Transport endpoint is not connected
:

(In reply to comment #1)
> Use the namelookup off option for turning off name resolution. See RC10
> relnotes FAQ section for more info.

Since we are not using hostnames in the volfiles how does this option fit in there?.

I am using rc9 since rc10 is not an option with the performance regression we saw with replicate and currently this setup is distributed+replicated. There is already another bug id which is filed against it.

(In reply to comment #6)
> (In reply to comment #1)
> > Use the namelookup off option for turning off name resolution. See RC10
> > relnotes FAQ section for more info.
> 
> Since we are not using hostnames in the volfiles how does this option fit in
> there?.
It doesnt matter, the getaddrinfo function ends up doing a lookup anyway even if the IP address is specified, I guess that is the bug. I need to detect whether we have addresses or hostnames specified in the rpc-auth.

Try with namelookup off, that will confirm our thinking.

> 
> I am using rc9 since rc10 is not an option with the performance regression we
> saw with replicate and currently this setup is distributed+replicated. There is
> already another bug id which is filed against it.

Yes. 1174.

Harsha, did you try with namelookup off?

raise to P1 major

(In reply to comment #9)
> raise to P1 major

Was this seen again? Can you please try with namelookup off option? Will help me confirm whether the problem is what I think it is. Thanks.

I've been looking at the code to fix this but it will need quite a bit of change just to detect whether the filters contain IPs or hostnames. It sounds simple but the way the code is organised, it is not. I am of the opinion that the simpler fix is to just use namelookup off in setups where DNS is unreachable.

Please re-open if using namelookup is a pain point. Thanks.