Bug 1676356
Summary: | glusterfs FUSE client crashing every few days with 'Failed to dispatch handler' | ||
---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Raghavendra G <rgowdapp> |
Component: | write-behind | Assignee: | bugs <bugs> |
Status: | CLOSED NEXTRELEASE | QA Contact: | |
Severity: | urgent | Docs Contact: | |
Priority: | unspecified | ||
Version: | 6 | CC: | bugs, guillaume.pavese, peljasz, vpvainio |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | 1674406 | Environment: | |
Last Closed: | 2019-03-12 05:17:56 UTC | Type: | --- |
Regression: | --- | Mount Type: | fuse |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1671556, 1674406, 1678570, 1691292 | ||
Bug Blocks: | 1667103, 1672818, 1732875 |
Description
Raghavendra G
2019-02-12 03:25:12 UTC
I see these in 6.5: ... [2019-10-10 15:41:55.251100] E [MSGID: 101046] [dht-common.c:11220:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 15:41:55.251179] E [MSGID: 101046] [dht-common.c:11221:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null The message "E [MSGID: 101046] [dht-common.c:11220:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null" repeated 2 times between [2019-10-10 15:41:55.251100] and [2019-10-10 15:41:55.261152] The message "E [MSGID: 101046] [dht-common.c:11221:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null" repeated 2 times between [2019-10-10 15:41:55.251179] and [2019-10-10 15:41:55.261155] u[2019-10-10 15:52:07.263547] E [MSGID: 101046] [dht-common.c:11220:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 15:52:07.263620] E [MSGID: 101046] [dht-common.c:11221:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 15:52:08.526779] E [MSGID: 108006] [afr-common.c:5318:__afr_handle_child_down_event] 0-USER-HOME-replicate-0: All subvolumes are down. Going offline until at least one of them comes back up. [2019-10-10 15:52:08.528208] I [io-stats.c:4027:fini] 0-USER-HOME: io-stats translator unloaded [2019-10-10 15:52:10.441283] E [MSGID: 101046] [dht-common.c:11247:dht_pt_fgetxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 15:52:10.441387] E [MSGID: 101046] [dht-common.c:11248:dht_pt_fgetxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 15:52:10.555957] E [MSGID: 108006] [afr-common.c:5318:__afr_handle_child_down_event] 0-USER-HOME-replicate-0: All subvolumes are down. Going offline until at least one of them comes back up. [2019-10-10 15:52:10.557136] I [io-stats.c:4027:fini] 0-USER-HOME: io-stats translator unloaded ... also upgrade from 4.something. And interestingly, if quotas are in use on paths in the volume then windows shares (Samba) deny to copy new files saying that 0 bytes is free. (in case it might have something to do with above log errors) more: ... [2019-10-10 16:03:12.287199] E [MSGID: 101046] [dht-common.c:11220:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 16:03:12.287226] E [MSGID: 101046] [dht-common.c:11221:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 16:03:12.288849] W [MSGID: 114031] [client-rpc-fops_v2.c:921:client4_0_getxattr_cbk] 0-USER-HOME-client-13: remote operation failed. Path: /jt455/1st_READhowBackupsWork.txt (f9a6bfac-6942-4906-b145-4a351b873e39). Key: (null) [Permission denied] [2019-10-10 16:03:12.290847] W [MSGID: 114031] [client-rpc-fops_v2.c:921:client4_0_getxattr_cbk] 0-USER-HOME-client-14: remote operation failed. Path: /jt455/1st_READhowBackupsWork.txt (f9a6bfac-6942-4906-b145-4a351b873e39). Key: (null) [Permission denied] [2019-10-10 16:03:12.291491] W [MSGID: 114031] [client-rpc-fops_v2.c:921:client4_0_getxattr_cbk] 0-USER-HOME-client-15: remote operation failed. Path: /jt455/1st_READhowBackupsWork.txt (f9a6bfac-6942-4906-b145-4a351b873e39). Key: (null) [Permission denied] [2019-10-10 16:03:12.291623] W [dict.c:618:dict_del] (-->/usr/lib64/glusterfs/6.5/xlator/cluster/replicate.so(+0x16690) [0x7f9eded79690] -->/usr/lib64/glusterfs/6.5/xlator/cluster/distribute.so(+0x37d12) [0x7f9ede8cad12] -->/lib64/libglusterfs.so.0(dict_del+0x86) [0x7f9ee9e6e746] ) 0-dict: !this || key=trusted.glusterfs.dht [Invalid argument] [2019-10-10 16:03:12.291679] E [MSGID: 101046] [dht-common.c:11220:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null [2019-10-10 16:03:12.291708] E [MSGID: 101046] [dht-common.c:11221:dht_pt_getxattr_cbk] 0-USER-HOME-dht: dict is null |