Bug 1281347
| Summary: | Though directories are deleted from mount, still able to see them on cold bricks | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | RajeshReddy <rmekala> |
| Component: | tier | Assignee: | Nithya Balachandran <nbalacha> |
| Status: | CLOSED WORKSFORME | QA Contact: | Nag Pavan Chilakam <nchilaka> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rhgs-3.1 | CC: | mzywusko, nbalacha, rhs-bugs, rmekala, storage-qa-internal |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-12-15 04:16:32 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
RajeshReddy
2015-11-12 11:02:26 UTC
After remounting, able to see those directories from the mount [root@dht-rhs-23 disrep_tier]# pwd /mnt/disrep_tier [root@dht-rhs-23 disrep_tier]# ls -lrth total 5.5K drwxr-xr-x. 2 root root 6 Nov 12 11:28 big drwxr-xr-x. 2 root root 58 Nov 12 11:29 bug drwxr-xr-x. 2 root root 6 Nov 12 12:22 test drwxr-xr-x. 2 root root 6 Nov 12 15:50 down drwxr-xr-x. 2 root root 6 Nov 16 15:45 data4 drwxr-xr-x. 2 root root 6 Nov 16 15:45 data5 drwxr-xr-x. 2 root root 6 Nov 16 15:45 data6 drwxr-xr-x. 2 root root 6 Nov 16 15:45 data7 drwxr-xr-x. 2 root root 6 Nov 16 15:45 data8 drwxr-xr-x. 2 root root 6 Nov 16 15:45 data9 drwxr-xr-x. 4 root root 57 Nov 16 15:50 linux sosreports are available @ /home/repo/sosreports/bug.1284928 on rhsqe-repo.lab.eng.blr.redhat.com Updated client sosreport available @ /home/repo/sosreports/bug.1284928 on rhsqe-repo.lab.eng.blr.redhat.com The client logs do not have any errors related to dir deletes. However, there are lots of error messages indicating that some bricks were not reachable. Where any bricks down during these tests? [2015-11-12 11:45:06.450045] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-disrep_tier-client-2: changing port to 49171 (from 0) [2015-11-12 11:45:06.455800] E [socket.c:2278:socket_connect_finish] 0-disrep_tier-client-2: connection to 10.70.36.42:49171 failed (Connection refused) [2015-11-12 11:45:09.460433] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-disrep_tier-client-3: changing port to 49170 (from 0) [2015-11-12 11:45:09.471182] E [socket.c:2278:socket_connect_finish] 0-disrep_tier-client-3: connection to 10.70.36.43:49170 failed (Connection refused) [2015-11-12 11:46:31.920475] I [MSGID: 109063] [dht-layout.c:702:dht_layout_normalize] 0-disrep_tier-cold-dht: Found anomalies in /big (gfid = 82d2ddc2-d36c-4f7d-b308-39534e558371). Holes=1 overlaps=0 [2015-11-12 11:46:32.659562] I [MSGID: 109063] [dht-layout.c:702:dht_layout_normalize] 0-disrep_tier-cold-dht: Found anomalies in / (gfid = 00000000-0000-0000-0000-000000000001). Holes=1 overlaps=0 [2015-11-12 11:46:32.659626] W [MSGID: 109005] [dht-selfheal.c:1795:dht_selfheal_directory] 0-disrep_tier-cold-dht: Directory selfheal failed: 1 subvolumes down.Not fixing. path = /, gfid = 00000000-0000-0000-0000-000000000001 The message "I [MSGID: 108006] [afr-common.c:4008:afr_local_init] 0-disrep_tier-replicate-1: no subvolumes up" repeated 19 times between [2015-11-12 11:46:31.906727] and [2015-11-12 11:46:32.690197] [2015-11-12 11:46:32.690997] I [MSGID: 109063] [dht-layout.c:702:dht_layout_normalize] 0-disrep_tier-cold-dht: Found anomalies in /big (gfid = 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0 [2015-11-12 11:46:32.691073] W [MSGID: 109005] [dht-selfheal.c:1795:dht_selfheal_directory] 0-disrep_tier-cold-dht: Directory selfheal failed: 1 subvolumes down.Not fixing. path = /big, gfid = 82d2ddc2-d36c-4f7d-b308-39534e558371 [2015-11-12 12:14:14.410012] W [MSGID: 109075] [dht-diskusage.c:49:dht_du_info_cbk] 0-disrep_tier-cold-dht: failed to get disk info from disrep_tier-replicate-1 [Transport endpoint is not connected] The [2015-11-12 12:14:15.799370] W [MSGID: 109011] [dht-layout.c:191:dht_layout_search] 0-disrep_tier-cold-dht: no subvolume for hash (value) = 3553706931 [2015-11-12 12:14:15.799485] W [MSGID: 109011] [dht-layout.c:191:dht_layout_search] 0-disrep_tier-cold-dht: no subvolume for hash (value) = 3781782680 [2015-11-12 12:14:15.799518] W [MSGID: 109011] [dht-layout.c:191:dht_layout_search] 0-disrep_tier-cold-dht: no subvolume for hash (value) = 2593024302 [2015-11-12 12:14:15.799541] W [MSGID: 109011] [dht-layout.c:191:dht_layout_search] 0-disrep_tier-cold-dht: no subvolume for hash (value) = 4122822141 While performing delete operations no brick was down Tested with glusterfs-server-3.7.5-11, created 10 directories with each directory having 1k files and deleted parent directory and after deletion not able to see any sub directories on both cold and hot tier |