Bug 1477951 - [Gluster-block]: Block-delete reports a success, but the block still exists on the volume
[Gluster-block]: Block-delete reports a success, but the block still exists o...
Status: NEW
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: gluster-block (Show other bugs)
3.3
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Prasanna Kumar Kalever
Rahul Hinduja
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-08-03 05:58 EDT by Sweta Anandpara
Modified: 2017-08-07 12:10 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Sweta Anandpara 2017-08-03 05:58:19 EDT
Description of problem:
======================
During multiple rounds of block-creates and block-deletes, it was observed, that 'gluster-block delete <volname>/<blockname>' reported a success on CLI. However, 'gluster-block list <volname>' showed the presence/existence of the said block. 

There is no error mentioned in the log files as well. Please find the CLI logs below. 
It can so happen that I have same block names across different volumes, for which something is not working correctly in the delete path. Whatever that is, it is going unnoticed/uncaught.. hence the command reports a success.

I'll be trying this on unique block names and updating further. 

Version-Release number of selected component (if applicable):
============================================================
glusterfs-3.8.4-36, gluster-block-0.2.1-6, tcmu-runner-1.2.0-11

How reproducible:
================
Seen it twice


Additional info:
===============
// please note "vol4/bk1" in the logs below.

[root@dhcp47-84 ~]# gluster-block list vol4
bk1
[root@dhcp47-84 ~]# 
[root@dhcp47-84 ~]# gluster-block delete vol4/bk1            >> we delete vol4/bk1
SUCCESSFUL ON:   10.70.46.18
RESULT: SUCCESS
[root@dhcp47-84 ~]# gluster-block list vol5
bk2
[root@dhcp47-84 ~]# gluster-block delete vol5/bk2
SUCCESSFUL ON:   10.70.47.84 10.70.46.244 10.70.46.18
RESULT: SUCCESS
[root@dhcp47-84 ~]# gluster-block list vol5
*Nil*
[root@dhcp47-84 ~]# gluster-block list vol6
bk2
[root@dhcp47-84 ~]# gluster-block delete vol6/bk2
SUCCESSFUL ON:   10.70.46.18 10.70.47.84 10.70.46.244
RESULT: SUCCESS
[root@dhcp47-84 ~]# gluster-block list vol6
*Nil*
[root@dhcp47-84 ~]# gluster-block list vol4                  >>> vol4 lists 'bk1' even though we deleted it at [1]
bk1
[root@dhcp47-84 ~]# gluster-block info vol4/bk1
NAME: bk1
VOLUME: vol4
GBID: e9bf8d58-323b-47e9-b64c-8567cca22029
SIZE: 1048576
HA: 3
PASSWORD: 
BLOCK CONFIG NODE(S): 10.70.46.18
[root@dhcp47-84 ~]# vim /var/log/gluster-block/gluster-blockd.log 
[root@dhcp47-84 ~]# mkdir /mnt/vol4
[root@dhcp47-84 ~]# mount -t glusterfs 10.70.47.84:vol4 /mnt/vol4
[root@dhcp47-84 ~]# cat /mnt/vol4/block-meta/bk1 
VOLUME: vol4
GBID: e9bf8d58-323b-47e9-b64c-8567cca22029
SIZE: 1048576
HA: 3
ENTRYCREATE: INPROGRESS
ENTRYCREATE: SUCCESS
10.70.47.84: CONFIGINPROGRESS
10.70.46.244: CONFIGINPROGRESS
10.70.46.18: CONFIGINPROGRESS
10.70.47.84: CONFIGFAIL
10.70.46.244: CONFIGFAIL
10.70.46.18: CONFIGFAIL
10.70.46.244: CLEANUPINPROGRESS
10.70.46.18: CLEANUPINPROGRESS
10.70.47.84: CLEANUPINPROGRESS
10.70.47.84: CLEANUPINPROGRESS
10.70.46.244: CLEANUPINPROGRESS
10.70.46.18: CLEANUPINPROGRESS
10.70.46.244: CLEANUPFAIL
10.70.47.84: CLEANUPFAIL
10.70.46.18: CLEANUPFAIL
10.70.47.84: CLEANUPINPROGRESS
10.70.46.244: CLEANUPINPROGRESS
10.70.46.18: CLEANUPINPROGRESS
10.70.46.244: CLEANUPSUCCESS
10.70.47.84: CLEANUPSUCCESS
10.70.46.18: CLEANUPSUCCESS
10.70.46.18: CLEANUPINPROGRE[root@dhcp47-84 ~]# 
[root@dhcp47-84 ~]# 
[root@dhcp47-84 ~]#
Comment 2 Sweta Anandpara 2017-08-03 06:00:27 EDT
Please note that we'll have to RCA this further IF I hit this again on unique block names. Not proposing it as a blocker for rhgs3.3.0 for now.
Comment 3 Sweta Anandpara 2017-08-03 06:03:43 EDT
Gluster-block logs present in this location: http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/<bugnumber>/

Note You need to log in before you can comment on or make changes to this bug.