Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
First Last Prev Next    This bug is not in your last search results.
Bug 1111479 - [SNAPSHOT]: Attaching a new node to the cluster while snapshot delete was in progress, deleted snapshots successfuly but gluster snapshot list shows some of the the snaps is still present
[SNAPSHOT]: Attaching a new node to the cluster while snapshot delete was in ...
Status: CLOSED DEFERRED
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: snapshot (Show other bugs)
3.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: rjoseph
SNAPSHOT
:
Depends On:
Blocks: 1087818 1115806
  Show dependency tree / graph
 
Reported: 2014-06-20 02:58 EDT by senaik
Modified: 2016-09-17 09:02 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Known Issue
Doc Text:
Attaching and detaching new node to the cluster while snapshot is in progress leads to inconsistent snapshot. Workaround: Do not attach or detach new node to the cluster operation while snapshot is in progress.
Story Points: ---
Clone Of:
: 1115806 (view as bug list)
Environment:
Last Closed: 2016-01-29 08:42:01 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

  None (edit)
Description senaik 2014-06-20 02:58:31 EDT 
Description of problem:
=======================
Attaching a new node to the cluster while snapshot delete is in progress, deleted snapshots successfuly but gluster snapshot list shows some of the successfully deleted snapshosts are still present

Version-Release number of selected component (if applicable):
=============================================================
glusterfs 3.6.0.19 built on Jun 18 2014


How reproducible:
=================
1/1


Steps to Reproduce:
====================

-4 node cluster (snapshot13-16)
-created few snapshots with IO going on
-deleted snapshots in loop
-while delete in is in progress, probed another node (10.70.44.54)
for i in {1..20} ; do time gluster --mode=script snapshot delete snap_1_$i ; done
snapshot delete: snap_1_1: snap removed successfully

real    0m2.057s
user    0m0.206s
sys    0m0.045s
snapshot delete: snap_1_2: snap removed successfully

real    0m1.719s
user    0m0.197s
sys    0m0.057s
snapshot delete: snap_1_3: snap removed successfully

real    0m1.807s
user    0m0.217s
sys    0m0.045s
snapshot delete: snap_1_4: snap removed successfully----------------> removed successfully

real    0m1.739s
user    0m0.220s
sys    0m0.033s
snapshot delete: snap_1_5: snap removed successfully----------------> removed successfully

real    0m1.839s
user    0m0.203s
sys    0m0.049s
snapshot delete: failed: snap snap_1_6 might not be in an usable state.--> probe was in progress
Snapshot command failed

real    0m0.522s
user    0m0.203s
sys    0m0.047s
snapshot delete: snap_1_7: snap removed successfully

real    0m4.233s
user    0m0.218s
sys    0m0.038s
snapshot delete: snap_1_8: snap removed successfully

real    0m2.918s
user    0m0.214s
sys    0m0.040s
.
.
.
snapshot delete: snap_1_20: snap removed successfully

real    0m2.043s
user    0m0.211s
sys    0m0.040s


gluster snapshot list still shows snap_1_4 and snap_1_5

[root@snapshot16 ~]# gluster snapshot list
snap_1_4
snap_1_5
snap_1_6
snap_1_21
snap_1_22
snap_1_23

gluster snapshot status snap_1_5
Snapshot Status : failed: Commit failed on localhost. Please check log file for details.
Snapshot command failed

 gluster snapshot status snap_1_4
Snapshot Status : failed: Commit failed on localhost. Please check log file for details.
Snapshot command failed

gluster snapshot info snap_1_5
Snapshot                  : snap_1_5
Snap UUID                 : 39d5dfbe-a54e-4637-8151-fbc98118ce4d
Created                   : 2014-06-19 14:51:42
Snap Volumes:

    Snap Volume Name          : 02a8fa61e9674aa1babac102011bb544
    Origin Volume name        : vol0
    Snaps taken for vol0      : 6
    Snaps available for vol0  : 250
    Status                    : Started


 gluster snapshot info snap_1_4
Snapshot                  : snap_1_4
Snap UUID                 : 2df58e99-9547-46bf-896f-b07316a1c27b
Created                   : 2014-06-19 14:51:26
Snap Volumes:

    Snap Volume Name          : 62b973d1b0ad408dae74468d923a2112
    Origin Volume name        : vol0
    Snaps taken for vol0      : 6
    Snaps available for vol0  : 250
    Status                    : Started


df -h does not list it , but gluster snapshot list and info shows the snapshot is still present.


Actual results:
==============
snap_1_4 and snap_1_5 were deleted successfully, but gluster snapshot list shows the snapshot is still present


Expected results:
================
gluster snapshot list should not list the deleted snapshots.


Additional info:
Comment 3 Joseph Elwin Fernandes 2014-07-03 03:28:17 EDT 
The issue is same as bug 1114403. The issue is glusterd quorum failure in snap phases,

Glusterd Log from snapshot-15 host during deletion of snap_1_6 : 

[2014-06-19 09:27:45.937337] W [glusterd-utils.c:13069:glusterd_snap_quorum_check] 0-management: glusterds are not in quorum
[2014-06-19 09:27:45.937396] W [glusterd-mgmt.c:1846:glusterd_mgmt_v3_initiate_snap_phases] 0-management: quorum check failed


The fix is submitted upstream, in the review process 

Fix submitted upstream:

REVIEW: http://review.gluster.org/8200 (glusterd/snapshot: fixing glusterd quorum during snap operation) posted (#1) for review on master by Joseph Fernandes (josferna@redhat.com)

I have tested the fix with this scenario and it fixed this issue.
Comment 4 Shalaka 2014-09-20 23:50:40 EDT 
Please review and sign-off edited doc text.
Comment 5 Avra Sengupta 2015-03-30 05:55:13 EDT 
Not targetting for 3.1
Comment 7 Vivek Agarwal 2015-04-21 07:51:07 EDT 
based on commen 5, removing from the list for 3.1
Comment 8 Avra Sengupta 2015-08-12 01:43:49 EDT 
This Bug is not fixed with the submitted patch and it requires design changes in glusterd. Hence moving this back to New.
Comment 10 Avra Sengupta 2016-01-29 08:42:01 EST 
Current Gluster architecture does not support implementation of this feature. Therefore this feature request is deferred till Gluterd 2.0.
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.