Bug 1044476

Summary:	[SNAPSHOT]: gluster snapshot list does not show any snaps taken on the volume after restarting glusterd
Product:	Red Hat Gluster Storage	Reporter:	senaik
Component:	snapshot	Assignee:	Joseph Elwin Fernandes <josferna>
Status:	CLOSED ERRATA	QA Contact:	senaik
Severity:	high	Docs Contact:
Priority:	high
Version:	rhgs-3.0	CC:	josferna, nsathyan, rhs-bugs, rjoseph, sankarshan, ssamanta, storage-qa-internal, vmallika
Target Milestone:	---
Target Release:	RHGS 3.0.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	SNAPSHOT
Fixed In Version:	3.4.1.3.snap.mar13.2014	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-09-22 19:30:43 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	1039956, 1054151

Description senaik 2013-12-18 11:56:43 UTC

Description of problem:
=======================
After restarting glusterd, gluster snapshot list does not show any snaps that are taken on the volume 

Version-Release number of selected component (if applicable):
============================================================
glusterfs 3.4.0.snap.dec03.2013git


How reproducible:


Steps to Reproduce:
===================
1.Create a volume and take multiple snapshots on the volume 

2.List all the snapshots on the volume 
gluster snapshot list volume1

Volume Name : volume1
Number of snaps taken : 5
Number of snaps available : 251

	Snap Name : vol1_snap
	Snap Time : 2013-12-17 11:13:15
	Snap ID : 618fcc8c-c017-4836-bfad-58d806fd0b9d

	Snap Name : vol1_snap_1
	Snap Time : 2013-12-17 11:14:08
	Snap ID : d71559be-d4a1-455b-adc0-a76c3e2bda2f

	Snap Name : Cons_Grp1_volume1_snap
	Snap Time : 2013-12-17 11:32:37
	Snap ID : fe63dd9c-232e-4606-bdf5-a7b6b684b849

	Snap Name : CG2
	Snap Time : 2013-12-17 11:33:25
	Snap ID : a727b820-f714-41ed-b435-325058347853

	Snap Name : CG3_volume1_snap
	Snap Time : 2013-12-18 04:51:57
	Snap ID : 1e13fb84-ee3c-4f7c-8ad7-d2f47cab44f2


3.Restart glusterd 

4.List snapshots again 
gluster snapshot list volume1

Volume Name : volume1
Number of snaps taken : 0
Number of snaps available : 256

5. List snapshots from another peer 

gluster snapshot list volume1

Volume Name : volume1
Number of snaps taken : 5
Number of snaps available : 251

	Snap Name : vol1_snap
	Snap Time : 2013-12-17 11:13:13
	Snap ID : f39820ee-cd70-4fbf-a6db-a90c5295eda9

	Snap Name : vol1_snap_1
	Snap Time : 2013-12-17 11:14:07
	Snap ID : 7c0d410e-d863-4835-a614-a5a50e550de8

	Snap Name : Cons_Grp1_volume1_snap
	Snap Time : 2013-12-17 11:32:34
	Snap ID : 3db28014-db5f-4456-ad38-b64f83672aea

	Snap Name : CG2
	Snap Time : 2013-12-17 11:33:23
	Snap ID : d5b2a740-9237-45b4-bb9d-61d4699586bb

	Snap Name : CG3_volume1_snap
	Snap Time : 2013-12-18 04:51:52
	Snap ID : 7286fd02-e2c3-44b8-b28e-6437792d2f7a



Actual results:
==============
After restarting glusterd, gluster snapshot list does not show any snaps that are taken on the volume 


Expected results:
================
Gluster snapshot list should show all the snaps taken on the volume at any time irrespective of restarting glusterd .

Additional info:

Comment 2 Sachin Pandit 2014-01-16 06:12:53 UTC

This was happening if the snap description contained multiple words.
Till now we were handling the (key,value) containing only single value ( no space in between), But in Description the user can specify multiple words and while reading the value during glusterd restart there was mismatch in the key value pair. This has been fixed in patch http://review.gluster.org/#/c/6543/.

Comment 3 senaik 2014-01-20 09:48:52 UTC

Version : glusterfs 3.4.1.snap.jan15.2014git

Repeated the steps as mentioned in Steps to reproduce, all the snapshots taken on the volume are listed after restarting glusterd . Marking the bug as 'Verified'.

Comment 4 senaik 2014-02-07 08:56:00 UTC

Version : glusterfs 3.4.1.0.snap.feb05.2014git
=======

Able to recreate the issue on the latest build . 

1)Create a dist rep volume (vol0)
2)Create 10 snapshots on the volume 
for i in {1..10} ; do gluster snapshot create vol0 -n Snap"$i"; done
snapshot create: Snap1: snap created successfully
snapshot create: Snap2: snap created successfully
snapshot create: Snap3: snap created successfully
snapshot create: Snap4: snap created successfully
snapshot create: Snap5: snap created successfully
snapshot create: Snap6: snap created successfully
snapshot create: Snap7: snap created successfully
snapshot create: Snap8: snap created successfully
snapshot create: Snap9: snap created successfully
snapshot create: Snap10: snap created successfully

3)List the snapshots from all nodes 
gluster snapshot list 

Volume Name                  : vol0
Number of snaps taken        : 10
Number of snaps available    : 246

	Snap Name         : Snap1
	Snap Time         : 2014-02-07 07:14:01
	Snap UUID         : a34a19b2-5b97-43b6-a87c-e15441a0bff9

	Snap Name         : Snap2
	Snap Time         : 2014-02-07 07:14:05
	Snap UUID         : 86fe222c-030d-4165-81a0-2a3a8ce16923

	Snap Name         : Snap3
	Snap Time         : 2014-02-07 07:14:09
	Snap UUID         : 780af025-e20e-4325-83d5-644519da8cc9

	Snap Name         : Snap4
	Snap Time         : 2014-02-07 07:14:13
	Snap UUID         : ba85712e-3c07-45d5-920b-2f653c323971

	Snap Name         : Snap5
	Snap Time         : 2014-02-07 07:14:17
	Snap UUID         : f908708a-287d-407f-8db0-6673ae7a1d79

	Snap Name         : Snap6
	Snap Time         : 2014-02-07 07:14:22
	Snap UUID         : 4e516653-92e9-4ada-8f8d-cfb84b783351

	Snap Name         : Snap7
	Snap Time         : 2014-02-07 07:14:26
	Snap UUID         : 8d6ebf82-591d-4cc7-a883-c77dd7bc639e

	Snap Name         : Snap8
	Snap Time         : 2014-02-07 07:14:30
	Snap UUID         : 99aabe00-d33d-4273-abbe-b1ccdb64e1ee

	Snap Name         : Snap9
	Snap Time         : 2014-02-07 07:14:34
	Snap UUID         : da6bc3eb-9b27-4a4b-bde1-6ecd9a386d7d

	Snap Name         : Snap10
	Snap Time         : 2014-02-07 07:14:38
	Snap UUID         : 0a3c85ef-e017-4077-b8c4-fb25f58a2025

Volume Name                  : vol1
Number of snaps taken        : 0
Number of snaps available    : 256

Volume Name                  : vol2
Number of snaps taken        : 0
Number of snaps available    : 256

 gluster snapshot list vol0

Volume Name                  : vol0
Number of snaps taken        : 10
Number of snaps available    : 246

	Snap Name         : Snap1
	Snap Time         : 2014-02-07 07:14:01
	Snap UUID         : a34a19b2-5b97-43b6-a87c-e15441a0bff9

	Snap Name         : Snap2
	Snap Time         : 2014-02-07 07:14:05
	Snap UUID         : 86fe222c-030d-4165-81a0-2a3a8ce16923

	Snap Name         : Snap3
	Snap Time         : 2014-02-07 07:14:09
	Snap UUID         : 780af025-e20e-4325-83d5-644519da8cc9

	Snap Name         : Snap4
	Snap Time         : 2014-02-07 07:14:13
	Snap UUID         : ba85712e-3c07-45d5-920b-2f653c323971

	Snap Name         : Snap5
	Snap Time         : 2014-02-07 07:14:17
	Snap UUID         : f908708a-287d-407f-8db0-6673ae7a1d79

	Snap Name         : Snap6
	Snap Time         : 2014-02-07 07:14:22
	Snap UUID         : 4e516653-92e9-4ada-8f8d-cfb84b783351

	Snap Name         : Snap7
	Snap Time         : 2014-02-07 07:14:26
	Snap UUID         : 8d6ebf82-591d-4cc7-a883-c77dd7bc639e

	Snap Name         : Snap8
	Snap Time         : 2014-02-07 07:14:30
	Snap UUID         : 99aabe00-d33d-4273-abbe-b1ccdb64e1ee

	Snap Name         : Snap9
	Snap Time         : 2014-02-07 07:14:34
	Snap UUID         : da6bc3eb-9b27-4a4b-bde1-6ecd9a386d7d

	Snap Name         : Snap10
	Snap Time         : 2014-02-07 07:14:38
	Snap UUID         : 0a3c85ef-e017-4077-b8c4-fb25f58a2025

4) Restart glusterd 

5) List Snapshot again 

 gluster snapshot list vol0

Volume Name                  : vol0
Number of snaps taken        : 0
Number of snaps available    : 256
[root@snapshot-01 ~]# gluster snapshot list vol0

Volume Name                  : vol0
Number of snaps taken        : 0
Number of snaps available    : 256

The snaps taken on the volume are not listed

Comment 5 Joseph Elwin Fernandes 2014-02-18 08:57:26 UTC

Assigning the bug to myself

Comment 7 Vijaikumar Mallikarjuna 2014-03-24 07:17:13 UTC

Patch http://review.gluster.org/#/c/7006/ fixes this issue.
Moving the bug to ON_QA

Comment 8 senaik 2014-03-26 12:15:51 UTC

Version : glusterfs 3.4.1.6.snap.mar25.2014
=======

With the new snap driven changes, snapshot list will only list snapshots & snapshot info will give details about the snapshot 


1) After restarting glusterd 'gluster snapshot list' shows all the snapshots taken on the volume 

[root@snapshot-01 ~]# gluster snapshot list
snap_vol1_1
snap_vol1_2
snap_vol1_3
snap_vol1_4
snap_vol1_5
snap_vol1_6
snap_vol1_7
snap_vol1_8
snap_vol1_9
snap_vol1_10

Restart glusterd 

List snapshots again 

[root@snapshot-01 ~]# gluster snapshot list
snap_vol1_1
snap_vol1_2
snap_vol1_3
snap_vol1_4
snap_vol1_5
snap_vol1_6
snap_vol1_7
snap_vol1_8
snap_vol1_9
snap_vol1_10
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2) Snapshot info shows all the the information related to the snapshots:

gluster snapshot info volume vol1
Volume Name               : vol1
Snaps Taken               : 10
Snaps Available           : 246
	Snapshot                  : snap_vol1_1
	Snap UUID                 : d45ccab8-8b06-4472-87e6-f329fa2322de
	Created                   : 2014-03-26 09:07:23
	Status                    : Started
 
	Snapshot                  : snap_vol1_2
	Snap UUID                 : 14636ecb-238e-49bb-974a-4ddc817b0a89
	Created                   : 2014-03-26 09:07:29
	Status                    : Started
 
	Snapshot                  : snap_vol1_3
	Snap UUID                 : 9b1a7b09-f885-49b9-a3f8-39bd51a3694e
	Created                   : 2014-03-26 09:07:33
	Status                    : Started


 gluster snapshot info snap_vol1_2
Snapshot                  : snap_vol1_2
Snap UUID                 : 14636ecb-238e-49bb-974a-4ddc817b0a89
Created                   : 2014-03-26 09:07:29
Snap Volumes:

	Snap Volume Name          : f0c649d867da4f58840c5136da1c5d55
	Origin Volume name        : vol1
	Snaps taken for vol1      : 10
	Snaps available for vol1  : 246
	Status                    : Started

Restart glusterd 

Use snapshot info command again and check the number of snaps taken and number of snaps available.

gluster snapshot info volume vol1
Volume Name               : vol1
Snaps Taken               : 10
Snaps Available           : 246
	Snapshot                  : snap_vol1_1
	Snap UUID                 : d45ccab8-8b06-4472-87e6-f329fa2322de
	Created                   : 2014-03-26 09:07:23
	Status                    : Started
 
	Snapshot                  : snap_vol1_2
	Snap UUID                 : 14636ecb-238e-49bb-974a-4ddc817b0a89
	Created                   : 2014-03-26 09:07:29
	Status                    : Started
 
	Snapshot                  : snap_vol1_3
	Snap UUID                 : 9b1a7b09-f885-49b9-a3f8-39bd51a3694e
	Created                   : 2014-03-26 09:07:33
	Status                    : Started


gluster snapshot info snap_vol1_2
Snapshot                  : snap_vol1_2
Snap UUID                 : 14636ecb-238e-49bb-974a-4ddc817b0a89
Created                   : 2014-03-26 09:07:29
Snap Volumes:

	Snap Volume Name          : f0c649d867da4f58840c5136da1c5d55
	Origin Volume name        : vol1
	Snaps taken for vol1      : 10
	Snaps available for vol1  : 246
	Status                    : Started



Number of snaps taken and number of snaps available remains the same after restarting glusterd. Marking the bug as 'Verified'

Comment 9 Nagaprasad Sathyanarayana 2014-04-21 06:18:04 UTC

Marking snapshot BZs to RHS 3.0.

Comment 10 Nagaprasad Sathyanarayana 2014-05-19 10:56:40 UTC

Setting flags required to add BZs to RHS 3.0 Errata

Comment 13 errata-xmlrpc 2014-09-22 19:30:43 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1278.html