Bug 1038973

Summary:	[SNAPSHOT] : Snapshot create command failed
Product:	Red Hat Gluster Storage	Reporter:	senaik
Component:	snapshot	Assignee:	Avra Sengupta <asengupt>
Status:	CLOSED ERRATA	QA Contact:	senaik
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	rhgs-3.0	CC:	asengupt, nsathyan, rhs-bugs, sdharane, ssamanta, storage-qa-internal
Target Milestone:	---
Target Release:	RHGS 3.0.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	SNAPSHOT
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-09-22 19:30:00 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description senaik 2013-12-06 09:33:32 UTC

Description of problem:
=======================
Snapshot create command fails with following error :

snapshot create: failed: Commit failed on localhost. Please check log file for details.
Snapshot command failed

Version-Release number of selected component (if applicable):
=============================================================
glusterfs 3.4.0.snap.dec03.2013git

How reproducible:


Steps to Reproduce:
===================

1.Create a distribute volume with 4 bricks and start it 

gluster v create vol6 10.70.43.32:/brick2/c1 10.70.43.151:/brick2/c2 10.70.43.71:/brick2/c3 10.70.43.32:/brick2/c4
volume create: vol6: success: please start the volume to access data
[root@snapshot-01 glusterfs]# gluster v start vol6
volume start: vol6: success

2.On the second node ,mount the volume and create 2 files 

3.On the third node , take a snapshot of the volume . Command failed with below error :
snapshot create: failed: Commit failed on localhost. Please check log file for details.
Snapshot command failed


[root@snapshot-03 glusterfs]# df
Filesystem                      1K-blocks    Used Available Use% Mounted on
/dev/vda3                         8255928 2569752   5266800  33% /
tmpfs                             4029816       0   4029816   0% /dev/shm
/dev/vda1                         1032088   62276    917384   7% /boot
/dev/mapper/VolGroup0-thin_vol0   8275968   33264   8242704   1% /brick0
/dev/mapper/VolGroup0-thin_vol1   8275968   33072   8242896   1% /brick1
/dev/mapper/VolGroup0-thin_vol2   8275968   33296   8242672   1% /brick2
/dev/mapper/VolGroup0-thin_vol3   8275968   33072   8242896   1% /brick3
/dev/mapper/VolGroup1-thin_vol4   8275968   33072   8242896   1% /brick4
/dev/mapper/VolGroup1-thin_vol5   8275968   33072   8242896   1% /brick5
/dev/mapper/VolGroup1-thin_vol6   8275968   33072   8242896   1% /brick6
/dev/mapper/VolGroup1-thin_vol7   8275968   33072   8242896   1% /brick7
10.70.43.136:/rpm_vol             8256512 2457600   5380096  32% /mnt/snap
/dev/mapper/VolGroup0-snap4       8275968   33264   8242704   1% /run/gluster/snaps/snap4/dev-mapper-VolGroup0-snap4-brick
/dev/mapper/VolGroup0-snap4.1     8275968   33264   8242704   1% /run/gluster/snaps/snap4.1/dev-mapper-VolGroup0-snap4.1-brick
/dev/mapper/VolGroup0-snap5       8275968   33264   8242704   1% /run/gluster/snaps/snap5/dev-mapper-VolGroup0-snap5-brick
/dev/mapper/VolGroup0-snap6       8275968   33296   8242672   1% /run/gluster/snaps/snap6/dev-mapper-VolGroup0-snap6-brick


Actual results:
===============
Snapshot create fails 


Expected results:
=================
Creating snapshot should not fail 


Additional info:
==============

--------------part of log------------------------------

[2013-12-06 02:12:11.866427] I [socket.c:3548:socket_init] 0-tcp.snap6-server: using system polling thread
[2013-12-06 02:12:11.870410] E [posix.c:4754:init] 0-snap6-posix: mismatching volume-id (d2a838ef-1024-49f2-bf63-9f36544454ea) received. already is a part of volume 5e25b270-4479-451c-a5ab-155c321b40c9 
[2013-12-06 02:12:11.870482] E [xlator.c:403:xlator_init] 0-snap6-posix: Initialization of volume 'snap6-posix' failed, review your volfile again
[2013-12-06 02:12:11.870506] E [graph.c:307:glusterfs_graph_init] 0-snap6-posix: initializing translator failed
[2013-12-06 02:12:11.870521] E [graph.c:502:glusterfs_graph_activate] 0-graph: init failed
[2013-12-06 02:12:11.871083] W [glusterfsd.c:1086:cleanup_and_exit] (-->/usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5) [0x31cd20e0d5] (-->/usr/sbin/glusterfsd(mgmt_getspec_cbk+0x30b) [0x40b6ab] (-->/usr/sbin/glusterfsd(glusterfs_process_volfp+0x106) [0x405096]))) 0-: received signum (0), shutting down
----------------------------------------------------------

Comment 1 senaik 2013-12-06 09:42:07 UTC

[root@snapshot-01 glusterfs]# gluster v i vol6
 
Volume Name: vol6
Type: Distribute
Volume ID: 5e25b270-4479-451c-a5ab-155c321b40c9
Status: Started
Snap Volume: no
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: 10.70.43.32:/brick2/c1
Brick2: 10.70.43.151:/brick2/c2
Brick3: 10.70.43.71:/brick2/c3
Brick4: 10.70.43.32:/brick2/c4


logs can be found at : 

http://rhsqe-repo.lab.eng.blr.redhat.com/bugs_necessary_info/snapshots/1038973/

Comment 3 Avra Sengupta 2013-12-26 08:25:00 UTC

Fixed volume ID mismatches with http://review.gluster.org/#/c/6345/

Comment 4 senaik 2014-01-07 13:16:16 UTC

Currently if any create /delete command fails (due to any reason )we get the error message that Commit has failed as mentioned in the description . It would be better if the user gets a meaningful message instead of 'Commit Failed'

For Eg: 

As of now if snapshot delete command fails we get the following message:

gluster snapshot delete -v vol1 -s SNAP2
snapshot remove: failed: Commit failed on 10.70.43.151. Please check log file for details.
Snapshot command failed

We could provide a message as: 

snapshot delete : failed : Please check the log file for details

Comment 5 senaik 2014-01-21 09:19:06 UTC

Version : 3.4.1.snap.jan15.2014git
=======
Repeated the steps as mentioned in 'Description' snapshot create command does not fail. Marking the bug as 'Verified'

Comment 7 Nagaprasad Sathyanarayana 2014-04-21 06:17:48 UTC

Marking snapshot BZs to RHS 3.0.

Comment 8 Nagaprasad Sathyanarayana 2014-05-19 10:56:36 UTC

Setting flags required to add BZs to RHS 3.0 Errata

Comment 11 errata-xmlrpc 2014-09-22 19:30:00 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1278.html