Bug 1094714 - [SNAPSHOT]: snapshot creation fails but df -h shows the snapshot brick as mounted
Summary: [SNAPSHOT]: snapshot creation fails but df -h shows the snapshot brick as mou...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: snapshot
Version: rhgs-3.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: RHGS 3.0.0
Assignee: Vijaikumar Mallikarjuna
QA Contact: Rahul Hinduja
URL:
Whiteboard: SNAPSHOT
Depends On:
Blocks: 1094720
TreeView+ depends on / blocked
 
Reported: 2014-05-06 10:51 UTC by Rahul Hinduja
Modified: 2016-09-17 13:00 UTC (History)
6 users (show)

Fixed In Version: glusterfs-3.6.0-3.0.el6rhs
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1094720 (view as bug list)
Environment:
Last Closed: 2014-09-22 19:36:37 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2014:1278 0 normal SHIPPED_LIVE Red Hat Storage Server 3.0 bug fix and enhancement update 2014-09-22 23:26:55 UTC

Description Rahul Hinduja 2014-05-06 10:51:48 UTC
Description of problem:
=======================

In one of my scenario the snapshot creation failed due to commit failed on local host, no lv is created for the snapshot brick but df -h shows it as mounted. 

##Below shows that the snapshot(snap3) creation for vol0 failed
[root@snapshot09 ~]# time gluster snapshot create snap3 vol0
snapshot create: failed: Commit failed on localhost. Please check log file for details.
Snapshot command failed

real	0m3.191s
user	0m0.169s
sys	0m0.027s
[root@snapshot09 ~]#

## Checking any lv created for snap3 using lvs

[root@snapshot09 ~]# lvs
  LV         VG            Attr       LSize   Pool       Origin Data%  Move Log Cpy%Sync Convert
  thin_pool0 VolGroup0     twi-a-tz-- 180.00g                     0.06                          
  thin_pool1 VolGroup0     twi-a-tz-- 180.00g                     0.06                          
  thin_vol0  VolGroup0     Vwi-aotz-- 100.00g thin_pool0          0.05                          
  thin_vol1  VolGroup0     Vwi-aotz-- 100.00g thin_pool0          0.05                          
  thin_vol2  VolGroup0     Vwi-aotz-- 100.00g thin_pool1          0.05                          
  thin_vol3  VolGroup0     Vwi-aotz-- 100.00g thin_pool1          0.05                          
  thin_pool2 VolGroup1     twi-a-tz-- 180.00g                     0.06                          
  thin_pool3 VolGroup1     twi-a-tz-- 180.00g                     0.06                          
  thin_vol4  VolGroup1     Vwi-aotz-- 100.00g thin_pool2          0.05                          
  thin_vol5  VolGroup1     Vwi-aotz-- 100.00g thin_pool2          0.05                          
  thin_vol6  VolGroup1     Vwi-aotz-- 100.00g thin_pool3          0.05                          
  thin_vol7  VolGroup1     Vwi-aotz-- 100.00g thin_pool3          0.05                          
  lv_root    vg_snapshot09 -wi-ao----  44.51g                                                   
  lv_swap    vg_snapshot09 -wi-ao----   5.00g                                                   
[root@snapshot09 ~]# 

## Checking df -h

[root@snapshot09 ~]# df -h 
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_snapshot09-lv_root
                       44G  1.9G   40G   5% /
tmpfs                 4.0G     0  4.0G   0% /dev/shm
/dev/vda1             485M   34M  426M   8% /boot
/dev/mapper/VolGroup0-thin_vol0
                      100G   33M  100G   1% /brick0
/dev/mapper/VolGroup0-thin_vol1
                      100G   33M  100G   1% /brick1
/dev/mapper/VolGroup0-thin_vol2
                      100G   33M  100G   1% /brick2
/dev/mapper/VolGroup0-thin_vol3
                      100G   33M  100G   1% /brick3
/dev/mapper/VolGroup1-thin_vol4
                      100G   33M  100G   1% /brick4
/dev/mapper/VolGroup1-thin_vol5
                      100G   33M  100G   1% /brick5
/dev/mapper/VolGroup1-thin_vol6
                      100G   33M  100G   1% /brick6
/dev/mapper/VolGroup1-thin_vol7
                      100G   33M  100G   1% /brick7
/dev/mapper/VolGroup0-a98f98898ade4433b2272b430cf7779f
                       44G  1.9G   40G   5% /var/run/gluster/snaps/a98f98898ade4433b2272b430cf7779f/brick1
/dev/mapper/VolGroup0-a98f98898ade4433b2272b430cf7779f
                       44G  1.9G   40G   5% /var/run/gluster/snaps/a98f98898ade4433b2272b430cf7779f/brick3
[root@snapshot09 ~]# 



Version-Release number of selected component (if applicable):
==============================================================

glusterfs-3.5qa2-0.425.git9360107.el6rhs.x86_64


How reproducible:
=================
1/1


Steps to Reproduce:
===================
1. Create a Volume from bricks consist of same volume group
2. Create a snapshot of the volume, the snapshot creation fails


Actual results:
===============

Snapshot creation fails but the snapshot bricks are mounted

Expected results:
=================
When snapshot creation failed and no LV is created the mounted bricks should be unmounted.


Additional info:
=================

Log snippet:
============

[2014-05-06 18:01:20.475025] D [glusterd-utils.c:11605:glusterd_mount_lvm_snapshot] 0-management: mounting the snapshot logical device /dev/VolGroup0/a98f98898ade4433b2272b430cf7779f successful
[2014-05-06 18:01:20.475133] W [glusterd-snapshot.c:3369:glusterd_snap_brick_create] 0-management: stat of the brick /var/run/gluster/snaps/a98f98898ade4433b2272b430cf7779f/brick3/b3(brick mount: /var/run/gluster/snaps/a98f98898ade4433b2272b430cf7779f/brick3) failed (No such file or directory)
[2014-05-06 18:01:20.475164] W [glusterd-snapshot.c:3388:glusterd_snap_brick_create] 0-management: unmounting the snap brick mount /var/run/gluster/snaps/a98f98898ade4433b2272b430cf7779f/brick3
[2014-05-06 18:01:20.475222] E [glusterd-snapshot.c:3557:glusterd_take_brick_snapshot] 0-management: not able to create the brickinfo for the snap snap3, volume vol0
[2014-05-06 18:01:20.475287] E [glusterd-snapshot.c:3699:glusterd_do_snap_vol] 0-management: Failed to take snapshot for snapshot09.lab.eng.blr.redhat.com:/brick3/b3
[2014-05-06 18:01:20.475323] D [glusterd-utils.c:7359:glusterd_brick_stop] 0-management: About to stop glusterfs for brick snapshot09.lab.eng.blr.redhat.com:/var/run/gluster/snaps/a98f98898ade4433b2272b430cf7779f/brick1/b2
[2014-05-06 18:01:20.475383] D [glusterd-utils.c:7369:glusterd_brick_stop] 0-management: returning 0 
[2014-05-06 18:01:20.475405] D [glusterd-utils.c:7359:glusterd_brick_stop] 0-management: About to stop glusterfs for brick snapshot09.lab.eng.blr.redhat.com:/var/run/gluster/snaps/a98f98898ade4433b2272b430cf7779f/brick3/b3

Comment 3 Vijaikumar Mallikarjuna 2014-05-06 11:42:23 UTC
Patch http://review.gluster.org/#/c/7581/ will fix this issue.

Comment 4 Nagaprasad Sathyanarayana 2014-05-06 12:06:01 UTC
Fixing RHS 3.0 flags.

Comment 5 Nagaprasad Sathyanarayana 2014-05-06 12:12:54 UTC
Fixing RHS 3.0 flags.

Comment 6 Nagaprasad Sathyanarayana 2014-05-19 10:56:36 UTC
Setting flags required to add BZs to RHS 3.0 Errata

Comment 7 senaik 2014-05-21 09:29:52 UTC
As per bz 1054705 , snap creation is now successful on volume with bricks from same VG, unable to reproduce the issue with steps as mentioned in Description. 

Marking bug as 'Verified', 1054705 has also been verified.

Comment 9 errata-xmlrpc 2014-09-22 19:36:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1278.html


Note You need to log in before you can comment on or make changes to this bug.