Bug 861497

Summary:	Deleting volume sometimes leaves directory in /var/lib/glusterd
Product:	[Community] GlusterFS	Reporter:	Shawn Heisey <redhat>
Component:	glusterd	Assignee:	Junaid <junaid>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	medium
Version:	3.3.0	CC:	gluster-bugs, jdarcy, junaid, redhat, vagarwal
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	glusterfs-3.4.0	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-07-24 17:53:31 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Shawn Heisey 2012-09-28 20:12:10 UTC

Description of problem:
If there is any extraneous file in /var/lib/glusterd/$volname, deleting the volume will not delete that folder.  This in turn will make glusterd refuse to start the next time that is required.

Steps to Reproduce:
1. Put an extraneous file (such as run/swift.pid) into the volume directory.
2. Stop and delete the volume.
3. Restart glusterd.
  
Actual results:
Glusterd will not start, because it sees a volume directory and cannot find the info file.

Expected results:
It should allow startup (with logged warnings/errors) when it discovers that none of the expected files are in the volume directory.  If some of the files exist but others don't, it would be appropriate to not allow startup.  If a startup is allowed when a volume directory exists but none of the volume files exist, it may be a good idea to require additional action (after manually deleting the directory and any files it may contain) before a volume with the same name can be created.  Perhaps some special cleanup or force option on the delete command.  I leave that in your capable hands.

Additional info:
The specific file that seems to have caused this is a pid file from gluster-swift:
/var/lib/glusterd/vols/$volname/run/swift.pid

Comment 1 Shawn Heisey 2012-09-28 20:18:57 UTC

An alternate solution (one that sounds like a good option to me) would be to forcibly remove the volume directory and everything in it when a volume delete is issued.  The directory is created by glusterd, and in theory should never be modified by anything external, so allowing glusterd sole control over its existence doesn't seem like too much of a stretch.

I think it's a bug that gluster-swift puts its pidfile in a directory managed by glusterd.  If someone knows where I can file that bug, I would be happy to do so.

Comment 2 Jeff Darcy 2012-09-28 21:08:09 UTC

Swift shouldn't be putting a PID file in /var/lib/anything; that's what /var/run is for.  Even so, the condition described here shouldn't prevent glusterd from starting.

Comment 3 Vijay Bellur 2013-02-07 22:02:34 UTC

CHANGE: http://review.gluster.org/4417 (object-storage: Store the lock file in /var/run/swift.) merged in master by Anand Avati (avati)

Comment 4 Junaid 2013-02-08 06:17:38 UTC

Fixed in upstream master.