Red Hat Bugzilla – Bug 861497
Deleting volume sometimes leaves directory in /var/lib/glusterd
Last modified: 2013-08-06 18:39:00 EDT
Description of problem:
If there is any extraneous file in /var/lib/glusterd/$volname, deleting the volume will not delete that folder. This in turn will make glusterd refuse to start the next time that is required.
Steps to Reproduce:
1. Put an extraneous file (such as run/swift.pid) into the volume directory.
2. Stop and delete the volume.
3. Restart glusterd.
Glusterd will not start, because it sees a volume directory and cannot find the info file.
It should allow startup (with logged warnings/errors) when it discovers that none of the expected files are in the volume directory. If some of the files exist but others don't, it would be appropriate to not allow startup. If a startup is allowed when a volume directory exists but none of the volume files exist, it may be a good idea to require additional action (after manually deleting the directory and any files it may contain) before a volume with the same name can be created. Perhaps some special cleanup or force option on the delete command. I leave that in your capable hands.
The specific file that seems to have caused this is a pid file from gluster-swift:
An alternate solution (one that sounds like a good option to me) would be to forcibly remove the volume directory and everything in it when a volume delete is issued. The directory is created by glusterd, and in theory should never be modified by anything external, so allowing glusterd sole control over its existence doesn't seem like too much of a stretch.
I think it's a bug that gluster-swift puts its pidfile in a directory managed by glusterd. If someone knows where I can file that bug, I would be happy to do so.
Swift shouldn't be putting a PID file in /var/lib/anything; that's what /var/run is for. Even so, the condition described here shouldn't prevent glusterd from starting.
CHANGE: http://review.gluster.org/4417 (object-storage: Store the lock file in /var/run/swift.) merged in master by Anand Avati (firstname.lastname@example.org)
Fixed in upstream master.