Bug 762574 (GLUSTER-842)

Summary: No space left on device error
Product: [Community] GlusterFS Reporter: Gluster Fan <jaz.cheng>
Component: stripeAssignee: Amar Tumballi <amarts>
Status: CLOSED WONTFIX QA Contact:
Severity: medium Docs Contact:
Priority: urgent    
Version: 3.0.3CC: gluster-bugs, vijay, vraman
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Gluster Fan 2010-04-21 09:31:54 UTC
Whichever translator I use, I will eventually run into this 
situation where I will try to write a file and gluster will report "no 
free space" while there is free space, either because it needs to 
write an entry for each file in every node, or because the hash function 
used to decide the node is not able to fall back to another node when 
this one is full?

To me, this makes the whole glusterfs totally useless, as I will 
randomly get no free space errors even if there is space, so when will resolve this problem?

Comment 1 Amar Tumballi 2010-05-07 04:21:11 UTC
This is quite tricky to fix. 

By the design of 'cluster/stripe' we can't fix this behavior at all. If the backend on one of the server belonging to stripe's subvolume is full, stripe-write()'s will fail reporting No Space Left on device. 

In distribute, this can be solved iff we come up with a proper solution, where all the clients understand that one server is above certain disk limit, and exclude that volume from calculating hash value for a given filename. With 3.0.x releases, we log the info about disks getting full as per 'min-disk-free' option. 
This option can be tunable, so admin can get to know about disks getting full before it hits 100% full.

Ideally, if disks are getting 90% full, that means time to extend the storage has come.

The issue of ENOSPC (No Space left on device) will be addressed only in future releases, as this is currently not considered high priority.

Comment 2 Amar Tumballi 2010-07-23 12:31:20 UTC
can't add this feature in 3.1 release.

Comment 3 Vijay Bellur 2010-10-27 06:02:54 UTC
(In reply to comment #0)
> Whichever translator I use, I will eventually run into this 
> situation where I will try to write a file and gluster will report "no 
> free space" while there is free space, either because it needs to 
> write an entry for each file in every node, or because the hash function 
> used to decide the node is not able to fall back to another node when 
> this one is full?
> 
> To me, this makes the whole glusterfs totally useless, as I will 
> randomly get no free space errors even if there is space, so when will resolve
> this problem?

This is a design limitation and we do not intend to address this as of now.