Hi,
Here the system df and the Gluster is reporting the different percentage of disk use.

[........]
Below are the output from 'statfs()' call of customer.


> This brick are now to 89%:
>
>
> [root@llfe894 ~]# strace -f -v df -h
> statfs("/data", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096,
> f_blocks=233970473, f_bfree=38103468, f_bavail=26026705,
> f_files=241565696, f_ffree=239328261, f_fsid={0, 0}, f_namelen=255,
> f_frsize=4096}) = 0
> write(1, "/dev/mapper/ddf1_RAIDp3\n", 24/dev/mapper/ddf1_RAIDp3
> ) = 24
> write(1, " 893G 748G"..., 50
> 893G 748G 100G 89% /data
> ) = 50
>

Here df is using the formula 'f_bavail * 100 / f_blocks' to get the
available space, and we as said earlier use formula 'f_bfree * 100 /
f_blocks'.


> percent = (statvfs->f_bfree * 100) / statvfs->f_blocks;
> > 'man statvfs'
> >
> > Where's:
> > f_bfree = /* # free blocks */
> > f_blocks = /* size of fs in f_frsize units */
>

long f_bavail; /* free blocks avail to non-superuser */


> > f_frsize = /* fragment size */
> > [...]
> >
>

That explains the current behavior of he seeing less than 'min-free-disk'
space in filesystem, but GlusterFS seeing more than 'min-free-disk'.

[........]

PATCH: http://patches.gluster.com/patch/7765 in master (dht disk usage: Correct free blk calculation)

Formula has changed to calculate the available space

if (statvfs && statvfs->f_blocks) {
                percent = (statvfs->f_bavail * 100) / statvfs->f_blocks;
                bytes = (statvfs->f_bavail * statvfs->f_frsize);
        }