Description of problem:

As volumes get fuller performance starts to degrade.  I was adding data to a 20 TB volume and I started to notice the perf hit when it hit 2 TB(10% full).  To quantify we created an empty volume on the same bricks and ran the following reproducer:

# time tar xf boost_1_57_0.tar.gz

On the fuller / aged volume it took significantly longer to untar.

Version-Release number of selected component (if applicable):

glusterfs-3.6.0.42-1.el6rhs.x86_64

How reproducible:

Every time I tried so far.

Steps to Reproduce:
1.  Create teo volumes on the same bricks
2.  Fill 1 up with data(~50%)
3.  Untar the files

Actual results:

Performance degrades when the volume hits ~10% usage.

Expected results:

Performance does not degrade as the volume gets fuller.

Additional info:

We have this reproed in the lab, ping me for access to the setuo.