Description of problem:

If newly added brick runs out of space then on the mount point duplicate files will appear

Version-Release number of selected component (if applicable):
Mainline

How reproducible:
Alwyas

Steps to Reproduce:
1.create a distribute volume with 1 brick
2.fill the mount point with 100 files of some size (make sure to cause space crunch)
3.add a new brick which is relatively low in disk space so that easily we will run out of space  
4. Initiate the rebalance
  
Actual results:
In the log actually shows "No space left on the device", but on the mount point ls will show some of the files duplicated .

Expected results:


Additional info: