Description of problem:
From Peter Portante's Fix (http://review.gluster.org/5670) for gluster-swift master (Havana) branch:
    Fix infinite loop for temp file renames on ENOENT
    
    For whatever reason, it appears that GlusterFS, or perhaps FUSE can
    continuously return ENOENT on a rename system call even when we have
    double checked that there is no reason to do so. That is a bug for
    that sub system. However, our response to that bug can result in an
    infinite loop, which is bad.
    
    This code reduces that to 10 attempts.
    
    In addition, we restructed the open retry loop to match, providing
    module constants for the upper bounds of both retry loops.

Peter has been able to see this issue on his multiple node performance tests
for RHS 2.1.  The fix needs to be backported to upstream gluster-swift Grizzly
branch.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: