Bug 1138390

+++ This bug was initially created as a clone of Bug #1123950 +++

Description of problem:
This problem is hit as a part of this test case, tests/bugs/bug-1117851.t about once every 100 files (based on the backend disk for the volume (i.e ram disk/ssd/others)).

The issue being seen when hashed and cached subvols for a file are the same and it is being renamed to another file whose hased subvol is different.

The root cause of this issue is due to the fact that both clients race to create the link and linkto file in the above scenario, and the losing client goes ahead and deletes the linkto file in its cleanup, thereby the actual rename attempted by the winning client fails, ending up in both clients failing to rename the file.

Fixing the part of the client that fails to create the linkto file, to not delete the linkto file will not be sufficient, as the losing client could have won that race (as link and linkto are wound in parallel). Which is present in this review, http://review.gluster.org/#/c/8338/

The additional fix to handle this failure, is to make the wind's to create the link and the linkto sequential, so that whichever client wins the link race, can then go ahead with creating the linkto file and hence have a clear client proceeding and the other client getting the required errors.

Version-Release number of selected component (if applicable):
Gluster master

How reproducible:
1 in 100 renames if run on bircks on SSD or RAM disk

Steps to Reproduce:
Test case, tests/bugs/bug-1117851.t treating warnings on file rename failures as errors (see comment in tesst case file)

Also, this should be a fork from bug #1117851, but as this is not a data loss, only refering the original bug here.