Bug 1724094

Summary:	New files created after toggling the cluster.weighted-rebalance option and doing rebalance on the volume, are not placed correctly as per the hash
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Sayalee <saraut>
Component:	distribute	Assignee:	Susant Kumar Palai <spalai>
Status:	CLOSED DUPLICATE	QA Contact:	Sayalee <saraut>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	rhgs-3.5	CC:	nbalacha, rhs-bugs, sankarshan, storage-qa-internal
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-06-28 02:30:22 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sayalee 2019-06-26 08:53:29 UTC

Description of problem:
On a heterogenous brick volume, newly created files were not placed correctly on the back-end, before creating new files, the cluster.weighted-rebalance option was disabled and rebalance triggered on the volume.

Version-Release number of selected component (if applicable):
6.0.6

How reproducible:
consistently

Steps to Reproduce:
1. Create a 2 brick distributed volume. (20G and 5G bricks)
2. Fuse mount the volume on a client node.
3. Create dir1 on the mount point.
4. Check the hash layout for the dir1 on the backend bricks.
5. Create files inside the dir from the mount point.
#cd dir1
#touch f17 and f8
6. Now, disable the cluster.weighted-rebalance option on the volume (it is set to "on" by default)
7. Trigger rebalance on the volume.
8. Check the hash layout of dir1 on backend bricks.
9. Create new files. Check if files are placed correctly on back-end bricks.  <-- Files not placed correctly.

---------As files were not placed correctly, did following steps-------

10. Now mount the volume on a new client node. 

11. Look up from new client:
#cd dir1/
# ls
f12  f17  f7  f8

12. Try creating the files f12 and f7 again. (f12 and f7 already exist)
# touch f7 f12
# ls
f12  f17  f7  f8

13. Check if files are placed correctly on back-end. <--- Files not placed correctly.

14. Now remove the files f12 and f7 and create them again on the new mount.
# rm -rf f7 f12
# ls
f17  f8
# touch f7 f12
# ls
f12  f17  f7  f8

15. Check if files are placed correctly on back-end. <--- Now files will be placed correctly.

Actual results:
On creating new files f12 and f7(which hash to the 5G brick according to the new layout) are created on 20G brick.

Expected results:
Files f12 and f7 should be created on the 5G brick as per the hash value of file and the hash layout.

Additional info:
sos-reports will be provided.