Bug 1572661

Summary: heketi is not showing actual used and free size of devices
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Nitin Goyal <nigoyal>
Component: heketiAssignee: John Mulligan <jmulligan>
Status: CLOSED ERRATA QA Contact: Nitin Goyal <nigoyal>
Severity: high Docs Contact:
Priority: high    
Version: rhgs-3.3CC: hchiramm, jmulligan, pprakash, rhs-bugs, rtalur, sankarshan, sarumuga, storage-qa-internal
Target Milestone: ---Keywords: Regression, TestBlocker
Target Release: CNS 3.10   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-12 09:22:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1568862    
Attachments:
Description Flags
this link contain all sosreports and logs none

Description Nitin Goyal 2018-04-27 14:19:34 UTC
Description of problem: heketi is not showing actual used and free size of devices. There is one device in which i have 1 brick of 2 gb and size of device is 99 gb, So free size will be 97 gb because of that 2 gb of brick, but it is showing 99 gb.

Version-Release number of selected component (if applicable):6.0.0-11


How reproducible:


Steps to Reproduce:
1.create cns setup.
2.delete the device where heketi db is stored.

Actual results: it is showing 99 gb of free disk space.

Expected results: It should show 97 gb of free space.

Comment 2 Nitin Goyal 2018-04-27 15:03:14 UTC
Created attachment 1427722 [details]
this link contain all sosreports and logs

Comment 4 John Mulligan 2018-05-02 13:41:39 UTC
Could you please elaborate on step 2, "delete the device where heketi db is stored"?  If you can please detail exactly what commands you ran.

Thanks.

Comment 5 Nitin Goyal 2018-05-03 06:39:51 UTC
Suppose we have 3 nodes and we have 3 devices on each node. After this we have heketi DB volume's brick on device sdc on node1. We need to delete sdc device from node1.

# heketi-cli device disable sdc_device_id_of_node1
# heketi-cli device remove sdc_device_id_of_node1
# heketi-cli device delete sdc_device_id_of_node1

Comment 6 John Mulligan 2018-05-03 13:53:11 UTC
Thanks for the steps. I can reproduce it myself now as well.

Comment 7 Nitin Goyal 2018-05-07 08:50:21 UTC
See the used and free size of devices in node 2

Cluster Id: ed30ac3be9f1c3d7341fbe228ef8884c

    File:  true
    Block: true

    Volumes:

	Name: heketidbstorage
	Size: 2
	Id: 939b50d3838e3461bb9a37398a36afc3
	Cluster Id: ed30ac3be9f1c3d7341fbe228ef8884c
	Mount: 10.70.46.114:heketidbstorage
	Mount Options: backup-volfile-servers=10.70.46.130,10.70.46.212
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: acd5fa5a75e66e5f48c0b1d87214e246
			Path: /var/lib/heketi/mounts/vg_c3788502514b7ba5d9db8874d2d96713/brick_acd5fa5a75e66e5f48c0b1d87214e246/brick
			Size (GiB): 2
			Node: d6981d69ea2d829440f6208445a5d274
			Device: c3788502514b7ba5d9db8874d2d96713

			Id: b7e7cf184f37a4c4c2f8fb42f50180e4
			Path: /var/lib/heketi/mounts/vg_66ede0d16ea3218ed31b15fb585f58cf/brick_b7e7cf184f37a4c4c2f8fb42f50180e4/brick
			Size (GiB): 2
			Node: 15502f7b3486d9ed2380cccda369df7c
			Device: 66ede0d16ea3218ed31b15fb585f58cf

			Id: e27ee412f15355fb2c39025fc0a4ecae
			Path: /var/lib/heketi/mounts/vg_b325644642b912d2edd20a52fcd59348/brick_e27ee412f15355fb2c39025fc0a4ecae/brick
			Size (GiB): 2
			Node: 6848435f18d95398647af3df0af12d31
			Device: b325644642b912d2edd20a52fcd59348


    Nodes:

	Node Id: 15502f7b3486d9ed2380cccda369df7c
	State: online
	Cluster Id: ed30ac3be9f1c3d7341fbe228ef8884c
	Zone: 1
	Management Hostnames: dhcp46-114.lab.eng.blr.redhat.com
	Storage Hostnames: 10.70.46.114
	Devices:
		Id:27f659e59ffb54bc00a4d5725e8f9e36   Name:/dev/sdd            State:online    Size (GiB):99      Used (GiB):0       Free (GiB):99      
			Bricks:
		Id:42f517849d7e540a95b6f9dd20b122ea   Name:/dev/sde            State:online    Size (GiB):99      Used (GiB):0       Free (GiB):99      
			Bricks:
		Id:66ede0d16ea3218ed31b15fb585f58cf   Name:/dev/sdc            State:online    Size (GiB):99      Used (GiB):0       Free (GiB):99      
			Bricks:
				Id:b7e7cf184f37a4c4c2f8fb42f50180e4   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_66ede0d16ea3218ed31b15fb585f58cf/brick_b7e7cf184f37a4c4c2f8fb42f50180e4/brick

	Node Id: 6848435f18d95398647af3df0af12d31
	State: online
	Cluster Id: ed30ac3be9f1c3d7341fbe228ef8884c
	Zone: 2
	Management Hostnames: dhcp46-130.lab.eng.blr.redhat.com
	Storage Hostnames: 10.70.46.130
	Devices:
		Id:8bba7b683f561e1ba28568494c3806f8   Name:/dev/sde            State:online    Size (GiB):99      Used (GiB):17592186044385Free (GiB):130     
			Bricks:
		Id:a2c97dd2154d68f173db13444e513708   Name:/dev/sdd            State:offline   Size (GiB):99      Used (GiB):0       Free (GiB):99      
			Bricks:
		Id:b325644642b912d2edd20a52fcd59348   Name:/dev/sdc            State:online    Size (GiB):99      Used (GiB):17592186044375Free (GiB):140     
			Bricks:
				Id:e27ee412f15355fb2c39025fc0a4ecae   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_b325644642b912d2edd20a52fcd59348/brick_e27ee412f15355fb2c39025fc0a4ecae/brick

	Node Id: d6981d69ea2d829440f6208445a5d274
	State: online
	Cluster Id: ed30ac3be9f1c3d7341fbe228ef8884c
	Zone: 3
	Management Hostnames: dhcp46-212.lab.eng.blr.redhat.com
	Storage Hostnames: 10.70.46.212
	Devices:
		Id:26a3fa9fe58d6dffb04bc83cbd05576c   Name:/dev/sdd            State:online    Size (GiB):99      Used (GiB):0       Free (GiB):99      
			Bricks:
		Id:7a7d20c82233286bf7085dc413f5a8e1   Name:/dev/sde            State:online    Size (GiB):99      Used (GiB):0       Free (GiB):99      
			Bricks:
		Id:c3788502514b7ba5d9db8874d2d96713   Name:/dev/sdc            State:online    Size (GiB):99      Used (GiB):0       Free (GiB):99      
			Bricks:
				Id:acd5fa5a75e66e5f48c0b1d87214e246   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_c3788502514b7ba5d9db8874d2d96713/brick_acd5fa5a75e66e5f48c0b1d87214e246/brick

Comment 9 John Mulligan 2018-05-08 19:45:23 UTC
Additional fix for arbiter volumes: https://github.com/heketi/heketi/pull/1171

Comment 15 errata-xmlrpc 2018-09-12 09:22:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2686