Description of problem: Block volume creation should fail when the total capacity of block hosting volume is exhausted. For ex, when block hosting volume is 500 GB, the total size of all block devices that could be created should be limited to 500 GB and any attempt to create a block device over and above that mark should fail. Gluster allows to create block devices above the capacity of volume it is hosting on as only sparse files are created by default in gluster. so heketi cannot depend on gluster command to fail, rather the request to create block volume should not proceed from heketi to gluster if the capacity is exceeded. [root@dhcp47-10 ~]# heketi-cli blockvolume create --size=480 Name: blockvol_be664f518291761bf14fd5741362b5b8 Size: 480 Volume Id: be664f518291761bf14fd5741362b5b8 Cluster Id: 6bed12159b987cf70338ac6d1d3d0720 Hosts: [10.70.46.248 10.70.47.49 10.70.47.72] IQN: iqn.2016-12.org.gluster-block:15695855-a4ab-4f20-9670-6eb9901ba1a1 LUN: 0 Hacount: 3 Username: Password: Block Hosting Volume: 78dfd43a5abacb9e8365500d54d4f6e5 [root@dhcp47-10 ~]# heketi-cli blockvolume create --size=30 Name: blockvol_0ced9f74bb5cb6d364a35f2752603c6c Size: 30 Volume Id: 0ced9f74bb5cb6d364a35f2752603c6c Cluster Id: 6bed12159b987cf70338ac6d1d3d0720 Hosts: [10.70.46.248 10.70.47.49 10.70.47.72] IQN: iqn.2016-12.org.gluster-block:cfa768eb-723a-4a30-8b41-fdc56dfab6a9 LUN: 0 Hacount: 3 Username: Password: Block Hosting Volume: 78dfd43a5abacb9e8365500d54d4f6e5 [root@dhcp47-10 ~]# heketi-cli blockvolume create --size=30 Name: blockvol_b7d4e9737f20d25f0b305a10dd519076 Size: 30 Volume Id: b7d4e9737f20d25f0b305a10dd519076 Cluster Id: 6bed12159b987cf70338ac6d1d3d0720 Hosts: [10.70.46.248 10.70.47.49 10.70.47.72] IQN: iqn.2016-12.org.gluster-block:04a8d622-314d-49bf-80c4-28ad2137ded6 LUN: 0 Hacount: 3 Username: Password: Block Hosting Volume: 78dfd43a5abacb9e8365500d54d4f6e5 [root@dhcp47-10 ~]# heketi-cli blockvolume list Id:0ced9f74bb5cb6d364a35f2752603c6c Cluster:6bed12159b987cf70338ac6d1d3d0720 Name:blockvol_0ced9f74bb5cb6d364a35f2752603c6c Id:a3b7006bde73028b651d7362f6ba78a1 Cluster:6bed12159b987cf70338ac6d1d3d0720 Name:blockvol_a3b7006bde73028b651d7362f6ba78a1 Id:b7d4e9737f20d25f0b305a10dd519076 Cluster:6bed12159b987cf70338ac6d1d3d0720 Name:blockvol_b7d4e9737f20d25f0b305a10dd519076 Id:be664f518291761bf14fd5741362b5b8 Cluster:6bed12159b987cf70338ac6d1d3d0720 Name:blockvol_be664f518291761bf14fd5741362b5b8 [root@dhcp47-10 ~]# heketi-cli volume list Id:78dfd43a5abacb9e8365500d54d4f6e5 Cluster:6bed12159b987cf70338ac6d1d3d0720 Name:vol_78dfd43a5abacb9e8365500d54d4f6e5 [block] Id:fee4f0b4b98d1c468d1ebac9ed5acd44 Cluster:6bed12159b987cf70338ac6d1d3d0720 Name:heketidbstorage [root@dhcp47-10 ~]# heketi-cli volume info 78dfd43a5abacb9e8365500d54d4f6e5 Name: vol_78dfd43a5abacb9e8365500d54d4f6e5 Size: 500 Volume Id: 78dfd43a5abacb9e8365500d54d4f6e5 Cluster Id: 6bed12159b987cf70338ac6d1d3d0720 Mount: 10.70.46.248:vol_78dfd43a5abacb9e8365500d54d4f6e5 Mount Options: backup-volfile-servers=10.70.47.49,10.70.47.72 Block: true Free Size: -50 ---->>> SIZE GONE NEGATIVE Block Volumes: [0ced9f74bb5cb6d364a35f2752603c6c a3b7006bde73028b651d7362f6ba78a1 b7d4e9737f20d25f0b305a10dd519076 be664f518291761bf14fd5741362b5b8] Durability Type: replicate Distributed+Replica: 3 Version-Release number of selected component (if applicable): heketi-client-5.0.0-4.el7rhgs.x86_64 cns-deploy-5.0.0-6.el7rhgs.x86_64 How reproducible: Always Steps to Reproduce: 1. create a block hosting device of 500GB 2. Try to create multiple block devices over above 500 GB mark Actual results: block vol creation succeeds oversubscribing from 500GB Expected results: block volume creation should fail Additional info:
Fixed in upstream: https://github.com/heketi/heketi/pull/793/commits/ca6a846df3200a0a11aaf87b79648c29b303559f
Verified the fix in build - cns-deploy-5.0.0-25.el7rhgs.x86_64 Heketi now fails appropriately when a block device of size greater than that of block hosting volume is requested. Access Modes: Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 11s 11s 1 gluster.org/glusterblock 695d0b4b-8ed9-11e7-be38-0a580a8100dd Normal Provisioning External provisioner is provisioning volume for claim "storage-project/test-04" 10s 10s 1 gluster.org/glusterblock 695d0b4b-8ed9-11e7-be38-0a580a8100dd Warning ProvisioningFailed Failed to provision volume with StorageClass "glusterblockip": glusterblock: failed to create volume: [heketi] error creating volume No space 11s 9s 3 persistentvolume-controller Normal ExternalProvisioning cannot find provisioner "gluster.org/glusterblock", expecting that a volume for the claim is provisioned either manually or via external software Moving the bug to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:2879