Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: If we create PVC on top of cns block device, PVC will be of size specified, but underlying block device will be bigger always 1 GB Version-Release number of selected component (if applicable): atomic-openshift-3.7.0-0.127.0.git.0.459b70b.el7.x86_64 atomic-openshift-master-3.7.0-0.127.0.git.0.459b70b.el7.x86_64 python-heketi-5.0.0-15.el7rhgs.x86_64 atomic-registries-1.19.1-3.gitb39a783.el7.x86_64 atomic-openshift-clients-3.7.0-0.127.0.git.0.459b70b.el7.x86_64 atomic-openshift-excluder-3.7.0-0.127.0.git.0.459b70b.el7.noarch atomic-openshift-node-3.7.0-0.127.0.git.0.459b70b.el7.x86_64 atomic-openshift-sdn-ovs-3.7.0-0.127.0.git.0.459b70b.el7.x86_64 heketi-5.0.0-15.el7rhgs.x86_64 heketi-client-5.0.0-15.el7rhgs.x86_64 atomic-openshift-docker-excluder-3.7.0-0.127.0.git.0.459b70b.el7.noarch tuned-profiles-atomic-openshift-node-3.7.0-0.127.0.git.0.459b70b.el7.x86_64 cns-deploy-5.0.0-52.el7rhgs.x86_64 How reproducible: always Steps to Reproduce: 1. create cns cluster with block device support 2. create storage class 3. create PVC of size 1 Gib Actual results: PVC will of requested size, but size of device mounted inside pod will be 1 GB bigger Expected results: size of mounted device to be same as size of PVC Additional info: # oc get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE fio-pod-fx4h7 1/1 Running 0 3m 172.23.2.76 gprfc040.sbu.lab.eng.bos.redhat.com fio-pod-hvp1t 1/1 Running 0 3m 172.22.0.23 gprfc031.sbu.lab.eng.bos.redhat.com fio-pod-p3z56 1/1 Running 0 3m 172.21.0.27 gprfc021.sbu.lab.eng.bos.redhat.com [root@gprfc051 ~/oseinstall/svt_elko/openshift_scalability]# oc get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE pvc2vt0jef23n Bound pvc-e6734659-af3a-11e7-96f3-d4bed9f5103b 2Gi RWO glusterblock 3m pvco1ppdehe5u Bound pvc-e7561bd3-af3a-11e7-96f3-d4bed9f5103b 2Gi RWO glusterblock 3m pvcr1kxmsw62c Bound pvc-e832fbe5-af3a-11e7-96f3-d4bed9f5103b 2Gi RWO glusterblock 3m [root@gprfc051 ~/oseinstall/svt_elko/openshift_scalability]# oc exec fio-pod-fx4h7 -- df -h | grep gluster /dev/sdb 3.0G 33M 3.0G 2% /mnt/glusterfs ssh gprfc040.sbu.lab.eng.bos.redhat.com "fdisk -l | grep sd | grep -v sda" Disk /dev/sdb: 3221 MB, 3221225472 bytes, 6291456 sectors Disk /dev/sdc: 3221 MB, 3221225472 bytes, 6291456 sectors Disk /dev/sdd: 3221 MB, 3221225472 bytes, 6291456 sectors also I tried to create block volume manually and map it to node steps: -- # heketi-cli blockvolume create --size 1 Name: blockvol_bb14712ec6c8e54f581a57a5254f7f46 Size: 1 Volume Id: bb14712ec6c8e54f581a57a5254f7f46 Cluster Id: 6dc76e0d5aa56479393a53afd17bba6b Hosts: [10.16.153.78 10.16.153.81 10.16.153.93] IQN: iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1 LUN: 0 Hacount: 3 Username: Password: Block Hosting Volume: 53891699edcc38480b72bc255ade30d1 [root@gprfc051 /usr/share/heketi]# iscsiadm -m discovery --op update --type st --portal 10.16.153.78 10.16.153.78:3260,1 iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1 10.16.153.81:3260,2 iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1 10.16.153.93:3260,3 iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1 [root@gprfc051 /usr/share/heketi]# iscsiadm --mode node -l all Logging in to [iface: default, target: iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1, portal: 10.16.153.78,3260] (multiple) Logging in to [iface: default, target: iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1, portal: 10.16.153.81,3260] (multiple) Logging in to [iface: default, target: iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1, portal: 10.16.153.93,3260] (multiple) Login to [iface: default, target: iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1, portal: 10.16.153.78,3260] successful. Login to [iface: default, target: iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1, portal: 10.16.153.81,3260] successful. Login to [iface: default, target: iqn.2016-12.org.gluster-block:80fcc554-17ef-4487-955e-db49305033d1, portal: 10.16.153.93,3260] successful. [root@gprfc051 /usr/share/heketi]# fdisk -l |grep sd Disk /dev/sda: 500.1 GB, 500107862016 bytes, 976773168 sectors /dev/sda1 * 2048 2099199 1048576 83 Linux /dev/sda2 2099200 976773119 487336960 8e Linux LVM Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors Description of problem: Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info:
Elvir, this is actually working as expected. That said, if you want to have exactly "2GB" of space, you can create a PVC with "2GB" in its definition, this should create exactly "2GB" of storage. But if you request "2GiB", by PVC spec definition it create and round to next GB thats why you are getting "3GB" in mount point. Because GiB is 1024^3 where GB is 1000^3 , by PVC spec definition the claim should satisfy >= requested size and never less than the requested size. In short, this is not a bug rather working as expected. However please let me know if you have any question on this.
(In reply to Humble Chirammal from comment #1) > Elvir, this is actually working as expected. That said, if you want to have > exactly "2GB" of space, you can create a PVC with "2GB" in its definition, > this should create exactly "2GB" of storage. But if you request "2GiB", by > PVC spec definition it create and round to next GB thats why you are getting > "3GB" in mount point. Because GiB is 1024^3 where GB is 1000^3 , by PVC spec > definition the claim should satisfy >= requested size and never less than > the requested size. > > In short, this is not a bug rather working as expected. However please let > me know if you have any question on this. This is only visible and case when cns block is used, cns file has sizes ok: PVC size == size of mount inside pod.
(In reply to Elvir Kuric from comment #3) > (In reply to Humble Chirammal from comment #1) > > Elvir, this is actually working as expected. That said, if you want to have > > exactly "2GB" of space, you can create a PVC with "2GB" in its definition, > > this should create exactly "2GB" of storage. But if you request "2GiB", by > > PVC spec definition it create and round to next GB thats why you are getting > > "3GB" in mount point. Because GiB is 1024^3 where GB is 1000^3 , by PVC spec > > definition the claim should satisfy >= requested size and never less than > > the requested size. > > > > In short, this is not a bug rather working as expected. However please let > > me know if you have any question on this. > > This is only visible and case when cns block is used, cns file has sizes ok: > PVC size == size of mount inside pod. Agreed, but this has been corrected or adjusted for File volumes in upstream and it should be available from next OCP version onwards. The main inconsistency occured as Heketi only know about "GB" in its request.
This is fixed in latest gluster-block provisioner container: rhgs-gluster-block-prov-container-3.3.1-1 and cns-deploy-6.0.0-2.el7rhgs
verified in the following build - oc version oc v3.9.3 kubernetes v1.9.1+a0ce1bc657 features: Basic-Auth GSSAPI Kerberos SPNEGO Server https://dhcp47-22.lab.eng.blr.redhat.com:8443 openshift v3.9.3 kubernetes v1.9.1+a0ce1bc657 heketi version - heketi-6.0.0-6.el7rhgs.x86_64 [root@dhcp47-22 ~]# oc get pods NAME READY STATUS RESTARTS AGE glusterblock-storage-provisioner-dc-1-f2b5r 1/1 Running 0 3d glusterfs-storage-6d2rm 1/1 Running 0 3d glusterfs-storage-9whjp 1/1 Running 0 3d glusterfs-storage-pkw6f 1/1 Running 0 3d heketi-storage-1-q7fgw 1/1 Running 0 3d mongodb-01-1-qclfw 1/1 Running 0 21m mongodb-02-1-drh5l 1/1 Running 0 5m [root@dhcp47-22 ~]# oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE claim1 Bound pvc-f5b6de5b-22cb-11e8-853f-005056a5c33b 5Gi RWO gluster-block 3d mongodb-01 Bound pvc-6259dff2-25b0-11e8-a357-005056a5c33b 1Gi RWO gluster-block 21m mongodb-02 Bound pvc-7b8bf382-25b1-11e8-a357-005056a5c33b 2Gi RWO gluster-block 14m [root@dhcp47-22 ~]# oc rsh mongodb-01-1-qclfw sh-4.2# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/docker-253:2-33576569-5a56029d00dc677d497bbfeb6ce8edd49cd04271ff7eb5aec930a443899e1642 10G 462M 9.6G 5% / tmpfs 24G 0 24G 0% /dev tmpfs 24G 0 24G 0% /sys/fs/cgroup /dev/mapper/vg_rhel_dhcp47----104--var-lv_var 59G 2.2G 57G 4% /etc/hosts shm 64M 0 64M 0% /dev/shm /dev/sdg 1014M 233M 782M 23% /var/lib/mongodb/data ----> 1GB device tmpfs 24G 16K 24G 1% /run/secrets/kubernetes.io/serviceaccount tmpfs 24G 0 24G 0% /proc/scsi tmpfs 24G 0 24G 0% /sys/firmware sh-4.2# exit [root@dhcp47-22 ~]# oc rsh mongodb-02-1-drh5l sh-4.2# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/docker-253:2-33576537-e121a59b05a18d0bc1c8dc1f52470703330b142b91f3a757d5c337404fa761b0 10G 462M 9.6G 5% / tmpfs 24G 0 24G 0% /dev tmpfs 24G 0 24G 0% /sys/fs/cgroup /dev/mapper/vg_rhel_dhcp47----104--var-lv_var 59G 4.4G 55G 8% /etc/hosts shm 64M 0 64M 0% /dev/shm /dev/sdg 2.0G 233M 1.8G 12% /var/lib/mongodb/data -------> 2 GB device tmpfs 24G 16K 24G 1% /run/secrets/kubernetes.io/serviceaccount tmpfs 24G 0 24G 0% /proc/scsi tmpfs 24G 0 24G 0% /sys/firmware
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:0642