Red Hat Bugzilla – Bug 1451013
PVC has to be minimum 4 GB when created dynamically via storage class which uses EBS device type : io1
Last modified: 2017-08-16 15:51 EDT
Description of problem: When kubernetes.io/aws-ebs dynamic storage provisioner is used with OCP on EC2, creation of new volumes will fail for case when a) volume type specified in storage class is io1 b) volume size is less than 4 GB When executed from command in "oc describe pvc <pvcname>" there will be error message as specified in "Additional info" section of this BZ. When tried to create PVC via OCP web interface for this case, there will not be visible error message and PVC will hang in "Pending" state indefinitely without giving reason why it is in "Pending state" Amazon ec2 specifies at below link that minimum io1 device size is 4GB -> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html Version-Release number of selected component (if applicable): atomic-openshift-sdn-ovs-3.5.5.9-1.git.0.d220e61.el7.x86_64 atomic-openshift-clients-3.5.5.9-1.git.0.d220e61.el7.x86_64 atomic-openshift-node-3.5.5.9-1.git.0.d220e61.el7.x86_64 atomic-openshift-3.5.5.9-1.git.0.d220e61.el7.x86_64 tuned-profiles-atomic-openshift-node-3.5.5.9-1.git.0.d220e61.el7.x86_64 atomic-openshift-docker-excluder-3.5.5.9-1.git.0.d220e61.el7.noarch atomic-openshift-master-3.5.5.9-1.git.0.d220e61.el7.x86_64 atomic-openshift-excluder-3.5.5.9-1.git.0.d220e61.el7.noarch How reproducible: always Steps to Reproduce: configure storage class and dynamic provisioning to use aws-ebs provisioner and io1 device type, and try to create PVC smaller than 4 GB example storage class oc describe storageclass ec2class Name: ec2class IsDefaultClass: No Annotations: <none> Provisioner: kubernetes.io/aws-ebs Parameters: type=io1,zone=us-west-2b No events. Actual results: pvc will not be created. In "oc describe pvc pvcname" will be visible reason why is that, via web interface no reason will be given. Expected results: When user specify PVC smaller than 4 GB for storage class which uses as type : io1 - to prevent creation of such PVC and to print error message Additional info: # oc describe pvc 2pvctest Name: 2pvctest Namespace: cnscluster StorageClass: ec2class Status: Pending Volume: Labels: <none> Capacity: Access Modes: Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 10s 3s 2 {persistentvolume-controller } Warning ProvisioningFailed Failed to provision volume with StorageClass "ec2class": InvalidParameterValue: The volume size is invalid for io1 volumes: 1 GiB. io1 volumes must be at least 4 GiB in size. Please specify a volume size above the minimum limit. status code: 400, request id: Description of problem: Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info:
The storage layer is behaving properly here. There is a limit of 4GB for EBS volumes, so the provisioning fails and sets the error message on the PVC. The error is in the web UI which doesn't properly handle the failed provisioning. Moving from storage to website.
Does anyone have a running instance on AWS that has this setup that I can connect to and use. Otherwise are there instructions on getting the console to work from an AWS instance. Thanks.
1. First, per Brad Childs, the backend is properly not binding due to the size requirement. This translates to the user as stuck in pending state with a warning in the event list on the PVC. 2. When a user creates the PVC, it is a valid claim and there are no errors that we in the console can or should show. 3. The only way for a user to know what is going on is to do a describe on the cmd line of the pvc. Solution: Add the event data from the describe of the PVC on the PVC detail page which is not a bug fix as much as it is a feature request. PR will be open shortly.
PR 1587 has been open to address this issue. https://github.com/openshift/origin-web-console/pull/1587 Note that this issue is not a bug as much as a poor user experience. As such, the solution simply gives more information to the user. The PVC will still not bind in this situation since it is not of the required size, but at least a user can now see why it is not working.
After configuring storage class and dynamic provisioning to use aws-ebs provisioner and io1 device type, and try to create PVC smaller than 4 GB, will show the events information section on the specific PVC detail page and still remains "Pending state". Version OpenShift Master: v3.6.86 Kubernetes Master: v1.6.1+5115d708d7
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1716