Description of problem: Dynamically provisioned storage does not work. In freshly installed cluster, when I create a PVC of storage class "standard", and use it in a pod, I expect a volume to be provisioned in OpenStack but it stays in unbound state forver. Version-Release number of the following components: $ openshift-install version openshift-install v4.2.0-201908282219-dirty built from commit 4f3e73a0143ba36229f42e8b65b6e65342bb826b release image registry.svc.ci.openshift.org/ocp/release@sha256:7319aa4ba33821a78710b27f9f4bbef68daee41ab3e9a7286976f0d2e050a85b How reproducible: Steps to Reproduce: 1. Install new OCP cluster on OpenStack 13 2. oc new-project testpvc 3. oc apply -f pvc.yaml 4. oc apply -f pod.yaml content of pvc.yaml: kind: PersistentVolumeClaim apiVersion: v1 metadata: name: cinder-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: standard content of pod.yaml: kind: Pod metadata: name: example labels: app: hello-openshift spec: volumes: - name: cinder-claim persistentVolumeClaim: claimName: cinder-claim containers: - name: hello-openshift image: openshift/hello-openshift ports: - containerPort: 8080 volumeMounts: - mountPath: "/usr/share/nginx/html" name: cinder-claim Actual results: $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE cinder-claim Pending standard 2m25s $ oc get pod NAME READY STATUS RESTARTS AGE example 0/1 Pending 0 2m20s $ oc get ev -n testpvc LAST SEEN TYPE REASON OBJECT MESSAGE 2m31s Normal WaitForFirstConsumer persistentvolumeclaim/cinder-claim waiting for first consumer to be created before binding Expected results: A new volume is provisioned in OpenStack, a PV is created in OpenShift and PVC bounds to this PV. Additional info:
Hi! I can't reproduce the bug on MOC: http://paste.openstack.org/show/l3tvMtlEZjlVtitUjnVL/ Can you provide any additional information about that can help me to investigate the issue?
Tested the issue on Upshift cloud and couldn't reproduce it as well: http://paste.openstack.org/show/ITkURf5eqPmjMRXHCizB/
@ppitonak Can you tell me exactly what kind of configuration you're having trouble with?
I'm installing cluster on internal Red Hat infra - UpShift/PSI. I usually try latest green 4.2.0-0.nightly build. Firstly I allocate a floating IP and set lbFloatingIP in install-config, I'm using this install-config: apiVersion: v1 baseDomain: crt.codereadyqe.com compute: - hyperthreading: Enabled name: worker platform: openstack: type: ci.m1.xlarge replicas: 3 controlPlane: hyperthreading: Enabled name: master platform: {} replicas: 3 metadata: creationTimestamp: null name: tmp networking: clusterNetwork: - cidr: 10.128.0.0/14 hostPrefix: 23 machineCIDR: 192.168.0.0/16 networkType: OpenShiftSDN serviceNetwork: - 172.30.0.0/16 platform: openstack: cloud: openstack computeFlavor: ci.m1.xlarge externalNetwork: provider_net_cci_10 lbFloatingIP: "xxx" octaviaSupport: "0" region: regionOne trunkSupport: "1" pullSecret: xxx sshKey: | ssh-rsa xxx Then I run following commands so that I can use nightly builds $ export OPENSHIFT_INSTALL_OS_IMAGE_OVERRIDE=rhcos-42.80.20190827.1 $ openshift-install --dir=$CLUSTER_NAME --log-level debug create manifests $ sed -i "s/api.openshift.com\/api\/upgrades_info\/v1\/graph/openshift-release.svc.ci.openshift.org\/graph/" $CLUSTER_NAME/manifests/cvo-overrides.yaml Then I configure our domain in Route53, I just add an A record for cluster's API $ aws route53 change-resource-record-sets --hosted-zone-id $ZONE_ID --change-batch '{ "Comment": "Update A record for cluster API", "Changes": [ { "Action": "CREATE", "ResourceRecordSet": { "Name": "api.'$CLUSTER_NAME'.'$DOMAIN'", "Type": "A", "TTL": 60, "ResourceRecords": [ { "Value": "'$FIP'" } ] } } ] }' Then install cluster by running $ openshift-install --log-level debug create cluster --dir=$CLUSTER_NAME After cluster is installed, I allocate a new floating IP $ openstack floating ip create --description "$CLUSTER_NAME-ingress" -f value -c floating_ip_address --port $INGRESS_PORT $NETWORK setup DNS records in Route53 $ aws route53 change-resource-record-sets --hosted-zone-id $ZONE_ID --change-batch '{ "Comment": "Update A record for cluster API", "Changes": [ { "Action": "CREATE", "ResourceRecordSet": { "Name": "*.apps.'$CLUSTER_NAME'.'$DOMAIN'", "Type": "A", "TTL": 60, "ResourceRecords": [ { "Value": "'$FIP'" } ] } } ] }' and follow instructions in bug description I did try the installation * with a different account * with a different flavor, external network, with private domain registered not on AWS Route53 * with latest build of installer (from source) * with 4.2.0-ci build of installer * with password in clouds.yaml or ~/.config/openstack/secure.yaml
I simply can't reproduce it... The only one thing I noticed is that your pod.yaml is incorrect and requires "apiVersion: v1" This are my results on MOC and Upshift: $ oc new-project testpvc Now using project "testpvc" on server "https://api.mfedosin.shiftstack.com:6443". You can add applications to this project with the 'new-app' command. For example, try: oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git to build a new example application in Ruby. $ oc apply -f pvc-pp.yaml persistentvolumeclaim/cinder-claim created $ oc apply -f pod-pp.yaml pod/example created $ oc get pod NAME READY STATUS RESTARTS AGE example 1/1 Running 0 93s My install-config.yaml for Upshift: apiVersion: v1 baseDomain: shiftstack.com clusterID: 3f3030b8-7f91-4ff3-9015-7e10147bd6e2 compute: - hyperthreading: Enabled name: worker platform: openstack: type: ocp-master replicas: 3 controlPlane: hyperthreading: Enabled name: master platform: {} replicas: 3 metadata: name: mfedosin networking: clusterNetworks: - cidr: 10.128.0.0/14 hostSubnetLength: 9 serviceCIDR: 172.30.0.0/16 machineCIDR: 10.0.128.0/17 type: OpenshiftSDN platform: openstack: cloud: upshift externalNetwork: provider_net_shared_2 region: regionOne computeFlavor: ocp-master lbFloatingIP: 10.0.79.139 pullSecret: | {"auths": { "quay.io": { "auth": "xxx", "email": "" }}} sshKey: | ssh-rsa xxx
The only thing I can suggest is that you use an outdated RHCOS image, because we had a similar bug, but it was fixed about a month ago.
Verified on 4.2.0-0.nightly-2019-09-11-032939 # oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE cinder-claim Bound pvc-13f25765-d46f-11e9-8730-fa163e0445b6 1Gi RWO standard 16m # oc get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-13f25765-d46f-11e9-8730-fa163e0445b6 1Gi RWO Delete Bound testpvc/cinder-claim standard 115s # oc get pods NAME READY STATUS RESTARTS AGE example 1/1 Running 0 119s
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922