This is clone of bug #1737897 Description of problem: With AWS EBS CSI driver installed, both in-tree and CSI provision will provision volumes for stroage class with provisioner "kubernetes.io/aws-ebs". It's caused by CSI migration turned on by default in the external CSI provisioner. It affects only CSI drivers that have migration implemented, i.e. AWS EBS, CSI PD and OpenStack, other CSI drivers are OK. Since we don't ship nor support CSI drivers for these clouds, it's not super serious error, but it should be fixed. Version-Release number of selected component (if applicable): (reported by the provisioner): v4.2.0-201909081401-0-g75088ae-dirty How reproducible: always Steps to Reproduce: 1. install a cluster on AWS 2. check the default storage class uses provisioner: kubernetes.io/aws-ebs 3. install AWS CSI driver 4. create a PVC (using the default storage class) Actual results: AWS EBS CSI driver provisions the volume: $ oc logs -n kube-system <provisioner pod> <provisioner container> I0911 08:00:12.787394 1 controller.go:354] translating storage class parameters for in-tree plugin kubernetes.io/aws-ebs to CSI I0911 08:00:12.802651 1 controller.go:451] CreateVolumeRequest {Name:pvc-1cea5684-d46a-11e9-904d-020ff73b0330 CapacityRange:required_bytes:1073741824 VolumeCapabilities:[mount:<fs_type:"ext4" > access_mode:<mode:SINGLE_NODE_WRITER > ] Parameters:map[encrypted:true type:gp2] Secrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:requisite:<segments:<key:"topology.ebs.csi.aws.com/zone" value:"us-east-2a" > > requisite:<segments:<key:"topology.ebs.csi.aws.com/zone" value:"us-east-2b" > > preferred:<segments:<key:"topology.ebs.csi.aws.com/zone" value:"us-east-2a" > > preferred:<segments:<key:"topology.ebs.csi.aws.com/zone" value:"us-east-2b" > > XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} I0911 08:00:12.802831 1 connection.go:180] GRPC call: /csi.v1.Controller/CreateVolume I0911 08:00:12.802843 1 connection.go:181] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.ebs.csi.aws.com/zone":"us-east-2a"}},{"segments":{"topology.ebs.csi.aws.com/zone":"us-east-2b"}}],"requisite":[{"segments":{"topology.ebs.csi.aws.com/zone":"us-east-2a"}},{"segments":{"topology.ebs.csi.aws.com/zone":"us-east-2b"}}]},"capacity_range":{"required_bytes":1073741824},"name":"pvc-1cea5684-d46a-11e9-904d-020ff73b0330","parameters":{"encrypted":"true","type":"gp2"},"volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]} I0911 08:00:19.369130 1 connection.go:183] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.ebs.csi.aws.com/zone":"us-east-2a"}}],"capacity_bytes":1073741824,"volume_id":"vol-025d1c353748c897c"}} I0911 08:00:19.372560 1 connection.go:184] GRPC error: <nil> I0911 08:00:19.372578 1 controller.go:498] create volume rep: {CapacityBytes:1073741824 VolumeId:vol-025d1c353748c897c VolumeContext:map[] ContentSource:<nil> AccessibleTopology:[segments:<key:"topology.ebs.csi.aws.com/zone" value:"us-east-2a" > ] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} I0911 08:00:19.372709 1 controller.go:573] successfully created PV {GCEPersistentDisk:nil AWSElasticBlockStore:&AWSElasticBlockStoreVolumeSource{VolumeID:vol-025d1c353748c897c,FSType:ext4,Partition:0,ReadOnly:false,} HostPath:nil Glusterfs:nil NFS:nil RBD:nil ISCSI:nil Cinder:nil CephFS:nil FC:nil Flocker:nil FlexVolume:nil AzureFile:nil VsphereVolume:nil Quobyte:nil AzureDisk:nil PhotonPersistentDisk:nil PortworxVolume:nil ScaleIO:nil Local:nil StorageOS:nil CSI:nil} I0911 08:00:19.372862 1 controller.go:1306] provision "kube-system/pvc5" class "gp2": volume "pvc-1cea5684-d46a-11e9-904d-020ff73b0330" provisioned I0911 08:00:19.372891 1 controller.go:1323] provision "kube-system/pvc5" class "gp2": succeeded I0911 08:00:19.372900 1 volume_store.go:154] Saving volume pvc-1cea5684-d46a-11e9-904d-020ff73b0330 I0911 08:00:19.418227 1 volume_store.go:157] Volume pvc-1cea5684-d46a-11e9-904d-020ff73b0330 saved Expected results: Only in-tree volume plugin provisions the volume.
It is failed on 4.2.0-0.nightly-2019-09-15-052022 There is some error message need fix here. 1.Create pvc using default sc gp2 2.Create a pod I0917 05:55:47.851186 1 csi-provisioner.go:177] Supports migration from in-tree plugin: kubernetes.io/aws-ebs I0917 05:55:47.852005 1 controller.go:680] Using saving PVs to API server in background I0917 05:55:47.852035 1 controller.go:770] Starting provisioner controller ebs.csi.aws.com_ip-10-0-152-107_cbcde704-d90f-11e9-87b2-068e32ee4a96! I0917 05:55:47.852125 1 volume_store.go:97] Starting save volume queue I0917 05:55:47.852361 1 reflector.go:123] Starting reflector *v1.PersistentVolumeClaim (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/controller/controller.go:801 I0917 05:55:47.852383 1 reflector.go:161] Listing and watching *v1.PersistentVolumeClaim from sigs.k8s.io/sig-storage-lib-external-provisioner/controller/controller.go:801 I0917 05:55:47.852766 1 reflector.go:123] Starting reflector *v1.PersistentVolume (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/controller/controller.go:804 I0917 05:55:47.852781 1 reflector.go:161] Listing and watching *v1.PersistentVolume from sigs.k8s.io/sig-storage-lib-external-provisioner/controller/controller.go:804 I0917 05:55:47.853364 1 reflector.go:123] Starting reflector *v1.StorageClass (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/controller/controller.go:807 I0917 05:55:47.853388 1 reflector.go:161] Listing and watching *v1.StorageClass from sigs.k8s.io/sig-storage-lib-external-provisioner/controller/controller.go:807 I0917 05:55:47.952945 1 shared_informer.go:123] caches populated I0917 05:55:47.953146 1 controller.go:819] Started provisioner controller ebs.csi.aws.com_ip-10-0-152-107_cbcde704-d90f-11e9-87b2-068e32ee4a96! I0917 05:59:57.213902 1 controller.go:1199] provision "kube-system/pvc5" class "gp2": started I0917 05:59:57.222165 1 controller.go:1298] provision "kube-system/pvc5" class "gp2": volume provision ignored: ignored because PVC annotated with external-provisioner name kubernetes.io/aws-ebs does not match provisioner driver name ebs.csi.aws.com. This could mean the PVC is not migrated I0917 05:59:57.222208 1 controller.go:1014] Claim processing succeeded, removing PVC 57c27a28-d910-11e9-87f7-06c7d3dd1cd8 from claims in progress I0917 05:59:57.222263 1 event.go:209] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"kube-system", Name:"pvc5", UID:"57c27a28-d910-11e9-87f7-06c7d3dd1cd8", APIVersion:"v1", ResourceVersion:"58584", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "kube-system/pvc5" oc describe pvc pvc5 Name: pvc5 Namespace: kube-system StorageClass: gp2 Status: Bound Volume: pvc-57c27a28-d910-11e9-87f7-06c7d3dd1cd8 Labels: <none> Annotations: pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs volume.kubernetes.io/selected-node: ip-10-0-138-245.us-east-2.compute.internal Finalizers: [kubernetes.io/pvc-protection] Capacity: 1Gi Access Modes: RWO VolumeMode: Filesystem Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal WaitForFirstConsumer 3m33s (x2 over 3m36s) persistentvolume-controller waiting for first consumer to be created before binding Normal Provisioning 3m21s ebs.csi.aws.com_ip-10-0-152-107_cbcde704-d90f-11e9-87b2-068e32ee4a96 External provisioner is provisioning volume for claim "kube-system/pvc5" Normal ProvisioningSucceeded 3m21s persistentvolume-controller Successfully provisioned volume pvc-57c27a28-d910-11e9-87f7-06c7d3dd1cd8 using kubernetes.io/aws-ebs
Update the bug status to "Verified".
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922