Description of problem: After changing the IOPS size, for gp3 AWS EBS type defined in the MachineSet the new value it’s not configured properly within AWS EBS. Version-Release number of selected component (if applicable): Tested with OpenShift Container Platform 4.8.26 installed on top of AWS How reproducible: Always Steps to Reproduce: ===> 1) Get MachineSet List `MachineSet` and use a suitable one ~~~ $ oc get machineset -n openshift-machine-api NAME DESIRED CURRENT READY AVAILABLE AGE aws-ocp-worker-us-east-1a 1 1 1 1 3h30m aws-ocp-worker-us-east-1b 1 1 1 1 3h30m aws-ocp-worker-us-east-1c 1 1 1 1 3h30m aws-ocp-worker-us-east-1d 0 0 3h30m aws-ocp-worker-us-east-1e 0 0 3h30m aws-ocp-worker-us-east-1f 0 0 3h30m ~~~ ===> 2) Scale down Deleting the node to scale up after with new configurations ~~~ $ oc scale --replicas=0 machineset aws-ocp-worker-us-east-1a -n openshift-machine-api machineset.machine.openshift.io/aws-ocp-worker-us-east-1a scaled ~~~ ===> 3) Change AWS EBS configs The current config might have `gp2` settings adapt to `gp3` ~~~ $ oc edit machineset aws-ocp-worker-us-east-1a -n openshift-machine-api ~~~ ##### Old config ~~~ ... spec: metadata: {} providerSpec: value: ami: id: ami-06e6531aef9359b1e apiVersion: awsproviderconfig.openshift.io/v1beta1 blockDevices: - ebs: encrypted: true iops: 0 kmsKey: arn: "" volumeSize: 120 volumeType: gp2 ... ~~~ ##### New config ~~~ ... spec: metadata: {} providerSpec: value: ami: id: ami-06e6531aef9359b1e apiVersion: awsproviderconfig.openshift.io/v1beta1 blockDevices: - ebs: encrypted: true iops: 5000 kmsKey: arn: "" volumeSize: 120 volumeType: gp3 ... ~~~ Save and continue ===> 4) Scale up To create a new node at AWS and apply new configurations ~~~ $ oc scale --replicas=1 machineset aws-ocp-worker-us-east-1a -n openshift-machine-api machineset.machine.openshift.io/aws-ocp-worker-us-east-1a scaled ~~~ It's expected that the current output changes from: ~~~ $ oc get machineset -n openshift-machine-api NAME DESIRED CURRENT READY AVAILABLE AGE aws-ocp-worker-us-east-1a 0 0 3h32m aws-ocp-worker-us-east-1b 1 1 1 1 3h32m aws-ocp-worker-us-east-1c 1 1 1 1 3h32m aws-ocp-worker-us-east-1d 0 0 3h32m aws-ocp-worker-us-east-1e 0 0 3h32m aws-ocp-worker-us-east-1f 0 0 3h32m ~~~ To: ~~~ $ oc get machineset -n openshift-machine-api NAME DESIRED CURRENT READY AVAILABLE AGE aws-ocp-worker-us-east-1a 1 1 1 1 3h35m aws-ocp-worker-us-east-1b 1 1 1 1 3h35m aws-ocp-worker-us-east-1c 1 1 1 1 3h35m aws-ocp-worker-us-east-1d 0 0 3h35m aws-ocp-worker-us-east-1e 0 0 3h35m aws-ocp-worker-us-east-1f 0 0 3h35m ~~~ ===> 5) Confirm via AWS console It's possible to confirm that configured value, `5000`, was not applied. Actual results: New node it's created with an EBS, `gp3`, configured with `3000` IOPS. Expected results: New node created with an EBS, `gp3`, configured with `5000` IOPS.
Currently we only set the IOPS for `io1` block devices (https://github.com/openshift/machine-api-provider-aws/blob/main/pkg/actuators/machine/instances.go#L255). Based on the documentation (https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/create-volume.html) it seems we should also be including `io2` and `gp3` disks in this check. Should be a relatively straight forward fix.
Verified on 4.10.0-0.nightly-2022-01-17-223655 Steps: 1.Get MachineSet liuhuali@Lius-MacBook-Pro huali-test % oc get machineset NAME DESIRED CURRENT READY AVAILABLE AGE huliu-035-xlb2l-worker-us-east-2a 1 1 1 1 3h7m huliu-035-xlb2l-worker-us-east-2b 1 1 1 1 3h7m huliu-035-xlb2l-worker-us-east-2c 1 1 1 1 3h7m 2.Scale down liuhuali@Lius-MacBook-Pro huali-test % oc scale --replicas=0 machineset huliu-035-xlb2l-worker-us-east-2a machineset.machine.openshift.io/huliu-035-xlb2l-worker-us-east-2a scaled 3.Change AWS EBS configs liuhuali@Lius-MacBook-Pro huali-test % oc edit machineset huliu-035-xlb2l-worker-us-east-2a machineset.machine.openshift.io/huliu-035-xlb2l-worker-us-east-2a edited ##### Old config ~~~ ... spec: metadata: {} providerSpec: value: ami: id: ami-0c5b210f3fd086930 apiVersion: awsproviderconfig.openshift.io/v1beta1 blockDevices: - ebs: encrypted: true iops: 0 kmsKey: arn: "" volumeSize: 120 volumeType: gp3 ... ~~~ ##### New config ~~~ ... spec: metadata: {} providerSpec: value: ami: id: ami-0c5b210f3fd086930 apiVersion: awsproviderconfig.openshift.io/v1beta1 blockDevices: - ebs: encrypted: true iops: 5000 kmsKey: arn: "" volumeSize: 120 volumeType: gp3 ... ~~~ 4.Scale up liuhuali@Lius-MacBook-Pro huali-test % oc scale --replicas=1 machineset huliu-035-xlb2l-worker-us-east-2a machineset.machine.openshift.io/huliu-035-xlb2l-worker-us-east-2a scaled liuhuali@Lius-MacBook-Pro huali-test % oc get machineset NAME DESIRED CURRENT READY AVAILABLE AGE huliu-035-xlb2l-worker-us-east-2a 1 1 1 1 3h25m huliu-035-xlb2l-worker-us-east-2b 1 1 1 1 3h25m huliu-035-xlb2l-worker-us-east-2c 1 1 1 1 3h25m liuhuali@Lius-MacBook-Pro huali-test % 5.Confirm via AWS console New node created with an EBS, `gp3`, configured with `5000` IOPS.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056