Bug 1944268

The ap-northeast-3 endpoint was added to the ASK SDK in v1.37.24 [1]. The 4.6 installer is using v1.32.3.

As a workaround, the user could add the endpoints manually to the install config.

[1] https://github.com/aws/aws-sdk-go/commit/6a71e1594856a4350bed5c29ba63724b66372591

Manually defining AWS service endpoints for the ap-northeast-3 region doesn't appear to work.

Excerpt from install-config.yaml:

platform:
  aws:
    amiID: ami-0310bc3d6eec49e56
    region: ap-northeast-3
    serviceEndpoints:
    - name: ec2
      url: https://ec2.ap-northeast-3.amazonaws.com
    - name: elasticloadbalancing
      url: https://elasticloadbalancing.ap-northeast-3.amazonaws.com
    - name: iam
      url: https://iam.amazonaws.com
    - name: route53
      url: https://route53.amazonaws.com
    - name: s3
      url: https://s3.ap-northeast-3.amazonaws.com
    - name: sts
      url: https://sts.ap-northeast-3.amazonaws.com
    - name: tagging
      url: https://tagging.ap-northeast-3.amazonaws.com


% ./openshift-install create cluster --dir katherine
INFO Consuming Install Config from target directory
INFO Credentials loaded from the "openshift-dev" profile in file "/Users/katherine/.aws/credentials"
WARNING Failed to find information on quotas ec2/L-0263D0A3, ec2/L-1216C47A
INFO Creating infrastructure resources...
ERROR
ERROR Error: Error creating IAM Role katherine-2kjw8-bootstrap-role: SignatureDoesNotMatch: Credential should be scoped to a valid region, not 'ap-northeast-3'.
ERROR 	status code: 403, request id: 862e5492-15b9-4fa5-926a-c31518a9984a
ERROR
ERROR   on ../../../../private/var/folders/2m/t4ltl17174s0x9v935kr59v40000gn/T/openshift-install-702836295/bootstrap/main.tf line 51, in resource "aws_iam_role" "bootstrap":
ERROR   51: resource "aws_iam_role" "bootstrap" {
ERROR
ERROR
ERROR
ERROR Error: Error creating IAM Role katherine-2kjw8-worker-role: SignatureDoesNotMatch: Credential should be scoped to a valid region, not 'ap-northeast-3'.
ERROR 	status code: 403, request id: 8b4f0f0b-8299-4297-bf9b-c5f4f180dacb
ERROR
ERROR   on ../../../../private/var/folders/2m/t4ltl17174s0x9v935kr59v40000gn/T/openshift-install-702836295/iam/main.tf line 13, in resource "aws_iam_role" "worker_role":
ERROR   13: resource "aws_iam_role" "worker_role" {
ERROR
ERROR
ERROR
ERROR Error: Error creating IAM Role katherine-2kjw8-master-role: SignatureDoesNotMatch: Credential should be scoped to a valid region, not 'ap-northeast-3'.
ERROR 	status code: 403, request id: 6fd96bfa-636a-4df9-bd25-22e0384238ab
ERROR
ERROR   on ../../../../private/var/folders/2m/t4ltl17174s0x9v935kr59v40000gn/T/openshift-install-702836295/master/main.tf line 17, in resource "aws_iam_role" "master_role":
ERROR   17: resource "aws_iam_role" "master_role" {
ERROR
ERROR
FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change

Additionally, if I omit the service endpoints for IAM and Route 53 (since they're not region specific), then the installer throws an error:

% ./openshift-install create cluster --dir katherine
FATAL failed to fetch Metadata: failed to load asset "Install Config": platform.aws.serviceEndpoints: Invalid value: []aws.ServiceEndpoint{aws.ServiceEndpoint{Name:"ec2", URL:"https://ec2.ap-northeast-3.amazonaws.com"}, aws.ServiceEndpoint{Name:"elasticloadbalancing", URL:"https://elasticloadbalancing.ap-northeast-3.amazonaws.com"}, aws.ServiceEndpoint{Name:"s3", URL:"https://s3.ap-northeast-3.amazonaws.com"}, aws.ServiceEndpoint{Name:"sts", URL:"https://sts.ap-northeast-3.amazonaws.com"}, aws.ServiceEndpoint{Name:"tagging", URL:"https://tagging.ap-northeast-3.amazonaws.com"}}: [failed to find endpoint for service "iam": UnknownEndpointError: could not resolve endpoint
	partition: "all partitions", service: "iam", region: "ap-northeast-3", failed to find endpoint for service "route53": UnknownEndpointError: could not resolve endpoint
	partition: "all partitions", service: "route53", region: "ap-northeast-3"]

The installer does not need to be so restrictive when validation the service endpoints. The installer should accept a region [1] that matches the regex for a partition, even if the SDK does not know the region.

[1] https://github.com/openshift/installer/blob/6363f3ab700e3976e8655ba0e826843593c7c98f/pkg/asset/installconfig/aws/validation.go#L255-L264

verified. FAILED.

OCP version: 4.8.0-0.nightly-2021-04-29-222100

only the master nodes were created but no workers:

Status:
  Conditions:
    Last Transition Time:  2021-04-30T07:55:54Z
    Message:               Failed to check if machine exists: yunjiang-ap3-g5s68-worker-ap-northeast-3c-9lpgn: failed to create scope for machine: failed to create aws client: region "ap-northeast-3" not resolved: UnknownEndpointError: could not resolve endpoint
                           partition: "all partitions", service: "ec2", region: "ap-northeast-3"
    Reason:                ErrorCheckingProvider
    Status:                Unknown
    Type:                  InstanceExists
  Last Updated:            2021-04-30T07:55:54Z
  Phase:

This BZ just addresses installer support for the region. The in-cluster operators that failed when using the new region have been fixed in separate BZs. If there are other operators that fail, we should create additional BZs for those.

verified. PASS.
OCP version: 4.8.0-0.nightly-2021-06-08-005718

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438