Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1866723

Summary: [aws-custom-region] installer does not return a validation error if provide `unknow` region but do not provide service endpoints
Product: OpenShift Container Platform Reporter: Yunfei Jiang <yunjiang>
Component: InstallerAssignee: aos-install
Installer sub component: openshift-installer QA Contact: Yunfei Jiang <yunjiang>
Status: CLOSED NOTABUG Docs Contact:
Severity: high    
Priority: high CC: adahiya, wking
Version: 4.6   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-08-18 23:19:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yunfei Jiang 2020-08-06 08:25:42 UTC
according to document https://github.com/openshift/enhancements/blob/master/enhancements/installer/aws-custom-region-and-endpoints.md#custom-regions :
 
 af-south-1 shoudl be an `unknown` region, and
 >> If the user provides unknown region, and doesn't provide service endpoints for the services mentioned above the installer returns a validation error.
 
 try to install a cluster into af-south-1, but do not provide service endpoints, installer does not report any validation error, and starts to create resources.

install logs:
time="2020-05-25T05:43:20-04:00" level=info msg="Credentials loaded from the \"default\" profile in file \"/root/.aws/credentials\""
time="2020-08-05T23:23:21-04:00" level=info msg="Consuming Install Config from target directory"
time="2020-08-05T23:23:22-04:00" level=info msg="Credentials loaded from the \"default\" profile in file \"/root/.aws/credentials\""
time="2020-08-05T23:23:29-04:00" level=info msg="Creating infrastructure resources..."
time="2020-08-05T23:31:56-04:00" level=info msg="Waiting up to 20m0s for the Kubernetes API at https://api.yunjiang-afnoep.qe.devcluster.openshift.com:6443..."
time="2020-08-05T23:34:04-04:00" level=info msg="API v4.6.0-202008031851.p0-dirty up"
time="2020-08-05T23:34:04-04:00" level=info msg="Waiting up to 30m0s for bootstrapping to complete..."
time="2020-08-05T23:47:19-04:00" level=info msg="Destroying the bootstrap resources..."
time="2020-08-05T23:49:18-04:00" level=info msg="Waiting up to 30m0s for the cluster at https://api.yunjiang-afnoep.qe.devcluster.openshift.com:6443 to initialize..."


Version-Release number of the following components:
4.6.0-0.nightly-2020-08-04-210224

How reproducible:
100%

Steps to Reproduce:
1. Create install config
2. config region:
  aws:
    region: af-south-1
3. create cluster

Actual results:
Start to create infrastructure resources

Expected results:
Installer reports a validation error and exit

Additional info:

Comment 1 Abhinav Dahiya 2020-08-06 18:19:35 UTC
> af-south-1 shoudl be an `unknown` region, and


af-south-1 is not a region that requires user to provide the endpoints, because we already have the endpoints for that region in the embedded SDK.
So any user should be able to use that region using the install-config.yaml without also providing the endpoints.
Users cannot pick that region from TUI because we do not have an AMI for the region.

I do not think there is a bug here.

Comment 2 W. Trevor King 2020-08-06 23:53:12 UTC
The region is new in April [1].  Should this be a ticket to RHCOS about getting the AMI copied into the new region?  Prior art for eu-north-1 in bug 1765269.

[1]: https://aws.amazon.com/blogs/aws/now-open-aws-africa-cape-town-region/

Comment 3 Yunfei Jiang 2020-08-10 14:54:16 UTC
Hello Abhinav,

1. According to definition in the document https://github.com/openshift/enhancements/blob/master/enhancements/installer/aws-custom-region-and-endpoints.md#custom-regions 

>> The installer although should keep track if the specified region is known. A region is known when
>>    There is RHEL CoreOS AMI for the region known to the installer binary.
>>    The regions is one of the known regions to the AWS SDK vendored into the binary.`


So in my test cases, these three regions are custom/unknown region (af-south-1, ap-east-1, eu-south-1)

But from your description, af-south-1 is not an unknown region since it has been supported by SDK.

Need your confirm, af-south-1 will be a known region (after AMI uploaded), how about ap-east-1, eu-south-1? Will be the same as af-south-1 region as a known region?

2. the same issue was also occurred in region us-gov-west-1: openshift-installer does not report validation error and starts to create resources. 

INFO Consuming Install Config from target directory
INFO Credentials loaded from the "default" profile in file "/home/ec2-user/.aws/credentials"
INFO Creating infrastructure resources...

Comment 4 W. Trevor King 2020-08-10 21:36:03 UTC
$ git --no-pager log --oneline -1
514166ea6 (HEAD -> master, origin/release-4.7, origin/release-4.6, origin/master, origin/HEAD) Merge pull request #4026 from wking/try-openshift-com-moved
$ git --no-pager grep 'af-south-1\|ap-east-1\|eu-south-1'
pkg/types/aws/defaults/platform.go:             // "ap-east-1":      {"m5", "m4"},
vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go:     AfSouth1RegionID     = "af-south-1"     // Africa (Cape Town).
vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go:     ApEast1RegionID      = "ap-east-1"      // Asia Pacific (Hong Kong).
vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go:     EuSouth1RegionID     = "eu-south-1"     // Europe (Milan).
...
vendor/github.com/terraform-providers/terraform-provider-aws/aws/hosted_zones.go:       "af-south-1":     "Z11KHD8FBVPUYU",
vendor/github.com/terraform-providers/terraform-provider-aws/aws/hosted_zones.go:       "ap-east-1":      "ZNB98KWMFR0R6",
vendor/github.com/terraform-providers/terraform-provider-aws/aws/hosted_zones.go:       "eu-south-1":     "Z3IXVV8C73GIO3",

So looks pretty good to me once we get AMIs for the new regions.  You should be able to test by performing the cross-region AMI copy yourself and setting an explicit AMI via amiID [1].

[1]: https://github.com/openshift/installer/blob/master/docs/user/aws/customization.md#cluster-scoped-properties

Comment 5 Abhinav Dahiya 2020-08-11 17:01:19 UTC
> (af-south-1, ap-east-1, eu-south-1)

All three of the regions do not need any service endpoints to be set because they are already known by the SDK in use by installer.

As a general rule, the service endpoints are designed for 2 use cases:

- really private regions like C2S ans SC2S
- when users want to use alternate endpoints for region, like see here https://aws.amazon.com/compliance/fips/ for fips endpoints for GovCloud.
- when users are trying to use PrivateLink for suported services https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html


> Need your confirm, af-south-1 will be a known region (after AMI uploaded), how about ap-east-1, eu-south-1? Will be the same as af-south-1 region as a known region?

All reginos that have AMI published and updated in installer will show up in the Terminal Prompts. So I would say yes.

Comment 6 Yunfei Jiang 2020-08-12 02:30:06 UTC
Abhinav,

thanks for your confirm.

As my note in https://bugzilla.redhat.com/show_bug.cgi?id=1866723#c3

> 2. the same issue was also occurred in region us-gov-west-1: openshift-installer does not report validation error and starts to create resources. 
> 
> INFO Consuming Install Config from target directory
> INFO Credentials loaded from the "default" profile in file "/home/ec2-user/.aws/credentials"
> INFO Creating infrastructure resources...

I think it is a problem, and installer should provide validation capability.

Comment 7 Abhinav Dahiya 2020-08-12 17:39:52 UTC
(In reply to Yunfei Jiang from comment #6)
> Abhinav,
> 
> thanks for your confirm.
> 
> As my note in https://bugzilla.redhat.com/show_bug.cgi?id=1866723#c3
> 
> > 2. the same issue was also occurred in region us-gov-west-1: openshift-installer does not report validation error and starts to create resources. 
> > 
> > INFO Consuming Install Config from target directory
> > INFO Credentials loaded from the "default" profile in file "/home/ec2-user/.aws/credentials"
> > INFO Creating infrastructure resources...
> 
> I think it is a problem, and installer should provide validation capability.

^^ Can you explain a little more which exact case you would have seen the installer provide validation and what kind of validation.

Just to repeat from https://bugzilla.redhat.com/show_bug.cgi?id=1866723#c5

All regions including af-south-1, ap-east-1, eu-south-1, or us-gov-west-1, us-gov-east-1 would NOT need any service endpoints as the installer already has endpoints for these regions.

Comment 8 Abhinav Dahiya 2020-08-18 23:19:25 UTC
> af-south-1, ap-east-1, eu-south-1, or us-gov-west-1, us-gov-east-1

For all the regions mentioned above the installer does not need service endpoints and therefore validations are not required for these region.