Bug 1715635 - [DOCS] AWS UPI documentation does not instruct on creating the *.apps DNS record
Summary: [DOCS] AWS UPI documentation does not instruct on creating the *.apps DNS record
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Documentation
Version: 4.1.z
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.2.z
Assignee: Kathryn Alexander
QA Contact: Johnny Liu
Vikram Goyal
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-30 20:46 UTC by Matthew Staebler
Modified: 2020-01-16 10:07 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-22 16:05:06 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift installer pull 2221 'None' closed Bug 1715635: docs/user/aws/install_upi: Document bring-your-own-DNS 2020-08-17 18:56:11 UTC
Github openshift openshift-docs pull 17043 'None' closed Add GCP UPI install docs 2020-08-17 18:56:11 UTC
Github openshift openshift-docs pull 17190 'None' closed bug 1743483 adding DNS record data 2020-08-17 18:56:11 UTC

Description Matthew Staebler 2019-05-30 20:46:54 UTC
Document URL: https://docs.openshift.com/container-platform/4.1/installing/installing_aws_user_infra/installing-aws-user-infra.html

Section Number and Name: 

Describe the issue: The *.apps DNS record is required for installation to complete as it is needed for the authentication and console operators.

Suggestions for improvement: 

Additional information:

Comment 2 Johnny Liu 2019-05-31 01:33:26 UTC
As far as I know, *.apps DNS would be provisioned by ingress router, so I do not think this is an issue.

Comment 3 Matthew Staebler 2019-05-31 14:14:44 UTC
The ingress router is not creating any *.apps records. So maybe this is a bug with the ingress router then instead of with the docs.

Comment 4 Chris Callegari 2019-05-31 17:34:35 UTC
2019-05-31T17:29:03.729Z	INFO	operator.controller	controller/controller.go:101	reconciling	{"request": "openshift-ingress-operator/default"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a7cfb41276cf111e987000e616f5cccb", "dns name": "a7cfb41276cf111e987000e616f5cccb-1923067711.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "aa6d3339575a911e99a080a416d87bfe", "dns name": "aa6d3339575a911e99a080a416d87bfe-1287501337.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a4be8a3a2767811e99b9602fb71b1e13", "dns name": "a4be8a3a2767811e99b9602fb71b1e13-242697147.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "ad1febd93767e11e99a080a416d87bfe", "dns name": "ad1febd93767e11e99a080a416d87bfe-2084259110.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a98e7eafe78c411e9a1db02fb71b1e13", "dns name": "a98e7eafe78c411e9a1db02fb71b1e13-826423417.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a40b997b47b3911e9b4ce0eb27b1ef28", "dns name": "a40b997b47b3911e9b4ce0eb27b1ef28-1634745472.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a442d30927cd211e99062126c29a244b", "dns name": "a442d30927cd211e99062126c29a244b-1310351826.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.966Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a8a619a6f7d7311e9bd6212235e364a8", "dns name": "a8a619a6f7d7311e9bd6212235e364a8-277882918.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.967Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a50fdf6bd816511e9ad82022aedbe4da", "dns name": "a50fdf6bd816511e9ad82022aedbe4da-331156363.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.967Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a344e406d817711e9b0710e162b90e27", "dns name": "a344e406d817711e9b0710e162b90e27-306263612.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.967Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a59848b08821c11e99ffa0e877aca9fc", "dns name": "a59848b08821c11e99ffa0e877aca9fc-43722952.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.967Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "a09829fb5822111e99dbe02fb71b1e13", "dns name": "a09829fb5822111e99dbe02fb71b1e13-1299202005.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.967Z	INFO	operator.dns	aws/dns.go:199	found load balancer	{"name": "aff9afbe8830111e98d1a0e877aca9fc", "dns name": "internal-aff9afbe8830111e98d1a0e877aca9fc-1525245753.us-east-1.elb.amazonaws.com", "hosted zone ID": "Z35SXDOTRQ7X7K"}
2019-05-31T17:29:03.992Z	ERROR	operator.init.controller-runtime.controller	controller/controller.go:217	Reconciler error	{"controller": "operator-controller", "request": "openshift-ingress-operator/default", "error": "failed to ensure ingresscontroller: failed to ensure DNS for default: failed to ensure DNS record &{{ map[Name:upi-bqhv9-int kubernetes.io/cluster/upi-bqhv9:owned]} ALIAS *.apps.upi.aws.sysdeseng.com -> a13d5fb0683c411e9a0d902722c57c88-420581299.us-east-2.elb.amazonaws.com} for openshift-ingress-operator/default: failed to get hosted zone for load balancer target \"a13d5fb0683c411e9a0d902722c57c88-420581299.us-east-2.elb.amazonaws.com\": couldn't find hosted zone ID of ELB a13d5fb0683c411e9a0d902722c57c88-420581299.us-east-2.elb.amazonaws.com", "errorCauses": [{"error": "failed to ensure ingresscontroller: failed to ensure DNS for default: failed to ensure DNS record &{{ map[Name:upi-bqhv9-int kubernetes.io/cluster/upi-bqhv9:owned]} ALIAS *.apps.upi.aws.sysdeseng.com -> a13d5fb0683c411e9a0d902722c57c88-420581299.us-east-2.elb.amazonaws.com} for openshift-ingress-operator/default: failed to get hosted zone for load balancer target \"a13d5fb0683c411e9a0d902722c57c88-420581299.us-east-2.elb.amazonaws.com\": couldn't find hosted zone ID of ELB a13d5fb0683c411e9a0d902722c57c88-420581299.us-east-2.elb.amazonaws.com"}]}

Comment 5 Eric Paris 2019-05-31 18:43:58 UTC
We need a doc which describes how a user would determine the name of the ELB created by the router service and then tell the user to create CNAME records for that ELB. In UPI we have no promises that the ingress controller will have any ability to program route53. If it works, that's almost by accident   :)

I'm ok if the cloudformation template does leave things in a state where the ingress operator is able to finish the job, but this is NOT how most UPI customers should work.

Comment 6 Chris Callegari 2019-05-31 21:58:39 UTC
Now that it's clear what we need to do we can create that A/Alias in an aws cli command. No problem. I'll provide that cmd on Mon if someone else doesn't get to it first.

Comment 10 Chris Callegari 2019-07-25 13:15:11 UTC
Sorry everyone I lost track of this bugzilla.  I have a thorough command handy...

(awscli, jq from epel and yq from pip are required!)


SET ENV VARS
                          export CLUSTER_NAME=`cat metadata.json | jq -r .clusterName`
                          export HOSTED_ZONE_NAME=`cat install-config.yaml.bak | yq -r .baseDomain`
                          export INFRA_NAME=`cat metadata.json | jq -r .infraID`
                          export VPCID=`cat .stack-vpc.json | jq -r '.Stacks[0].Outputs[] | select(.OutputKey == "VpcId").OutputValue'`


CREATE APPS SecurityGroup
                          export sg_apps="`aws ec2 create-security-group \
                            --vpc-id ${VPCID} \
                            --group-name appsSg \
                            --description appsSg`"
                          printf "${sg_apps}" > .sg_apps
                          if [ "`aws ec2 describe-security-groups --query "SecurityGroups[? VpcId == '${VPCID}' && GroupName == 'appsSg'].IpPermissions" | jq '.[] | length'`" == 0 ]; then
                            aws ec2 authorize-security-group-ingress \
                              --group-id `echo ${sg_apps} | jq -r '.GroupId'` \
                              --ip-permissions '[
                                                  {"IpProtocol": "tcp", "FromPort": 80, "ToPort": 80, "IpRanges": [{"CidrIp": "0.0.0.0/0"}]},
                                                  {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0"}]}
                                                ]'
                          fi


CREATE APPS ELB
                          aws cloudformation describe-stacks --stack-name ${INFRA_NAME}-vpc > .stack-vpc.json
                          IFS=',' read -r -a subnet <<< `cat .stack-vpc.json | jq -r '.Stacks[0].Outputs[] | select(.OutputKey | contains("PrivateSubnetIds")).OutputValue'`
                          declare -p subnet
                          echo "Creating ELB..."
                          aws elb create-load-balancer \
                            --load-balancer-name apps-${INFRA_NAME} \
                            --listener \
                              Protocol=TCP,LoadBalancerPort=80,InstanceProtocol=TCP,InstancePort=80 \
                              Protocol=TCP,LoadBalancerPort=443,InstanceProtocol=TCP,InstancePort=443 \
                            --security-groups `cat .sg_apps | jq -r '.GroupId'` \
                            --scheme internet-facing \
                            --subnets ${subnet[0]} \
                            --tags Key=name,Value=apps-${INFRA_NAME} Key=kubernetes.io/service-name,Value=openshift-ingress/router-default Key=kubernetes.io/cluster/${CLUSTER_NAME},Value=owned
                          aws elb modify-load-balancer-attributes \
                            --load-balancer-name apps-${INFRA_NAME} \
                            --load-balancer-attributes '{
                                "CrossZoneLoadBalancing":{"Enabled":false},
                                "ConnectionDraining":{"Enabled":false}
                            }'
                          aws elb configure-health-check \
                            --load-balancer-name apps-${INFRA_NAME} \
                            --health-check Target=TCP:443,HealthyThreshold=2,Interval=5,Timeout=2,UnhealthyThreshold=2
                          for i in `seq 0 $(( ${#subnet[@]} - 1))`; do
                            if ((${#subnet[@]} == 1)); then
                              aws elb attach-load-balancer-to-subnets --load-balancer-name apps-${INFRA_NAME} --subnets ${subnet[0]}
                            elif ((${#subnet[@]} == 2)); then
                              aws elb attach-load-balancer-to-subnets --load-balancer-name apps-${INFRA_NAME} --subnets ${subnet[$((${i} % 2))]}
                            else
                              aws elb attach-load-balancer-to-subnets --load-balancer-name apps-${INFRA_NAME} --subnets ${subnet[${i}]}
                            fi
                          done
                          IFS=',' read -r -a instances <<< `aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" "Name=tag:aws:cloudformation:logical-id,Values=Worker0" --query "Reservations[? Instances[0].VpcId == '${VPCID}'] | [].Instances[0].InstanceId" --output text | sed -e "s#\s#,#g"`
                          declare -p instances
                          for i in `seq 0 $(( ${#instances[@]} - 1))`; do
                            aws elb register-instances-with-load-balancer --load-balancer-name apps-${INFRA_NAME} --instances ${instances[\${i}]}
                          done


CREATE APPS ROUTE53 Resource Record Set
                          echo "Discovering ELBs in VPC: ${VPCID}"
                          for i in 1 2 3 4 5; do
                            if `aws elb describe-load-balancers --load-balancer-names apps-${INFRA_NAME} > /dev/null 2>&1`; then
                              echo "Found ELB: apps-${INFRA_NAME} ..."
                              aws elb describe-load-balancers --load-balancer-names apps-${INFRA_NAME} > .elb-apps-${INFRA_NAME}
                              echo "Setting route53_apps_dns_alias_record.json: ResourceRecordSet.AliasTarget.DNSName value now"
                              cat route53_apps_dns_alias_record.json | \
                                jq ".Comment = \"${INFRA_NAME}\"" | \
                                jq ".Changes[0].ResourceRecordSet.Name = \"*.apps.${CLUSTER_NAME}.${HOSTED_ZONE_NAME}.\"" | \
                                jq ".Changes[0].ResourceRecordSet.AliasTarget.DNSName = \"dualstak.`cat .elb-apps-${INFRA_NAME} | jq -r '.LoadBalancerDescriptions[0].DNSName'`\"" | \
                                jq ".Changes[0].ResourceRecordSet.AliasTarget.HostedZoneId = \"`cat .elb-apps-${INFRA_NAME} | jq -r '.LoadBalancerDescriptions[0].CanonicalHostedZoneNameID'`\"" > \
                                .route53_apps_dns_alias_record.json && \
                                /bin/mv .route53_apps_dns_alias_record.json route53_apps_dns_alias_record.json
                                break
                            else
                              printf "*.apps ELB has not been created yet ... sleeping for 30 sec\n"
                              sleep 30
                            fi
                          done

                          aws route53 change-resource-record-sets --hosted-zone-id ${HOSTED_ZONE} --change-batch file://route53_apps_dns_alias_record.json

Comment 11 Chris Callegari 2019-07-25 13:26:18 UTC
Also worker nodes must be deployed before wait-for installation-complete command is run.  They are reqd to handle the ingress-router pods.

I have probably 2000 UPI deployments under my belt in the last 3 weeks using a Jenkins pipeline.  My workflow is as follows...

1) Verify inputs ... pull secrets, ssh keys, cidr's, etc
2) Prepare installer install-config, manifests, & ignition files
3) Prepare infrastructure vpc
4) Prepare infrastructure security group and roles
5) Prepare infrastructure network and load balancers
6) Prepare infrastructure S3 buckets
7) Prepare infrastructure bootstrap incl s3 put ign file
8) Prepare controlplane nodes
9) Prepare workers nodes
10) Prepare *.apps ELB and Route53 dns record
11) Run wait-for bootstrap-complete
12) Run wait-for installation-complete

Comment 14 W. Trevor King 2019-08-01 17:11:27 UTC
I just kicked off a round of our AWS UPI CI testing [1].  The procedure it uses is [2], and it does not do anything explicit around creating the *.apps record.  And here is the ingress controller creating it for us [3]:

  $ curl -s --compressed https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_installer/2123/pull-ci-openshift-installer-master-e2e-aws-upi/4/artifacts/e2e-aws-upi/must-gather/namespaces/openshift-ingress-operator/pods/ingress-operator-58f474b79-flmwm/ingress-operator/ingress-operator/logs/current.log | grep -1 '\*\.apps'
  2019-08-01T15:21:13.29444427Z 2019-08-01T15:21:13.294Z	INFO	operator.ingress_controller	ingress/controller.go:113	reconciling	{"request": "openshift-ingress-operator/default"}
  2019-08-01T15:21:13.475294314Z 2019-08-01T15:21:13.475Z	INFO	operator.ingress_controller	ingress/dns.go:35	created dnsrecord	{"dnsrecord": {"metadata":{"name":"default-wildcard","namespace":"openshift-ingress-operator","selfLink":"/apis/ingress.operator.openshift.io/v1/namespaces/openshift-ingress-operator/dnsrecords/default-wildcard","uid":"ff97e09c-b46f-11e9-b19a-0e7c537a4586","resourceVersion":"8802","generation":1,"creationTimestamp":"2019-08-01T15:21:13Z","labels":{"ingresscontroller.operator.openshift.io/owning-ingresscontroller":"default"},"ownerReferences":[{"apiVersion":"operator.openshift.io/v1","kind":"IngressController","name":"default","uid":"fcd2fe92-b46f-11e9-b19a-0e7c537a4586","controller":true,"blockOwnerDeletion":true}],"finalizers":["operator.openshift.io/ingress-dns"]},"spec":{"dnsName":"*.apps.ci-op-690h820f-9d51b.origin-ci-int-aws.dev.rhcloud.com.","targets":["afd5a8c49b46f11e9b19a0e7c537a458-111304965.us-east-1.elb.amazonaws.com"],"recordType":"CNAME"},"status":{}}}
  2019-08-01T15:21:13.618808325Z 2019-08-01T15:21:13.618Z	INFO	operator.dns	aws/dns.go:183	found hosted zone using tags	{"zone id": "Z2A83K5ILP3F12", "tags": {"Name":"ci-op-690h820f-9d51b-k9nr7-int","kubernetes.io/cluster/ci-op-690h820f-9d51b-k9nr7":"owned"}}
  2019-08-01T15:21:14.127951541Z 2019-08-01T15:21:14.127Z	INFO	operator.dns	aws/dns.go:322	updated DNS record	{"zone id": "Z2A83K5ILP3F12", "domain": "*.apps.ci-op-690h820f-9d51b.origin-ci-int-aws.dev.rhcloud.com.", "target": "afd5a8c49b46f11e9b19a0e7c537a458-111304965.us-east-1.elb.amazonaws.com", "response": "{\n  ChangeInfo: {\n    Id: \"/change/C3JBSPNG0D1ED1\",\n    Status: \"PENDING\",\n    SubmittedAt: 2019-08-01 15:21:14.342 +0000 UTC\n  }\n}"}
  2019-08-01T15:21:14.128113748Z 2019-08-01T15:21:14.128Z	INFO	operator.dns	aws/dns.go:281	upserted DNS record	{"record": {"metadata":{"name":"default-wildcard","namespace":"openshift-ingress-operator","selfLink":"/apis/ingress.operator.openshift.io/v1/namespaces/openshift-ingress-operator/dnsrecords/default-wildcard","uid":"ff97e09c-b46f-11e9-b19a-0e7c537a4586","resourceVersion":"8802","generation":1,"creationTimestamp":"2019-08-01T15:21:13Z","labels":{"ingresscontroller.operator.openshift.io/owning-ingresscontroller":"default"},"ownerReferences":[{"apiVersion":"operator.openshift.io/v1","kind":"IngressController","name":"default","uid":"fcd2fe92-b46f-11e9-b19a-0e7c537a4586","controller":true,"blockOwnerDeletion":true}],"finalizers":["operator.openshift.io/ingress-dns"]},"spec":{"dnsName":"*.apps.ci-op-690h820f-9d51b.origin-ci-int-aws.dev.rhcloud.com.","targets":["afd5a8c49b46f11e9b19a0e7c537a458-111304965.us-east-1.elb.amazonaws.com"],"recordType":"CNAME"},"status":{}}}
  2019-08-01T15:21:14.163673376Z 2019-08-01T15:21:14.163Z	INFO	operator.dns	aws/dns.go:322	updated DNS record	{"zone id": "Z2GYOLTZHS5VK", "domain": "*.apps.ci-op-690h820f-9d51b.origin-ci-int-aws.dev.rhcloud.com.", "target": "afd5a8c49b46f11e9b19a0e7c537a458-111304965.us-east-1.elb.amazonaws.com", "response": "{\n  ChangeInfo: {\n    Id: \"/change/C1MDSLUVY30OYI\",\n    Status: \"PENDING\",\n    SubmittedAt: 2019-08-01 15:21:14.377 +0000 UTC\n  }\n}"}
  2019-08-01T15:21:14.163705559Z 2019-08-01T15:21:14.163Z	INFO	operator.dns	aws/dns.go:281	upserted DNS record	{"record": {"metadata":{"name":"default-wildcard","namespace":"openshift-ingress-operator","selfLink":"/apis/ingress.operator.openshift.io/v1/namespaces/openshift-ingress-operator/dnsrecords/default-wildcard","uid":"ff97e09c-b46f-11e9-b19a-0e7c537a4586","resourceVersion":"8802","generation":1,"creationTimestamp":"2019-08-01T15:21:13Z","labels":{"ingresscontroller.operator.openshift.io/owning-ingresscontroller":"default"},"ownerReferences":[{"apiVersion":"operator.openshift.io/v1","kind":"IngressController","name":"default","uid":"fcd2fe92-b46f-11e9-b19a-0e7c537a4586","controller":true,"blockOwnerDeletion":true}],"finalizers":["operator.openshift.io/ingress-dns"]},"spec":{"dnsName":"*.apps.ci-op-690h820f-9d51b.origin-ci-int-aws.dev.rhcloud.com.","targets":["afd5a8c49b46f11e9b19a0e7c537a458-111304965.us-east-1.elb.amazonaws.com"],"recordType":"CNAME"},"status":{}}}
  2019-08-01T15:21:14.181285483Z 2019-08-01T15:21:14.181Z	DEBUG	operator.init.controller-runtime.controller	controller/controller.go:236	Successfully Reconciled	{"controller": "dns_controller", "request": "openshift-ingress-operator/default-wildcard"}

*.apps records are being created in both the public zone is Z2GYOLTZHS5VK [4] and the private zone Z2A83K5ILP3F12.  Of course, if you don't give the ingress operator sufficient credentials to create those records, you'll need to create them yourselves.  I'm fine mentioning that in the UPI docs and/or shifting our CI so that it tests a UPI flow where the cluster creds lack record-creation auth.  Do we have a set of permissions that we expect UPI clusters to have?  If we remove *all* resource-creation permissions, then we lose the ability to create LoadBalancer services [5], etc., etc.

[1]: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_installer/2123/pull-ci-openshift-installer-master-e2e-aws-upi/4
[2]: https://github.com/openshift/release/blob/92ea08af3806b213e1d6ae3f0ab58df943d5147f/ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml#L298-L680
[3]: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_installer/2123/pull-ci-openshift-installer-master-e2e-aws-upi/4/artifacts/e2e-aws-upi/must-gather/namespaces/openshift-ingress-operator/pods/ingress-operator-58f474b79-flmwm/ingress-operator/ingress-operator/logs/current.log
[4]: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_installer/2123/pull-ci-openshift-installer-master-e2e-aws-upi/4/artifacts/e2e-aws-upi/must-gather/cluster-scoped-resources/config.openshift.io/dnses.yaml
[5]: https://kubernetes.io/docs/concepts/services-networking/#loadbalancer

Comment 15 Rutvik 2019-08-06 09:53:07 UTC
Hello Chris and Team,

Thanks for all your inputs.

I have checked with the customer and he confirmed that the annotation has been added in the router service only.

-----
kind: Service
apiVersion: v1
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
  name: router-default
    namespace: openshift-ingress
-----

Regarding AWS ec2 api endpoint, they have not configured it yet but the ask is whether it is essential or not? If yes, is it documented somewhere?

Comment 16 W. Trevor King 2019-08-08 20:16:39 UTC
Talking this over among the installer team, we're thinking about moving the installer's docs to configure the *.apps records externally, and to clear the zone properties [1] in the installer-generated DNS config [2] so the ingress operator knows it is not supposed to create records.  We are not planning on adjusting our docs to limit LoadBalancer Service creation, registry storage creation, or other cluster-created resources at this time, although we can revisit those on a case-by-case basis in follow-up bugs.

[1]: https://github.com/openshift/api/blob/b76189cc788c46038952dba8fbccd76b3eba866f/config/v1/types_dns.go#L33-L50
[2]: https://github.com/openshift/installer/blob/b50a68e4b891800ed4823dbf81b21b501ab3c213/pkg/asset/manifests/dns.go#L56

Comment 17 W. Trevor King 2019-08-16 16:12:21 UTC
I see Scott changed the Target Release to 4.2.0.  I've changed Version to match, and we can clone a new bug if we decide we want to backport or officially WONTFIX the 4.1.z installer docs.  Not sure if openshift-docs would need its own bug for this?  They might decide to port changes to their 4.1.z branch even if the installer does not.

Comment 19 Johnny Liu 2019-08-28 11:17:12 UTC
Per https://github.com/openshift/installer/pull/2221/files, seem like this become a document, but no downstream offical doc PR for my review, so change state to "ASSIGNED".

Comment 20 W. Trevor King 2019-08-28 18:55:13 UTC
> ... but no downstream offical doc PR for my review...

No official docs from the installer repo, so I'm changing the component to Documentation (which is where the official docs come from) if we need official docs to close this bug.

Comment 22 Wenjing Zheng 2019-10-12 04:12:47 UTC
No update till now, set target release to 4.2.z.

Comment 23 W. Trevor King 2019-10-12 11:51:50 UTC
Docs are in flight with the to openshift-docs PRs I'm linking.

Comment 25 Kathryn Alexander 2019-10-17 15:46:31 UTC
Gaoyun Pei, will you PTAL? https://github.com/openshift/openshift-docs/pull/17190/

Comment 26 Gaoyun Pei 2019-10-25 09:51:40 UTC
Comment in the PR

Comment 27 Kathryn Alexander 2019-11-05 15:26:47 UTC
Jianlin, thank you for updating the bug, and my apologies for missing Gaoyun Pei's feedback! I've incorporated some changes to the PR. Will you please take a look?

Comment 28 Johnny Liu 2019-11-06 10:31:46 UTC
LGTM.


Note You need to log in before you can comment on or make changes to this bug.