Bug 1569631 - RFE: openshift-ansible aws provisioning - need a hook to apply DNS settings to the ELB
Summary: RFE: openshift-ansible aws provisioning - need a hook to apply DNS settings t...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 3.11.0
Assignee: Michael Gugino
QA Contact: sheng.lao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-19 15:49 UTC by Matt Woodson
Modified: 2018-10-11 07:19 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-11 07:19:10 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:2652 0 None None None 2018-10-11 07:19:39 UTC

Description Matt Woodson 2018-04-19 15:49:11 UTC
Description of problem:

The openshift-ansible aws provisioning configures and sets up ELB's that front the masters and the infra nodes.  There is no way to configure DNS to point to these ELB's.

This makes it incredibly hard to setup and openshift-ansible inventory and have it work as expected as dns names are relied upon.

I am asking for a hook of some sort so that we are able to apply DNS names to the ELB that can be used.

I would also want a way to identify the current DNS name of the ELB exposed so that I can use it to make my own CNAME settings in DNS.

Version-Release number of selected component (if applicable):

openshift-ansible 3.9, 3.10

Comment 1 Chris Callegari 2018-07-10 15:44:44 UTC
Ops reqs DNS hook to work with DynDns

Also...
* Ref arch uses AWS/Route53 and bind
* Mobile uses dnsmadeeasy
* Marriott Intl (Marriott Hotels) uses InfoBlox.  Note: Need to discover if there is api available or just uses ns key for updates.

Ref Arch conducted a hack day in Feb 2018.  My project was a dns provider framework and Route53 provider.  The great majority of this work is already complete ... additional provider need to be added.

Comment 2 Chris Callegari 2018-07-16 19:27:08 UTC
Mobile just gave me updated info regarding dns.  They are using Ops's dedicated code and no longer using dnsmadeeasy for production clusters.

Comment 3 Chris Callegari 2018-07-17 01:55:58 UTC
https://trello.com/c/R6yaXLpF/666-aws-dns-hook

Comment 4 Michael Gugino 2018-08-14 16:17:42 UTC
DNS hook & route53 added via https://github.com/openshift/openshift-ansible/pull/9492

For custom dns (hook), set the following in inventory:
openshift_aws_create_dns: True
openshift_aws_dns_provider: "custom" # ("route53" by default) or use route53 to utilize both route53 and custom dns hook.
openshift_aws_custom_dns_provider_role: "my-custom-role" # utilized with include_role.
openshift_aws_custom_provider_role_tasks: "some-tasks.yml" # optional, default is main.yml

All other variables for the role (which records to create, etc) are entirely up to the user.  We query elb dns information for you, and you can access this data via l_openshift_aws_elb_facts dictionary.  Example: l_openshift_aws_elb_facts[openshift_aws_elb_infra_name].dns_name

Comment 5 Scott Dodson 2018-08-14 21:25:03 UTC
Should be in openshift-ansible-3.11.0-0.15.0

Comment 6 sheng.lao 2018-08-21 02:57:26 UTC
1. set the following in inventory:
openshift_aws_create_dns: True

2. play with the provision_install.yml
nsible-playbook -i inventory.yaml -e @provisioning_vars.yaml playbooks/aws/openshift-cluster/provision_install.yml

3. error message:
TASK [openshift_aws : set elb fact dictionary] ***************************************************************************************************
task path: /root/openshift-ansible/roles/openshift_aws/tasks/build_elb_dict.yml:10
Monday 20 August 2018  22:38:30 -0400 (0:00:29.613)       0:01:49.232 *********
The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 138, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 509, in _execute
    self._task.post_validate(templar=templar)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/task.py", line 260, in post_validate
    super(Task, self).post_validate(templar)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/base.py", line 361, in post_validate
    value = templar.template(getattr(self, name))
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 523, in template
    disable_lookups=disable_lookups,
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 479, in template
    disable_lookups=disable_lookups,
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 722, in do_template
    res = j2_concat(rf)
  File "<template>", line 9, in root
  File "/root/openshift-ansible/roles/lib_utils/filter_plugins/oo_filters.py", line 679, in lib_utils_oo_list_of_dict_to_dict_from_key
    retrieved_val = item.get(keyname)
AttributeError: 'str' object has no attribute 'get'
fatal: [localhost]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}

Comment 7 sheng.lao 2018-08-21 02:58:31 UTC
1. Version of openshift-ansible
# git describe
openshift-ansible-3.11.0-0.19.0

Comment 8 Chris Callegari 2018-08-21 13:06:43 UTC
I confirm errors also

1. set the following in extra_vars:
openshift_aws_create_dns: True
openshift_aws_dns_provider: "route53"
# openshift_aws_dns_zone: ""
openshift_aws_dns_zone: "{{ openshift_aws_clusterid }}.sysdeseng.com"

# elb names we want to query to support dns record creation.
# you don't need to adjust this unless you have modified openshift_aws_elb_dict
openshift_aws_elb_names:
- "{{ openshift_aws_elb_master_internal_name }}"
- "{{ openshift_aws_elb_master_external_name }}"
- "{{ openshift_aws_elb_infra_name }}"

# l_openshift_aws_elb_facts is created by querying ec2 for all elb names in
# l_openshift_aws_elb_names via tasks/build_elb_dict.yml
openshift_aws_dns_records:
  # Pertains to inventory file key: openshift_master_cluster_public_hostname
  'api':
    type: 'CNAME'
    # A public or private vpc attached Route53 zone will be created based on
    # private_zone boolean.  Split-tier dns is supported.
    private_zone: False
    value: "{{ l_openshift_aws_elb_facts[openshift_aws_elb_master_external_name].dns_name }}"
  # Pertains to inventory file key: openshift_master_cluster_hostname
  'internal.api':
    type: 'CNAME'
    private_zone: False
    value: "{{ l_openshift_aws_elb_facts[openshift_aws_elb_master_internal_name].dns_name }}"
  # Pertains to inventory file key: openshift_master_default_subdomain
  '*.apps':
    type: "CNAME"
    private_zone: False
    value: "{{ l_openshift_aws_elb_facts[openshift_aws_elb_infra_name].dns_name }}"
  'logs':
    type: "CNAME"
    private_zone: False
    value: "{{ l_openshift_aws_elb_facts[openshift_aws_elb_infra_name].dns_name }}"
  'metrics':
    type: "CNAME"
    private_zone: False
    value: "{{ l_openshift_aws_elb_facts[openshift_aws_elb_infra_name].dns_name }}"
  'registry':
    type: "CNAME"
    private_zone: False
    value: "{{ l_openshift_aws_elb_facts[openshift_aws_elb_infra_name].dns_name }}"

# Allows users to add and recursively override
# https://docs.ansible.com/ansible/2.5/user_guide/playbooks_filters.html#combining-hashes-dictionaries
openshift_aws_dns_records_override: {}

l_openshift_aws_dns_records: "{{ openshift_aws_dns_records | combine(openshift_aws_dns_records_override, recursive=True) }}"


2. play with the provision.yml
ansible-playbook -i 3.10-hosts openshift-cluster/provision.yml -e @provisioning_vars.yml


Results...
TASK [openshift_aws : set elb fact dictionary] ***********************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'str' object has no attribute 'get'
fatal: [localhost]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}
	to retry, use: --limit @/home/ccallega/git/openshift-ansible/playbooks/aws/openshift-cluster/provision.retry



Verbose...
TASK [openshift_aws : set elb fact dictionary] ***********************************************************************************************************************
task path: /home/ccallega/git/openshift-ansible/roles/openshift_aws/tasks/build_elb_dict.yml:10
The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 138, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 509, in _execute
    self._task.post_validate(templar=templar)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/task.py", line 260, in post_validate
    super(Task, self).post_validate(templar)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/base.py", line 361, in post_validate
    value = templar.template(getattr(self, name))
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 523, in template
    disable_lookups=disable_lookups,
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 479, in template
    disable_lookups=disable_lookups,
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 722, in do_template
    res = j2_concat(rf)
  File "<template>", line 12, in root
  File "/home/ccallega/git/openshift-ansible/roles/lib_utils/filter_plugins/oo_filters.py", line 679, in lib_utils_oo_list_of_dict_to_dict_from_key
    retrieved_val = item.get(keyname)
AttributeError: 'str' object has no attribute 'get'

fatal: [localhost]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}
	to retry, use: --limit @/home/ccallega/git/openshift-ansible/playbooks/aws/openshift-cluster/provision.retry

PLAY RECAP ***********************************************************************************************************************************************************
localhost                  : ok=58   changed=2    unreachable=0    failed=1

Comment 9 Michael Gugino 2018-08-21 14:47:04 UTC
Need to get a patch out to fix this.  Most likely need to use from_json filter before sending to custom filter; results from elb_query task are most likely json.

Comment 10 Chris Callegari 2018-08-24 19:09:42 UTC
I also see this in the tasks to be executed...

TASK [include_role : {{ openshift_aws_custom_dns_provider_role }}] ********************************************************************************************

^ That is very confusing... looks like a failed var rendering to me.

Comment 11 Michael Gugino 2018-08-24 22:55:59 UTC
PR Created for elb set fact dictionary: https://github.com/openshift/openshift-ansible/pull/9761

Comment 12 Michael Gugino 2018-08-24 22:58:51 UTC
(In reply to Chris Callegari from comment #10)
> I also see this in the tasks to be executed...
> 
> TASK [include_role : {{ openshift_aws_custom_dns_provider_role }}]
> *****************************************************************************
> ***************
> 
> ^ That is very confusing... looks like a failed var rendering to me.

If openshift_aws_custom_dns_provider_role is defined, you'll see the role name here instead of htis skipped task.

Comment 14 sheng.lao 2018-09-10 08:19:54 UTC
Waiting the Brew Builds

Comment 15 Wei Sun 2018-09-11 03:41:16 UTC
The PR 9761 has been merged to openshift-ansible-3.11.0-0.25.0,please check if it could be verified against the latest build.

Comment 16 sheng.lao 2018-09-11 06:14:09 UTC
@Wei Sun PR-9761 has been merged to openshift-ansible-3.11.0-0.25.0, but state is changed to MODIFIED, in comment #12, so I think that the PR-9761 is not ready to test.

And I verify this bug with openshift-ansible-3.11.0-0.34.0, today.
But this is errata bug, so I need the accurate released version to verify.

Comment 19 errata-xmlrpc 2018-10-11 07:19:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2652


Note You need to log in before you can comment on or make changes to this bug.