Bug 1372373

Description of problem:

Installing OSCP 3.2 on RHEL 7.2 patched up to latest, in "Disconnected" mode as per https://access.redhat.com/documentation/en/openshift-enterprise/3.2/single/installation-and-configuration/#install-config-install-disconnected-install.

Install fails with:

fatal: [osetest2.hostonly]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "openshift_facts"}, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_ZVnDbI/ansible_module_openshift_facts.py\", line 2162, in <module>\r\n    main()\r\n  File \"/tmp/ansible_ZVnDbI/ansible_module_openshift_facts.py\", line 2143, in main\r\n    protected_facts_to_overwrite)\r\n  File \"/tmp/ansible_ZVnDbI/ansible_module_openshift_facts.py\", line 1619, in __init__\r\n    protected_facts_to_overwrite)\r\n  File \"/tmp/ansible_ZVnDbI/ansible_module_openshift_facts.py\", line 1652, in generate_facts\r\n    defaults = self.get_defaults(roles, deployment_type)\r\n  File \"/tmp/ansible_ZVnDbI/ansible_module_openshift_facts.py\", line 1694, in get_defaults\r\n    ip_addr = self.system_facts['ansible_default_ipv4']['address']\r\nKeyError: 'address'\r\n", "msg": "MODULE FAILURE", "parsed": false}
fatal: [osetest.hostonly]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "openshift_facts"}, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_qEu1xu/ansible_module_openshift_facts.py\", line 2162, in <module>\r\n    main()\r\n  File \"/tmp/ansible_qEu1xu/ansible_module_openshift_facts.py\", line 2143, in main\r\n    protected_facts_to_overwrite)\r\n  File \"/tmp/ansible_qEu1xu/ansible_module_openshift_facts.py\", line 1619, in __init__\r\n    protected_facts_to_overwrite)\r\n  File \"/tmp/ansible_qEu1xu/ansible_module_openshift_facts.py\", line 1652, in generate_facts\r\n    defaults = self.get_defaults(roles, deployment_type)\r\n  File \"/tmp/ansible_qEu1xu/ansible_module_openshift_facts.py\", line 1694, in get_defaults\r\n    ip_addr = self.system_facts['ansible_default_ipv4']['address']\r\nKeyError: 'address'\r\n", "msg": "MODULE FAILURE", "parsed": false}

Version-Release number of selected component (if applicable):

[root@osetest ansible]# yum list installed | grep openshift
atomic-openshift-utils.noarch    3.2.24-1.git.0.337259b.el7  @rhel-7-server-ose-3.2-rpms
openshift-ansible.noarch         3.2.24-1.git.0.337259b.el7  @rhel-7-server-ose-3.2-rpms
openshift-ansible-docs.noarch    3.2.24-1.git.0.337259b.el7  @rhel-7-server-ose-3.2-rpms
openshift-ansible-filter-plugins.noarch
openshift-ansible-lookup-plugins.noarch
openshift-ansible-playbooks.noarch
openshift-ansible-roles.noarch   3.2.24-1.git.0.337259b.el7  @rhel-7-server-ose-3.2-rpms
[root@osetest ansible]# yum list installed | grep ansible
ansible.noarch                   2.1.1.0-1.el7               @rhel-7-server-ose-3.2-rpms
openshift-ansible.noarch         3.2.24-1.git.0.337259b.el7  @rhel-7-server-ose-3.2-rpms
openshift-ansible-docs.noarch    3.2.24-1.git.0.337259b.el7  @rhel-7-server-ose-3.2-rpms
openshift-ansible-filter-plugins.noarch
openshift-ansible-lookup-plugins.noarch
openshift-ansible-playbooks.noarch
openshift-ansible-roles.noarch   3.2.24-1.git.0.337259b.el7  @rhel-7-server-ose-3.2-rpms


NOTE:  Problem initially manifested with Ansible 2.2 prerelease rpms which installed by default.  Backing up to 2.1.1.0 showed the problem persisted.

How reproducible:

Every time


Steps to Reproduce:
1. Install RHEL 7 on two VMs with internal-only networking
2. Configure yum repos to point to internal http repo and enable rhel-7-server-rpms; rhel-7-server-rpms; and rhel-7-server-ose-3.2-rpms
3. Update / install relevant packages as per disconnected install documentation
4. Configure /etc/ansible/hosts as per single-master example to reflect the two installed VM hosts, with one as master and the other as node-only
5. Run ansible-playbook -vvv /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml

Actual results:

Playbook install fails with error message above

Expected results:

Playbook succeeds


Additional info:

The error message is referencing failure getting IP from "ansible_default_ipv4":

ip_addr = self.system_facts['ansible_default_ipv4']['address']\r\nKeyError: 'address'\r\n", "msg": "MODULE FAILURE", "parsed": false}

.. we found that value to be NULL in the ansible facts (truncated output below):

root@osetest ~]# ansible osetest2.hostonly -m setup 
osetest2.hostonly | SUCCESS => {
    "ansible_facts": {
        "ansible_all_ipv4_addresses": [
            "192.168.100.144", 
            "172.17.0.1"
...
...
            "tz_offset": "+0100", 
            "weekday": "Thursday", 
            "weekday_number": "4", 
            "weeknumber": "35", 
            "year": "2016"
        }, 
        "ansible_default_ipv4": {}, 
...

Some investigation found that Ansible apparently uses:

ip -4 route get 8.8.8.8

.. to obtain ansible_default_ipv4, and on this disconnected system that returns nothing useful:

[root@osetest2 ~]# !100
ip -4 route get 8.8.8.8
RTNETLINK answers: Network is unreachable

Do we therefore need to find a different way to populate ansible_default_ipv4 on a system that's not connected to the internet, or we need to change the openshift installer playbooks to test for a different fact?

I'm not sure how to work around this issue currently for a disconnected install and I have a customer who needs me to perform this, probably within the next week.