Bug 1364160 - [3.5] facts collection for openshift.common.admin_binary does not seem to work in mixed environments
Summary: [3.5] facts collection for openshift.common.admin_binary does not seem to wor...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cluster Version Operator
Version: 3.2.1
Hardware: Unspecified
OS: Unspecified
high
low
Target Milestone: ---
: 3.5.z
Assignee: Andrew Butcher
QA Contact: Anping Li
URL:
Whiteboard:
: 1347181 1396254 (view as bug list)
Depends On:
Blocks: 1397530 1408663
TreeView+ depends on / blocked
 
Reported: 2016-08-04 15:01 UTC by Eric Rich
Modified: 2020-06-11 12:56 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
In an environment utilizing mixed containerized and RPM based installation methods the installer would fail to gather facts when a master and node used different installation methods. This issue has been addressed and mixed installations should work properly.
Clone Of:
: 1397530 (view as bug list)
Environment:
Last Closed: 2017-04-25 17:41:44 UTC
Target Upstream Version:
Embargoed:
sdodson: needinfo-


Attachments (Terms of Use)
Ansible inventory file and logs (120.15 KB, application/x-gzip)
2016-12-07 05:47 UTC, Anping Li
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:1140 0 normal SHIPPED_LIVE OpenShift Container Platform atomic-openshift-utils bug fix and enhancement 2017-04-25 21:41:12 UTC

Description Eric Rich 2016-08-04 15:01:49 UTC
Description of problem:

The facts collection for {{ openshift.common.admin_binary }} seems to occur on the local facts for a "node" and is not using the fact of the node to which the execution of the command is delegated to. This causes upgrades to fail with mixed RHEL & Atomic environments.

    During execution of "playbooks/common/openshift-cluster/upgrades/v3_1_to_v3_2/upgrade.yml
    The failing task:
    - name: Mark unschedulable if host is a node
        command: >
          {{ openshift.common.admin_binary }} manage-node {{ openshift.common.hostname | lower }} --schedulable=false
        delegate_to: "{{ groups.oo_first_master.0 }}"
        when: inventory_hostname in groups.oo_nodes_to_config

    Observed error message is:
    TASK [Mark unschedulable if host is a node] ************************************
    fatal: [node01.navy.eu-west-1.aws.openpaas.axa-cloud.com -> master01.navy.eu-west-1.aws.openpaas.axa-cloud.com]: FAILED! => {"changed":  false, "cmd": "oadm manage-node ip-10-191-2-209.eu-west-1.compute.internal --schedulable=false", "failed": true, "msg": "[Errno 2] No such file or directory", "rc": 2}

Version-Release number of selected component (if applicable): 3.2.1

How reproducible: Cluster Dependent needs a mixed RHEL & Atomic environments

Steps to Reproduce:
1. Deploy a mixed RHEL & Atomic environments
   Master (RPM) - Master (atomic) - Mastet (Atomic) 
   N Notes
2. Upgrade from 3.1 to 3.2 

Fails with: 

 - fail: msg: This playbook requires access to Docker 1.10 or later 
    when: g_docker_version.avail_version | default(g_docker_version.curr_version, true) | version_compare('1.10','<') 

Additional info:

The error is seen should not be happening, on master02 and master03, which are all on atomic, because we use facts gathering [0] to determine what binary [1] and its location you should be using. 

[0] https://github.com/openshift/openshift-ansible/blob/master/roles/openshift_facts/library/openshift_facts.py#L1700
[1] https://github.com/openshift/openshift-ansible/blob/master/roles/openshift_facts/library/openshift_facts.py#L1506-L1508

Comment 2 Devan Goodwin 2016-08-16 17:36:03 UTC
*** Bug 1347181 has been marked as a duplicate of this bug. ***

Comment 3 Gan Huang 2016-08-23 10:16:36 UTC
FYI, this issue prevents from adding containerized nodes in pre-existing rpm env.

Comment 4 Andrew Butcher 2016-11-22 20:01:14 UTC
Proposed fix: https://github.com/openshift/openshift-ansible/pull/2845

Comment 5 Brenton Leanhardt 2016-11-29 14:01:18 UTC
Will this make it in to 3.4?  I see the PR is merged.

Comment 6 Scott Dodson 2016-11-29 14:52:03 UTC
Yeah it's in.

Comment 8 Anping Li 2016-12-07 05:47:22 UTC
Created attachment 1228867 [details]
Ansible inventory file and logs

The openshift_master_certificates failed on the containerized Env.

Comment 9 Scott Dodson 2016-12-07 14:04:54 UTC
Yeah i ran into this yesterday, i think it's unique to situations where some masters are rpm and some are containerized. I think we only fixed containerized node against rpm master.

Comment 10 Brenton Leanhardt 2017-01-05 21:25:31 UTC
*** Bug 1396254 has been marked as a duplicate of this bug. ***

Comment 12 openshift-github-bot 2017-02-08 18:08:56 UTC
Commit pushed to master at https://github.com/openshift/openshift-ansible

https://github.com/openshift/openshift-ansible/commit/405bd70f0f94f4a45cb4b7cfc7634a82928b6b2e
Merge pull request #3278 from abutcher/mixed-env

Bug 1364160 - facts collection for openshift.common.admin_binary does not seem to work in mixed environments

Comment 14 Anping Li 2017-04-20 09:50:24 UTC
Test pass on openshift-ansible-3.5.55.

Comment 18 errata-xmlrpc 2017-04-25 17:41:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1140

Comment 19 Scott Dodson 2017-06-09 02:53:40 UTC
*** Bug 1387704 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.