Bug 1725834 - Scale up playbook tries to run and fails if openshift_node_group_name points to a non-existing group
Summary: Scale up playbook tries to run and fails if openshift_node_group_name points ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 3.11.z
Assignee: Russell Teague
QA Contact: Gaoyun Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-01 14:16 UTC by Pablo Alonso Rodriguez
Modified: 2023-03-24 15:01 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-16 11:57:10 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift openshift-ansible pull 11984 0 'None' closed Bug 1725834: Create configmaps only for used openshift_node_group_names 2020-06-09 18:19:54 UTC
Red Hat Product Errata RHBA-2019:4050 0 None None None 2019-12-16 11:57:26 UTC

Description Pablo Alonso Rodriguez 2019-07-01 14:16:48 UTC
Description of problem:

If a user modifies openshift_node_groups to add a new group (but does not reflect that change on the cluster) and tries to scale up to add a node of that group, scale up playbook tries the scale up anyway and fails (because the node points to a non-existing configmap).

Scale up playbooks should either update the config maps on openshift-node project or fail early during prechecks.

Version-Release number of the following components:
rpm -q openshift-ansible

openshift-ansible-3.11.98-1.git.0.3cfa7c3.el7.noarch

rpm -q ansible

ansible-2.6.16-1.el7ae.noarch

ansible --version

ansible 2.6.16
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/cloud-user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Mar 26 2019, 22:13:06) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

How reproducible:

Always

Steps to Reproduce:
1. Install a cluster with custom openshift_node_groups
2. Modify the inventory to add a new group to openshift_node_groups but do not modify configmaps at openshift-node project (so that configmap for the new group does not exist)
3. Try to scale up to add a new node on the new group

Actual results:

Scale up playbooks try to actually perform the scale up and fail (as one would expect). New configmap for new group does not exist in the openshift-node project.

Expected results:

Scale up playbook should take one of these actions, either:
- Reconcile the current value of openshift_node_groups with the one at the inventory BEFORE trying to scale up
- Fail early during pre-check phase and let user know how to fix it (maybe by running other playbook...)

Additional info:

This was easily workarounded by manually fixing the configmaps on openshift-node project.

Comment 1 Russell Teague 2019-07-01 14:28:57 UTC
Please attach Ansible log.

Comment 3 Pablo Alonso Rodriguez 2019-07-01 15:09:13 UTC
Logs attached privately. However, this should be easy to reproduce

Comment 6 Gaoyun Pei 2019-12-03 10:26:53 UTC
Could reproduce this bug with openshift-ansible-3.11.98-1.git.0.3cfa7c3.el7.noarch

Add a new node host to [new_nodes] group, set a new openshift_node_group_name to it, but without such node_group configmap under openshift-node project. 
The new node scale-up failed at TASK [openshift_manage_node : Wait for sync DS to set annotations on all nodes] ***

Works well with openshift-ansible-3.11.156-1.git.0.5f2cfa4.el7.noarch.rpm.

The scale-up playbook will create the missing configmap and add the node to the cluster successfully.

Comment 8 errata-xmlrpc 2019-12-16 11:57:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:4050


Note You need to log in before you can comment on or make changes to this bug.