Bug 1583157

Summary: Individual Component Playbooks for node need to be updated
Product: OpenShift Container Platform Reporter: Johnny Liu <jialiu>
Component: InstallerAssignee: Russell Teague <rteague>
Status: CLOSED CURRENTRELEASE QA Contact: Johnny Liu <jialiu>
Severity: high Docs Contact:
Priority: high    
Version: 3.10.0CC: aos-bugs, jokerman, mgugino, mmccomas, rteague, xtian
Target Milestone: ---Keywords: TestBlocker
Target Release: 3.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
During installer refactoring for bootstrapping, the node entry-point playbooks were removed. Entry-point playbooks for bootstrap preparation and joining nodes to the cluster have been added.
Story Points: ---
Clone Of:
: 1609678 (view as bug list) Environment:
Last Closed: 2018-12-20 21:36:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1609678    
Attachments:
Description Flags
installation log with inventory file embedded none

Description Johnny Liu 2018-05-28 10:58:03 UTC
Description of problem:
Here is 3.9 doc for "Individual Component Playbooks":
https://docs.openshift.com/container-platform/3.9/install_config/install/advanced_install.html#running-the-advanced-installation-individual-components
Obviously the playbooks path and order is changed in 3.10, after go through code, the order should be like this (if not, pls correct me):
openshift-checks/pre-install.yml
openshift-node/private/bootstrap.yml
openshift-etcd/config.yml
openshift-nfs/config.yml
openshift-loadbalancer/config.yml
openshift-master/config.yml
openshift-master/additional_config.yml
openshift-node/private/join.yml
openshift-glusterfs/config.yml
openshift-hosted/config.yml
openshift-web-console/config.yml
openshift-metrics/config.yml
openshift-logging/config.yml
openshift-prometheus/config.yml
openshift-monitoring/config.yml
openshift-monitor-availability/config.yml
openshift-service-catalog/config.yml
openshift-management/config.yml
openshift-descheduler/config.yml
openshift-node-problem-detector/config.yml
openshift-autoheal/config.yml

Here for "openshift-node/private/bootstrap.yml" and "openshift-node/private/join.yml", I did not find any <role>/config.yml entry playbook, we should keep them insistent like other roles.

Even this is by design, we need this bug to track doc update.

Version-Release number of the following components:
3.10.0-0.53.0


How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Johnny Liu 2018-05-29 09:04:23 UTC
I followed the playbook order in my initial report, failed.

############ ANSIBLE RUN: /home/slave3/workspace/Launch-Environment-Flexy/private-openshift-ansible/playbooks/openshift-node/private/bootstrap.yml ############################
[07:58:51] INFO> Shell Commands: ANSIBLE_FORCE_COLOR=true
ANSIBLE_CALLBACK_WHITELIST=profile_tasks
["ansible-playbook", "-v", "-i", "/home/slave3/workspace/Launch-Environment-Flexy/workdir/prod-jenkins-slave-5-10.8.243.159-3-0/inventory.openshift-ansible", "/home/slave3/workspace/Launch-Environment-Flexy/private-openshift-ansible/playbooks/openshift-node/private/bootstrap.yml"]
Using /etc/ansible/ansible.cfg as config file

PLAY [Node Preparation Checkpoint Start] ***************************************

TASK [Set Node preparation 'In Progress'] **************************************
Tuesday 29 May 2018  03:58:52 -0400 (0:00:00.059)       0:00:00.059 *********** 
ok: [ec2-34-229-102-35.compute-1.amazonaws.com] => {"ansible_stats": {"aggregate": true, "data": {"installer_phase_node": {"playbook": "(no entry point playbook)", "start": "20180529035852Z", "status": "In Progress", "title": "Node Preparation"}}, "per_host": false}, "changed": false, "failed": false}

PLAY [Only target nodes that have not yet been bootstrapped] *******************

TASK [Gathering Facts] *********************************************************
Tuesday 29 May 2018  03:58:52 -0400 (0:00:00.047)       0:00:00.106 *********** 
ok: [localhost]

TASK [add_host] ****************************************************************
Tuesday 29 May 2018  03:58:52 -0400 (0:00:00.524)       0:00:00.630 *********** 
 [WARNING]: Could not match supplied host pattern, ignoring:
oo_nodes_to_bootstrap

 [WARNING]: Could not match supplied host pattern, ignoring:
oo_exclude_bootstrapped_nodes


PLAY [Disable excluders] *******************************************************
skipping: no hosts matched

PLAY [Configure nodes] *********************************************************
skipping: no hosts matched

PLAY [node bootstrap config] ***************************************************
skipping: no hosts matched

PLAY [Re-enable excluder if it was previously enabled] *************************
skipping: no hosts matched

PLAY [Node Preparation Checkpoint End] *****************************************

TASK [Set Node preparation 'Complete'] *****************************************
Tuesday 29 May 2018  03:58:52 -0400 (0:00:00.033)       0:00:00.664 *********** 
ok: [ec2-34-229-102-35.compute-1.amazonaws.com] => {"ansible_stats": {"aggregate": true, "data": {"installer_phase_node": {"end": "20180529035852Z", "status": "Complete"}}, "per_host": false}, "changed": false, "failed": false}

PLAY RECAP *********************************************************************
ec2-34-229-102-35.compute-1.amazonaws.com : ok=2    changed=0    unreachable=0    failed=0   
localhost                  : ok=1    changed=0    unreachable=0    failed=0   

Tuesday 29 May 2018  03:58:52 -0400 (0:00:00.037)       0:00:00.702 *********** 
=============================================================================== 
Gathering Facts --------------------------------------------------------- 0.52s
Set Node preparation 'In Progress' -------------------------------------- 0.05s
Set Node preparation 'Complete' ----------------------------------------- 0.04s
add_host ---------------------------------------------------------------- 0.03s
waiting for operation up to 36000 seconds..

[07:58:53] INFO> Exit Status: 0
############ ANSIBLE END: /home/slave3/workspace/Launch-Environment-Flexy/private-openshift-ansible/playbooks/openshift-node/private/bootstrap.yml 


The role do nothing, due to openshift_is_bootstrapped is not set to true, this totally broke individual component playbook installation. Raise its severity.

Comment 2 Johnny Liu 2018-05-29 09:04:55 UTC
Created attachment 1445265 [details]
installation log with inventory file embedded

Comment 3 Michael Gugino 2018-05-29 12:01:37 UTC
You should not run any plays that are in <component>/private/ directories directly.  There should be a corresponding play in <component>/ directory.

Comment 4 Johnny Liu 2018-05-30 03:37:31 UTC
(In reply to Michael Gugino from comment #3)
> You should not run any plays that are in <component>/private/ directories
> directly.  
Totally agree.

>There should be a corresponding play in <component>/ directory.
The truth is false, there is no corresponding play in openshift-node/ directory, that is the core issue for this bug.

Comment 5 Johnny Liu 2018-05-31 02:30:58 UTC
Why this bug is changed to "Documentation" component, how to resolve comment 4?

>There should be a corresponding play in <component>/ directory.
The truth is false, there is no corresponding play in openshift-node/ directory, that is the core issue for this bug.

Comment 6 Scott Dodson 2018-05-31 13:10:22 UTC
We need to document which playbooks can be run in order and clearly outline which components you could previously run as individual components can be no longer be run individually.

Comment 7 Russell Teague 2018-06-15 19:37:21 UTC
Proposed: https://github.com/openshift/openshift-ansible/pull/8795

Current list of individual playbooks:
openshift-checks/pre-install.yml
openshift-node/bootstrap.yml
openshift-etcd/config.yml
openshift-nfs/config.yml
openshift-loadbalancer/config.yml
openshift-master/config.yml
openshift-master/additional_config.yml
openshift-node/join.yml
openshift-glusterfs/config.yml
openshift-hosted/config.yml
openshift-monitoring/config.yml
openshift-web-console/config.yml
openshift-metrics/config.yml
openshift-logging/config.yml
openshift-prometheus/config.yml
openshift-monitoring/config.yml
openshift-monitor-availability/config.yml
openshift-service-catalog/config.yml
openshift-management/config.yml
openshift-descheduler/config.yml
openshift-node-problem-detector/config.yml
openshift-autoheal/config.yml

Comment 8 openshift-github-bot 2018-06-18 20:57:52 UTC
Commit pushed to master at https://github.com/openshift/openshift-ansible

https://github.com/openshift/openshift-ansible/commit/a7aebd731b4e93e333667c47a92518870595a465
Merge pull request #8795 from mtnbikenc/fix-1583157

Bug 1583157 - Add openshift-node entry-point playbooks

Comment 9 Russell Teague 2018-06-19 13:27:12 UTC
release-3.10: https://github.com/openshift/openshift-ansible/pull/8847

Comment 11 Russell Teague 2018-06-20 19:27:09 UTC
openshift-ansible-3.10.2-1.git.190.5abfddb.el7

Comment 12 Johnny Liu 2018-06-21 08:09:50 UTC
Verified this bug with openshift-ansible-3.10.2-1.git.190.5abfddb.el7.noarch, and PASS.

Install an env following the playbooks list:
playbook_list: ["prerequisites.yml", "openshift-checks/pre-install.yml", "openshift-node/bootstrap.yml", "openshift-etcd/config.yml", "openshift-nfs/config.yml", "openshift-master/config.yml", "openshift-master/additional_config.yml", "openshift-node/join.yml", "openshift-hosted/config.yml", "openshift-web-console/config.yml", "openshift-prometheus/config.yml", "openshift-service-catalog/config.yml"]

Install succeed.