Bug 1817507
| Summary: | [OSP15->16] external_deploy_tasks skipped during Controller upgrade | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Jose Luis Franco <jfrancoa> |
| Component: | openstack-tripleo-common | Assignee: | Jose Luis Franco <jfrancoa> |
| Status: | CLOSED ERRATA | QA Contact: | David Rosenfeld <drosenfe> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 16.1 (Train) | CC: | jpretori, jschluet, mburns, owalsh, pkopec, sclewis, slinaber |
| Target Milestone: | beta | Keywords: | Reopened, Triaged |
| Target Release: | 16.1 (Train on RHEL 8.2) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-tripleo-common-11.3.3-0.20200413111906.d36c759.el8ost | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-07-29 07:51:01 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Jose Luis Franco
2020-03-26 13:14:10 UTC
Could this change https://review.opendev.org/#/c/690795/ have anything to do with the failure? placement password in keystone was not updated It seems that this task was missing to run during deploy tasks playbook: https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/keystone/keystone-container-puppet.yaml#L721-L755 mostly because the upgrade run command is running with the --limit Controller option and the External deploy tasks are run agains the Undercloud host: https://github.com/openstack/tripleo-heat-templates/blob/master/common/deploy-steps.j2#L654 Therefore, all the external deploy tasks are being skipped: 2020-03-25 16:56:41 | PLAY [External deployment step 1] ********************************************** 2020-03-25 16:56:41 | skipping: no hosts matched 2020-03-25 17:05:18 | PLAY [External deployment step 2] ********************************************** 2020-03-25 17:05:18 | skipping: no hosts matched 2020-03-25 17:15:45 | PLAY [External deployment step 3] ********************************************** 2020-03-25 17:15:45 | skipping: no hosts matched 2020-03-25 17:15:45 | 2020-03-25 17:20:01 | PLAY [External deployment step 4] ********************************************** 2020-03-25 17:20:01 | skipping: no hosts matched One option would be running the deploy_steps_playbook.yaml with --limit Undercloud always if the --limit option is used during the overcloud upgrade run command, but that could run some undesired steps. (In reply to Ollie Walsh from comment #2) > placement password in keystone was not updated Since you linked [1], which is merged, I'm going to take a gamble and close this as UPSTREAM (there's no customer case to ask for a build with an nvr). If this is incorrect and there's still work to do here, please re-open. [1] https://review.opendev.org/708412 Placement service has now the right password and the service starts correctly:
d86d35889b64 undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-nova-api:16.1_20200505.1 /usr/bin/bootstra... 2 hours ago Exited (0) 2 hours ago nova_api_ensure_default_cell 12.4MB (virtual 1.09GB)
40257cb54a97 undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-placement-api:16.1_20200505.1 /usr/bin/bootstra... 2 hours ago Exited (0) 2 hours ago placement_api_db_sync 12.4MB (virtual 590MB)
777c209decb1 undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-nova-api:16.1_20200505.1 /usr/bin/bootstra... 2 hours ago Exited (0) 2 hours ago nova_api_map_cell0 12.4MB (virtual 1.09GB)
008f375745a6 undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-placement-api:16.1_20200505.1 /usr/bin/bootstra... 2 hours ago Exited (0) 2 hours ago placement_api_db_extract_data_from_nova_api 12.4MB (virtual 590MB)
Controllers do also get properly upgraded:
2020-05-08 02:56:14 | TASK [include_tasks] ***********************************************************
2020-05-08 02:56:14 | Friday 08 May 2020 02:56:12 +0000 (0:00:00.533) 0:01:11.786 ************
2020-05-08 02:56:14 | skipping: [controller-0] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-05-08 02:56:14 | skipping: [controller-1] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-05-08 02:56:14 | skipping: [controller-2] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-05-08 02:56:14 |
2020-05-08 02:56:14 | TASK [include_tasks] ***********************************************************
2020-05-08 02:56:14 | Friday 08 May 2020 02:56:13 +0000 (0:00:00.526) 0:01:12.312 ************
2020-05-08 02:56:14 | skipping: [controller-0] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-05-08 02:56:14 | skipping: [controller-1] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-05-08 02:56:14 | skipping: [controller-2] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-05-08 02:56:14 |
2020-05-08 02:56:14 | PLAY RECAP *********************************************************************
2020-05-08 02:56:14 | controller-0 : ok=17 changed=5 unreachable=0 failed=0 skipped=35 rescued=0 ignored=0
2020-05-08 02:56:14 | controller-1 : ok=15 changed=4 unreachable=0 failed=0 skipped=36 rescued=0 ignored=0
2020-05-08 02:56:14 | controller-2 : ok=15 changed=4 unreachable=0 failed=0 skipped=36 rescued=0 ignored=0
2020-05-08 02:56:14 |
2020-05-08 02:56:14 | Friday 08 May 2020 02:56:13 +0000 (0:00:00.354) 0:01:12.667 ************
2020-05-08 02:56:14 | ===============================================================================
2020-05-08 02:56:14 |
2020-05-08 02:56:14 | Updated nodes - Controller
2020-05-08 02:56:14 | Success
2020-05-08 02:56:14 | 2020-05-08 02:56:14.491 75654 INFO tripleoclient.v1.overcloud_upgrade.MajorUpgradeRun [-] Completed Overcloud Upgrade Run for Controller with playbooks ['upgrade_steps_playbook.yaml', 'deploy_steps_playbook.yaml', 'post_upgrade_steps_playbook.yaml'] ^[[00m
Tested with t-c package:
openstack-tripleo-common-11.3.3-0.20200502033438.30ebd4d.el8ost.noarch
Logs: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/upgrades/view/upgrade/job/DFG-upgrades-upgrade-upgrade-15-16.1_director-rhel-virthost-3cont_2comp_3ceph-ipv4-geneve-HA/24/
If this bug requires doc text for errata release, please set the 'Doc Type' and provide draft text according to the template in the 'Doc Text' field. The documentation team will review, edit, and approve the text. If this bug does not require doc text, please set the 'requires_doc_text' flag to '-'. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3148 |