Bug 1589346 - Minor Update runs common_deploy_steps_tasks.yaml twice
Summary: Minor Update runs common_deploy_steps_tasks.yaml twice
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z1
: 13.0 (Queens)
Assignee: Jiri Stransky
QA Contact: Raviv Bar-Tal
URL:
Whiteboard:
: 1590597 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-08 18:21 UTC by Tim Rozet
Modified: 2022-07-09 11:34 UTC (History)
10 users (show)

Fixed In Version: python-tripleoclient-9.2.1-13.el7ost openstack-tripleo-heat-templates-8.0.2-43.el7ost
Doc Type: Bug Fix
Doc Text:
Service deployment tasks within the minor-update workflow were run twice caused by superfluous entries in the list of playbooks. This update removes the superfluous playbook entries and includes host preparation tasks directly in the updated playbook. Actions in minor version updates run once in the desired order.
Clone Of:
Environment:
Last Closed: 2018-07-19 14:27:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1776206 0 None None None 2018-06-11 10:59:11 UTC
OpenStack gerrit 575149 0 None MERGED Remove deploy playbook from `update run` 2020-08-05 15:28:17 UTC
OpenStack gerrit 575680 0 None MERGED Updates: run host_prep_tasks between update tasks and deployment tasks 2020-08-05 15:28:17 UTC
OpenStack gerrit 578044 0 None MERGED Use role_name instead of tripleo_role_name 2020-08-05 15:28:17 UTC
Red Hat Issue Tracker OSP-17203 0 None None None 2022-07-09 11:34:20 UTC
Red Hat Product Errata RHSA-2018:2214 0 None None None 2018-07-19 14:28:01 UTC

Description Tim Rozet 2018-06-08 18:21:30 UTC
Description of problem:
When issuing minor update in OSP13, there are multiple starts of the service container during update.  For example, looking at docker logs for opendaylight_api container:

Jun 08 13:25:01 controller-1 dockerd-current[19336]: + sudo -E kolla_set_configs
Jun 08 16:54:41 controller-1 dockerd-current[19336]: + sudo -E kolla_set_configs
Jun 08 17:41:42 controller-1 dockerd-current[19336]: + sudo -E kolla_set_configs

We can see that the first iteration is the original deployment, while the next 2 are part of the update.  Note, these do not include puppet containers.  Therefore something is triggering the container to go down then come back up a 2nd time during update.

Comment 1 Tim Rozet 2018-06-08 18:42:33 UTC
Jiri suggested that:
<jistr> so what i *think* right now, with the double-execution issue, it probably looks like:
<jistr> 1) update_tasks
<jistr> 2) deploy tasks (puppet, containers)
<jistr> 3) host_prep_tasks
<jistr> 4) deploy tasks again

This issue looks to be causing an issue with ODL update:
https://bugzilla.redhat.com/show_bug.cgi?id=1586171#c23

I'm trying to workaround it as mentioned in the above comment.

Comment 2 Jiri Stransky 2018-06-08 20:35:52 UTC
Based on comments at:

https://bugzilla.redhat.com/show_bug.cgi?id=1586171#c24
https://bugzilla.redhat.com/show_bug.cgi?id=1586171#c25

fixing this bug would likely not fix bug 1586171 fully, and we haven't observed any other breakages due to starting the containers twice, so i'd like to lower the severity rating for now. In case my assessment isn't accurate, feel free to re-triage.

Either way i'd still like to investigate/fix this soon.

Comment 4 Tim Rozet 2018-06-09 02:05:07 UTC
Hi Jiri, I prioritized it as high not because of bug 158671, but because it adds an extra 45min to the update process from what I can see in the logs.  I'm fine with a fix in Z.

Comment 5 Tim Rozet 2018-06-09 02:05:44 UTC
Sorry entered wrong bug in previous comment, bug 1586171

Comment 6 Jiri Stransky 2018-06-11 10:59:11 UTC
Ack, i didn't realize it would add that much time, i'll move the priority back to high. Also linking upstream bug report.

Comment 11 Jiri Stransky 2018-06-18 12:46:26 UTC
*** Bug 1590597 has been marked as a duplicate of this bug. ***

Comment 12 Jiri Stransky 2018-06-29 14:21:50 UTC
Merged to stable/queens.

Comment 24 errata-xmlrpc 2018-07-19 14:27:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2214


Note You need to log in before you can comment on or make changes to this bug.