Bug 1471721

Summary:	'openstack overcloud update ...' fails when yum is locked on an overcloud node
Product:	Red Hat OpenStack	Reporter:	Marios Andreou <mandreou>
Component:	openstack-tripleo-heat-templates	Assignee:	Marios Andreou <mandreou>
Status:	CLOSED ERRATA	QA Contact:	Raviv Bar-Tal <rbartal>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	11.0 (Ocata)	CC:	augol, dbecker, jschluet, jslagle, mandreou, mburns, morazi, pneedle, rhel-osp-director-maint, slinaber
Target Milestone:	z5	Keywords:	Triaged, ZStream
Target Release:	11.0 (Ocata)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-6.2.7-6.el7ost	Doc Type:	Enhancement
Doc Text:	In this enhancement, if a minor update is blocked by an existing yum process that prevents the package update, the process should exit with an appropriate error message. This was added because the minor update may appear to freeze, due to yum waiting for the existing yum.pid to exit; when it eventually fails it is not immediately clear why. As a result, if there is an existing yum process preventing the package update, then the minor update fails with a clear message to indicate this: "ERROR existing yum.pid detected - can't continue! Please ensure there is no other package update process for the duration of the minor update worfklow. Exiting".	Story Points:	---
Clone Of:	1471720	Environment:
Last Closed:	2018-05-18 17:02:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1467649, 1471720
Bug Blocks:

Description Marios Andreou 2017-07-17 10:29:26 UTC

+++ This bug was initially created as a clone of Bug #1471720 +++

+++ This bug was initially created as a clone of Bug #1467649 +++

Description of problem:

'openstack overcloud update ...' fails when yum is already locked on an overcloud node. The overcloud update subsequently times out waiting for yum_update.sh to obtain a list of packages requiring an update on that node.

Version-Release number of selected component (if applicable):

* Red Hat OpenStack Platform 9.
* RHEL 7.
* yum-3.4.3-150.el7.

How reproducible:

Every time.

Steps to Reproduce:

1. On an overcloud node, trigger a scenario where yum is locked. In a real world use-case, this could be due to a previous issue being experienced on an overcloud node.

2. Run 'openstack overcloud update ...'.

Actual results:

The overcloud update times out when the yum_update.sh script tries to obtain a list of packages requiring update from the node which is locked.

Expected results:

For the overcloud update sequence to handle this situation effectively, perhaps through clear error messaging or by handling of the yum lock.

Comment 1 Marios Andreou 2017-07-17 10:35:14 UTC

clones: 
OSP9 @ https://bugzilla.redhat.com/show_bug.cgi?id=1467649 
OSP10@ https://bugzilla.redhat.com/show_bug.cgi?id=1471720
OSP11@ https://bugzilla.redhat.com/show_bug.cgi?id=1471721

No BZ needed for OSP12 since the minor update workflow there is different. However  the files do still exist on master and we landed this one first https://review.openstack.org/#/c/483390/ (even though it won't be used by master/Pike) and then backported to stable/newton since that is the oldest we have right now. Will have to be d/stream for OSP9 which is what this bug was opened for in the first place. Once stable/newton merges I will post the d/stream review and attach to the OSP9 clone.

Comment 2 Lon Hohberger 2017-10-10 18:08:57 UTC

According to our records, this should be resolved by openstack-tripleo-heat-templates-6.2.0-3.el7ost.  This build is available now.

Comment 7 Marios Andreou 2017-12-18 15:20:32 UTC

just discussed on Upgrades scrum assigning to myself for investigation

Comment 9 Marios Andreou 2017-12-19 16:44:24 UTC

yprokule posted this https://review.openstack.org/#/c/528984/ adding to trackers

Comment 10 Yurii Prokulevych 2018-01-02 12:18:34 UTC

Cherry-pick to stable/ocata added - https://review.openstack.org/#/c/530725/

Comment 28 errata-xmlrpc 2018-05-18 17:02:55 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1627