Bug 1346009 - [RFE] Upgrades process logging needs improvement (.instack/install-undercloud.log and overcloud non heat stack-update)
Summary: [RFE] Upgrades process logging needs improvement (.instack/install-undercloud...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: Upstream M3
: 11.0 (Ocata)
Assignee: Marios Andreou
QA Contact: Amit Ugol
URL:
Whiteboard:
Depends On:
Blocks: 1426491
TreeView+ depends on / blocked
 
Reported: 2016-06-13 16:21 UTC by Alexander Chuzhoy
Modified: 2017-05-17 19:30 UTC (History)
12 users (show)

Fixed In Version: instack-undercloud-6.0.0-0.20170218232200.d695df8.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-17 19:30:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 425770 0 None None None 2017-02-06 12:15:59 UTC
OpenStack gerrit 427218 0 None None None 2017-02-06 12:16:25 UTC
Red Hat Product Errata RHEA-2017:1245 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 23:01:50 UTC

Description Alexander Chuzhoy 2016-06-13 16:21:06 UTC
rhel-osp-director: [RFE] Undercloud upgrade should have a respective output in .instack/install-undercloud.log

Environment:
instack-undercloud-2.2.7-7.el7ost.noarch

Deployed undercloud under 7.3:
#############################################################################
instack-install-undercloud complete.

The file containing this installation's passwords is at
/home/stack/undercloud-passwords.conf.

There is also a stackrc file at /home/stack/stackrc.

These files are needed to interact with the OpenStack services, and should be
secured.

#############################################################################




Upgraded to 8.0:
#############################################################################
Undercloud install complete.

The file containing this installation's passwords is at
/home/stack/undercloud-passwords.conf.

There is also a stackrc file at /home/stack/stackrc.

These files are needed to interact with the OpenStack services, and should be
secured.

#############################################################################


The message upon undercloud upgrade should say:
Undercloud upgrade to $version is complete.

There could be several upgrade (several versions) and there should be a respective indication in the log.

Comment 6 Marios Andreou 2017-01-27 16:18:54 UTC
jcoufal, follow up as I said I'd have a look here. Assuming I've understood the request correctly, we need the undercloud upgrade to signal that it is an upgrade vs an install in the instack-undercloud logfile and then for which version. 

For the first part it is very easy I have a simple review at https://review.openstack.org/#/c/425770/ that will say 'upgrade' vs 'install' using something Alex Schultz landed recently. For the version I'm not sure how we could do that, perhaps we could write a static string version at the end of the instack run?

In any case, wdyt? Will https://review.openstack.org/#/c/425770/ at least cover part of this? All the runs whether install or upgrade are logged and timestampled already in .instack/install-undercloud.log. Hopefully that review will make the file easier to read.

We could make this RFE be more generally about upgrades process improvements, and include some other things like a minor improvement to the upgrade-non-controller.sh script I'm working on (for better logging so it is somewhat related/relevant). WDYT?

thanks

Comment 8 Jaromir Coufal 2017-01-30 21:09:30 UTC
+1 Marios. Can you attach relevant reviews and change the title, please?

Comment 10 Marios Andreou 2017-02-06 12:20:21 UTC
Hi, just updated the title as per comment #8 and attached two reviews: 

https://review.openstack.org/#/c/427218/ Adds basic logging to the upgrade-non-controller.sh utility
This improves the overcloud upgrades process logging for the non controller nodes - actually, compute and swift only for OSP11 - other nodes are upgraded via heat stack-update with all the existing logging there.

https://review.openstack.org/#/c/425770/ Ammend the completion and error message to signal an upgrade
This indicates whether the undercloud was just installed or upgraded (successfully or when there is an error) in the .instack/install-undercloud.log for the undercloud upgrade.

Moving to POST as these merged

Comment 12 Steve Linabery 2017-02-21 20:55:15 UTC
patches have landed in rhos-11.0 builds:
instack-undercloud-6.0.0-0.20170218232200.d695df8.el7ost
openstack-tripleo-common-5.8.1-0.20170218163132.346461d.el7ost

Comment 14 Marius Cornea 2017-03-02 10:20:08 UTC
I see .instack/install-undercloud.log is not logging upgrade info, only instal. 

Nevertheless the 'openstack undercloud upgrade' points to an upgrade not install:

#############################################################################
Undercloud upgrade complete.

The file containing this installation's passwords is at
/home/stack/undercloud-passwords.conf.

There is also a stackrc file at /home/stack/stackrc.

These files are needed to interact with the OpenStack services, and should be
secured.

#############################################################################

Comment 15 Marios Andreou 2017-03-02 10:23:25 UTC
as discussed just now on irc thanks to mcornea i need to revisit this... it seems the undercloud upgrade isn't being logged in the install-undercloud.log so i have missed some wiring somewhere

Comment 16 Marios Andreou 2017-03-02 14:06:14 UTC
i spent a while staring at the code in https://github.com/openstack/instack-undercloud/blob/master/instack_undercloud/undercloud.py#L1419 trying to work out what on earth is going wrong with the logging. 

I eventually came across this from bnemec: https://review.openstack.org/#/c/439097/ Revert "Turn off propagation for undercloud logger" from last night in fact... 

So it seems the doesn't log to file bit that mcornea points out in comment #14 is not related to the fixes landed for this bug. I just checked in my environment and indeed, I see the problematic "LOG.propagate = False" in the local instack-undercloud-6.0.0-0.20170220135156.d695df8.el7ost.noarch

To test that you can check and remove as a workaround for now:

        #check
        grep -n 'LOG.propagate' /usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py
        #'patch'
        sudo cp /usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py /usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py.ORIG
        sudo sed -i 's/LOG.propagate = False//g' /usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py

Comment 17 errata-xmlrpc 2017-05-17 19:30:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245


Note You need to log in before you can comment on or make changes to this bug.