Bug 1083625 - Some operations broken during live upgrade from Havana to Icehouse
Summary: Some operations broken during live upgrade from Havana to Icehouse
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 4.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z5
: 4.0
Assignee: Nikola Dipanov
QA Contact: Dafna Ron
URL:
Whiteboard:
Depends On: 1066112 1150104
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-04-02 15:18 UTC by Russell Bryant
Modified: 2022-07-09 07:07 UTC (History)
9 users (show)

Fixed In Version: openstack-nova-2013.2.3-1.el6ost
Doc Type: Bug Fix
Doc Text:
Cause: Some RPC messages were not sending the correct data format when the version cap was set (for live upgrades situation). Consequence: During live upgrade from Havana to Icehouse, when there are compute nodes running different versions of the code - some operations would fail on the compute hosts that were not upgraded. Fix: Send proper data version in all cases. Result: All of the RPC calls succeed in the live migration scenario.
Clone Of:
Environment:
Last Closed: 2014-10-22 17:21:47 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1301436 0 None None None Never
OpenStack gerrit 84755 0 None None None Never
Red Hat Issue Tracker OSP-16517 0 None None None 2022-07-09 07:07:11 UTC
Red Hat Product Errata RHSA-2014:1689 0 normal SHIPPED_LIVE Important: openstack-nova security, bug fix, and enhancement update 2014-10-22 21:21:07 UTC

Description Russell Bryant 2014-04-02 15:18:42 UTC
The following havana-specific commit was merged a while ago:

commit 2c63cae921d024f39f7d14459873d20caa5cf9e0
Author: Russell Bryant <rbryant>
Date: Thu Dec 5 15:34:58 2013 -0500

    Allow resizes and migrates during live upgrade

    This patch completes the changes needed to havana's compute rpc api to
    allow live upgrades to icehouse. Specifically, these changes are needed
    to allow resizes and migrates to work. Before havana compute nodes get
    put in a mixed environment with icehouse compute nodes, they need to be
    speaking version 3.0 messages.

    Change-Id: If09bd38c9d8c3beb5b95107c497699dec47aa3ef
    Closes-bug: #1258253

Unfortunately, it doesn't work completely as expected. There are some messages that claim to be sending 3.0 messages, but have the data in the wrong format.

Comment 1 Nikola Dipanov 2014-04-14 10:20:26 UTC
This patch was merged to upstream stable/havana and picked up by the 2013.2.3 rebase. Moving to modified so we can get QE to look at this.

Comment 4 Ami Jeain 2014-04-23 08:28:38 UTC
no capacity to test it during the Havana release

Comment 14 Dafna Ron 2014-10-16 16:13:01 UTC
i'm failing QE to this bug since I cannot install the latest 4.0 due to selinux dependency issues + https://bugzilla.redhat.com/show_bug.cgi?id=1149000 

Applying Puppet manifests                         [ ERROR ]

ERROR : Error appeared during Puppet run: 10.35.117.18_prescript.pp
err: /Stage[main]//Package[openstack-selinux]/ensure: change from
absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y
install openstack-selinux' returned 1: Error: Package:
openstack-selinux-0.1.5-5.el6ost.noarch (rhelosp-4.0)
You will find full trace in log
/var/tmp/packstack/20141013-141151-1LMi4U/manifests/10.35.117.18_prescript.pp.log


Please check log file
/var/tmp/packstack/20141013-141151-1LMi4U/openstack-setup.log for more
information

Comment 15 Dafna Ron 2014-10-16 16:13:58 UTC
correct bug is: 1066112

Comment 16 Lon Hohberger 2014-10-17 15:14:36 UTC
I am pretty sure that this bug and bug 1066112 are not related.

It looks like the prescript failed - meaning it didn't get to mysql installation.  This would occur if you attempted to run packstack on 6.5 with the latest openstack-selinux (which *REQUIRES* 6.6).

If you intend to test on RHEL 6.5, you must preinstall openstack-selinux-0.1.3.

Comment 17 Dafna Ron 2014-10-17 15:23:49 UTC
1. if you look at bug:1149000 which was closed a adup - it will not work on 6.6 as well. 
2. I need to test upgrade and part of the live upgrade is testing if we can work in a mixed 6.5 and 6.6 env. - are you saying that this is not supported?

Comment 18 Lon Hohberger 2014-10-21 16:54:30 UTC
You can certainly run things in RHEL 6.5 for upgrade purposes.  It's just that there's no differentiation between "6.5" and "6.6" for OpenStack packages, they all just use the "RHEL Server" channel - which assumes the latest.

If running RHEL OSP 4 with 6.5, you must preinstall openstack-selinux-0.1.3-2.el6ost.

If running RHEL OSP 5 with 6.5, you must preinstall openstack-selinux-0.1.4-2.el6ost.

Alternatively, installing openstack

Comment 19 Lon Hohberger 2014-10-21 17:08:56 UTC
Oops, alternatively, installing selinux-policy from 6.6 would work.  On RHEL OSP 5, you could also install openstack-selinux-0.1.5-1.el6ost

Comment 20 Lon Hohberger 2014-10-21 17:12:19 UTC
I guess I should point out that all my comments are specifically directed at comment #14

Comment 21 Dafna Ron 2014-10-22 09:28:31 UTC
(In reply to Lon Hohberger from comment #18)
> You can certainly run things in RHEL 6.5 for upgrade purposes.  It's just
> that there's no differentiation between "6.5" and "6.6" for OpenStack
> packages, they all just use the "RHEL Server" channel - which assumes the
> latest.
> 
> If running RHEL OSP 4 with 6.5, you must preinstall
> openstack-selinux-0.1.3-2.el6ost.
> 
> If running RHEL OSP 5 with 6.5, you must preinstall
> openstack-selinux-0.1.4-2.el6ost.
> 
> Alternatively, installing openstack

an admin would most likely upgrade in stages :) 
I can now install rhel6.5 however, have yet managed to be able to migrate instances

Comment 23 errata-xmlrpc 2014-10-22 17:21:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2014-1689.html


Note You need to log in before you can comment on or make changes to this bug.