Bug 983226 - An error occurred while running the storage node upgrade: Process exited with an error: 1 during rhqctl upgrade from 4.8 to 4.9
An error occurred while running the storage node upgrade: Process exited with...
Status: CLOSED CURRENTRELEASE
Product: JBoss Operations Network
Classification: JBoss
Component: Upgrade (Show other bugs)
JON 3.2
x86_64 Linux
unspecified Severity high
: ER01
: JON 3.2.0
Assigned To: John Sanda
Mike Foley
:
Depends On:
Blocks: 951619
  Show dependency treegraph
 
Reported: 2013-07-10 14:47 EDT by Armine Hovsepyan
Modified: 2015-09-02 20:01 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-01-02 15:33:38 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
rhqctl.log (1.19 KB, text/x-log)
2013-07-10 14:47 EDT, Armine Hovsepyan
no flags Details
rhqctl-upgrade-4.8-4.9.log (3.97 KB, text/x-log)
2013-07-22 06:09 EDT, Armine Hovsepyan
no flags Details

  None (edit)
Description Armine Hovsepyan 2013-07-10 14:47:37 EDT
Created attachment 771771 [details]
rhqctl.log

Description of problem:
An error occurred while running the storage node upgrade: Process exited with an error: 1  during rhqctl upgrade from 4.8 to 4.9

Version-Release number of selected component (if applicable):
http://sourceforge.net/projects/rhq/files/rhq/rhq-4.8.0/
4.9 master build 2414

How reproducible:
2 out of 2

Steps to Reproduce:
1. run rhqctl install for 4.8
2. run ./rhq-server-4.9.0-SNAPSHOT/bin/rhqctl upgrade --from-server-dir=/home/hudson/rhq-server-4.8.0



Actual results:
An error occurred while running the storage node upgrade: Process exited with an error: 1  

Expected results:
rhq server with storage and agent is upgraded


Additional info:
log attached
Comment 1 John Sanda 2013-07-17 15:21:24 EDT
I introduced this regression with a change I made in the storage installer to look for the new cassandra-jvm.properties file instead of cassandra-env.sh. The JMX port is defined in cassandra-env.sh in RHQ 4.8. The port is defined in cassandra-jvm.properties in RHQ 4.9 and going forward. The issue is that cassandra-jvm.properties does not exist in 4.8.0. I need to update the installer upgrade code to migrate from cassandra-env.sh to cassandra-jvm.properties.
Comment 2 John Sanda 2013-07-19 13:52:21 EDT
The storage installer has been updated along with some automated tests so that correctly handles both cassandra-env.sh (for 4.8 installs) and cassandra-jvm.properties for later installs. The fix will be in build 2443 of the rhq-master Jenkins job.

master branch commit hash: 656cc606625
Comment 3 Armine Hovsepyan 2013-07-22 06:08:27 EDT
Upgrade is not working. Getting the following exception:
 "java.lang.RuntimeException: org.apache.cassandra.exceptions.ConfigurationException: Cannot add column family 'users' to non existing keyspace 'system_auth'"

Please get log attached.
Comment 4 Armine Hovsepyan 2013-07-22 06:09:13 EDT
Created attachment 776845 [details]
rhqctl-upgrade-4.8-4.9.log
Comment 5 John Sanda 2013-07-22 14:27:02 EDT
I am not exactly sure of the cause for that error but I suspect it is related to the fact that you cannot directly upgrade from a RHQ 4.8 install. This is because of the removal of native libraries from Cassandra that were present in 4.8. Because those libraries were packaged with Cassandra in RHQ 4.8, table compression may have been used if the native library support was available. Upgrading from 4.8 requires running a patch script against your 4.8 installation. I need to write up some docs on that script. I will update this bug with the link to those docs when I have them ready.
Comment 6 John Sanda 2013-07-22 15:22:22 EDT
Some instructions will be included in the RHQ 4.9 release notes which have not yet been created. For now, you run the patch script directly from the source tree in the master branch. The script is,

<rhq-source>/etc/scripts/rhq48-storage-patch/rhq48-storage-patch.sh

Instructions are provided as comments at the top of the script.
Comment 7 Armine Hovsepyan 2013-07-23 07:09:17 EDT
Verified.

storage patch worked, upgrade was possible from 4.8 to 4.9 after that. 
Could you please include the patch in rhq archive?

Thanks in advance.

Note You need to log in before you can comment on or make changes to this bug.