Bug 852400 - aeolus-upgrade fails to start postgresql and conductor-delayed_job service
Summary: aeolus-upgrade fails to start postgresql and conductor-delayed_job service
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: CloudForms Cloud Engine
Classification: Retired
Component: aeolus-configure
Version: 1.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
Assignee: Steve Linabery
QA Contact: Giulio Fidente
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-08-28 11:54 UTC by Aziza Karol
Modified: 2012-12-04 15:18 UTC (History)
5 users (show)

Fixed In Version: aeolus-conductor-0.13.14-1.el6cf
Clone Of:
Environment:
Last Closed: 2012-12-04 15:18:01 UTC
Embargoed:


Attachments (Terms of Use)
err (152.10 KB, image/png)
2012-08-28 11:56 UTC, Aziza Karol
no flags Details
logs (38.89 KB, application/x-gzip)
2012-08-28 11:57 UTC, Aziza Karol
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2012:1516 0 normal SHIPPED_LIVE CloudForms Cloud Engine 1.1 update 2012-12-04 19:51:45 UTC

Description Aziza Karol 2012-08-28 11:54:25 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.On fresh f16 test machine do yum install aeolus-all.The below aeolus-version will be installed.
#rpm -qa | grep aeolus
aeolus-conductor-doc-0.10.6-1.fc16.noarch
aeolus-all-0.10.6-1.fc16.noarch
rubygem-aeolus-image-0.5.0-1.fc16.noarch
rubygem-aeolus-cli-0.5.0-1.fc16.noarch
aeolus-configure-2.6.1-1.fc16.noarch
aeolus-conductor-0.10.6-1.fc16.noarch
aeolus-conductor-daemons-0.10.6-1.fc16.noarch
2.aeolus-configure -p ec2,mock
3.copy aeolus and katello repos in yum dir
4.yum update
following aeolus version got installed.
rpm -qa | grep aeolus
rubygem-aeolus-cli-0.7.0-0.20120828040024gitd64d64f.fc16.noarch
aeolus-configure-2.8.0-0.20120828040023gitbdcd549.fc16.noarch
aeolus-conductor-daemons-0.13.0-0.20120828080006git202d4f9.fc16.noarch
aeolus-all-0.13.0-0.20120828080006git202d4f9.fc16.noarch
rubygem-aeolus-image-0.6.0-0.20120828040028gite69e319.fc16.noarch
aeolus-conductor-0.13.0-0.20120828080006git202d4f9.fc16.noarch
aeolus-conductor-doc-0.13.0-0.20120828080006git202d4f9.fc16.noarch
5.stop aeolus services
6. ./aeolus-upgrade 
================================================================================
 Aeolus-conductor upgrade
================================================================================

1/3: Migrate aeolus-conductor rails database (0001_migrate_conductor_db.sh)
 Executes rake db:migrate for the aeolus-conductor database

Do you want to proceed? (y/n): y

Migrate aeolus-conductor rails database OK.

================================================================================

2/3: Execute aeolus-configure (0002_run_aeolus-configure.sh)
 Executes aeolus-configure. Note: this restarts aeolus services. Skip this step if you want to manually run aeolus-configure later.

Do you want to proceed? (y/n): y

Execute aeolus-configure OK.

================================================================================

3/3: Update aeolus-conductor Data Definition (0001_data_defn.sh)
 Adds a role definition to the existing seeded data definition. noop if role is already defined.

Do you want to proceed? (y/n): y

Update aeolus-conductor Data Definition OK.

================================================================================
Upgrade successful

  


Actual results:
The upgrade script shows successful however service like conductor-delayed_job and  postgresql failed to start.

# aeolus-check-services  |egrep 'post|delayed'
Redirecting to /bin/systemctl status  mongod.service
Redirecting to /bin/systemctl status  postgresql.service
Checking postgresql ...
 FAILURE: postgresql.service - PostgreSQL database server
	  Loaded: loaded (/lib/systemd/system/postgresql.service; enabled)
	 Process: 14719 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/postgresql.service
Redirecting to /bin/systemctl status  conductor-delayed_job.service
Redirecting to /bin/systemctl status  ntpd.service
Checking conductor-delayed_job ...
 FAILURE: conductor-delayed_job.service - aeolus delayed_job
	  Loaded: loaded (/lib/systemd/system/conductor-delayed_job.service; enabled)
	 Process: 15037 ExecStop=/usr/share/aeolus-conductor/script/delayed_job stop --pid-dir=/var/run/aeolus-conductor (code=exited, status=1/FAILURE)
	 Process: 14868 ExecStart=/usr/share/aeolus-conductor/script/delayed_job start -n 2 --pid-dir=/var/run/aeolus-conductor (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/conductor-delayed_job.service
[root@dell-pe2900-02 deltacloud-core]# 


Not able to access the application and displays errors on page.see attached screen shot.

errors in dbomatic log : ex:/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:950:in `initialize'

Attaching debug logs.


Expected results:
Upgrade should be successful


Additional info:

Comment 1 Aziza Karol 2012-08-28 11:56:06 UTC
Created attachment 607494 [details]
err

Comment 2 Aziza Karol 2012-08-28 11:57:04 UTC
Created attachment 607495 [details]
logs

Comment 3 Dave Johnson 2012-08-29 20:36:46 UTC
This is most certainly because 'aeolus-services stop' in the upgrade process stops postgres and the subsequent configure does nothing with postgres and delayed_jobs?

On a fresh v1.1 install, configure does not start delayed_jobs either?  Postgres is started which is not the case in a upgrade and I assume it deals with initial database setup on a fresh install.

Comment 4 Dave Johnson 2012-08-31 16:08:27 UTC
And this is occuring on RHEL...  I tried to upgrade from CFCE v1.0.1 to CFCE v1.1 and the same thing occurred.

Comment 5 Steve Linabery 2012-09-06 15:49:52 UTC
The problem here is that the upgrade stage that calls aeolus-configure should really not leave services running at the end of its execution. All upgrade stages should reasonably expect services to be stopped prior to executing; otherwise each stage would need to check service statuses. The correct behavior is for aeolus-upgrade to stop all services and leave them stopped at the end of the upgrade process.

Patch forthcoming.

Comment 7 Steve Linabery 2012-09-07 18:19:36 UTC
cb643d581523d69acdac8d36d5d937edac0900fe on master, in nightly build

Comment 8 Steve Linabery 2012-09-07 22:01:23 UTC
in build aeolus-conductor-0.13.3-1.el6cf

Comment 9 Steve Linabery 2012-09-07 22:03:44 UTC
setting to MODIFIED so I can attach to ET

Comment 10 Giulio Fidente 2012-09-26 14:48:13 UTC
works on rhel with aeolus-conductor-0.13.14-1.el6cf

Comment 12 errata-xmlrpc 2012-12-04 15:18:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-1516.html


Note You need to log in before you can comment on or make changes to this bug.