Bug 852400 - aeolus-upgrade fails to start postgresql and conductor-delayed_job service
aeolus-upgrade fails to start postgresql and conductor-delayed_job service
Status: CLOSED ERRATA
Product: CloudForms Cloud Engine
Classification: Red Hat
Component: aeolus-configure (Show other bugs)
1.1.0
Unspecified Unspecified
unspecified Severity high
: rc
: ---
Assigned To: Steve Linabery
Giulio Fidente
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-28 07:54 EDT by Aziza Karol
Modified: 2012-12-04 10:18 EST (History)
5 users (show)

See Also:
Fixed In Version: aeolus-conductor-0.13.14-1.el6cf
Doc Type: Bug Fix
Doc Text:
aeolus-upgrade ended with services running except for postgresql and conductor-delayed_job. The correct behavior was to leave no services running. This fix stops all services at the end of aeolus-upgrade. A future enhancement will prompt the user to optionally start all services at end of the upgrade process.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-12-04 10:18:01 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
err (152.10 KB, image/png)
2012-08-28 07:56 EDT, Aziza Karol
no flags Details
logs (38.89 KB, application/x-gzip)
2012-08-28 07:57 EDT, Aziza Karol
no flags Details

  None (edit)
Description Aziza Karol 2012-08-28 07:54:25 EDT
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.On fresh f16 test machine do yum install aeolus-all.The below aeolus-version will be installed.
#rpm -qa | grep aeolus
aeolus-conductor-doc-0.10.6-1.fc16.noarch
aeolus-all-0.10.6-1.fc16.noarch
rubygem-aeolus-image-0.5.0-1.fc16.noarch
rubygem-aeolus-cli-0.5.0-1.fc16.noarch
aeolus-configure-2.6.1-1.fc16.noarch
aeolus-conductor-0.10.6-1.fc16.noarch
aeolus-conductor-daemons-0.10.6-1.fc16.noarch
2.aeolus-configure -p ec2,mock
3.copy aeolus and katello repos in yum dir
4.yum update
following aeolus version got installed.
rpm -qa | grep aeolus
rubygem-aeolus-cli-0.7.0-0.20120828040024gitd64d64f.fc16.noarch
aeolus-configure-2.8.0-0.20120828040023gitbdcd549.fc16.noarch
aeolus-conductor-daemons-0.13.0-0.20120828080006git202d4f9.fc16.noarch
aeolus-all-0.13.0-0.20120828080006git202d4f9.fc16.noarch
rubygem-aeolus-image-0.6.0-0.20120828040028gite69e319.fc16.noarch
aeolus-conductor-0.13.0-0.20120828080006git202d4f9.fc16.noarch
aeolus-conductor-doc-0.13.0-0.20120828080006git202d4f9.fc16.noarch
5.stop aeolus services
6. ./aeolus-upgrade 
================================================================================
 Aeolus-conductor upgrade
================================================================================

1/3: Migrate aeolus-conductor rails database (0001_migrate_conductor_db.sh)
 Executes rake db:migrate for the aeolus-conductor database

Do you want to proceed? (y/n): y

Migrate aeolus-conductor rails database OK.

================================================================================

2/3: Execute aeolus-configure (0002_run_aeolus-configure.sh)
 Executes aeolus-configure. Note: this restarts aeolus services. Skip this step if you want to manually run aeolus-configure later.

Do you want to proceed? (y/n): y

Execute aeolus-configure OK.

================================================================================

3/3: Update aeolus-conductor Data Definition (0001_data_defn.sh)
 Adds a role definition to the existing seeded data definition. noop if role is already defined.

Do you want to proceed? (y/n): y

Update aeolus-conductor Data Definition OK.

================================================================================
Upgrade successful

  


Actual results:
The upgrade script shows successful however service like conductor-delayed_job and  postgresql failed to start.

# aeolus-check-services  |egrep 'post|delayed'
Redirecting to /bin/systemctl status  mongod.service
Redirecting to /bin/systemctl status  postgresql.service
Checking postgresql ...
 FAILURE: postgresql.service - PostgreSQL database server
	  Loaded: loaded (/lib/systemd/system/postgresql.service; enabled)
	 Process: 14719 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/postgresql.service
Redirecting to /bin/systemctl status  conductor-delayed_job.service
Redirecting to /bin/systemctl status  ntpd.service
Checking conductor-delayed_job ...
 FAILURE: conductor-delayed_job.service - aeolus delayed_job
	  Loaded: loaded (/lib/systemd/system/conductor-delayed_job.service; enabled)
	 Process: 15037 ExecStop=/usr/share/aeolus-conductor/script/delayed_job stop --pid-dir=/var/run/aeolus-conductor (code=exited, status=1/FAILURE)
	 Process: 14868 ExecStart=/usr/share/aeolus-conductor/script/delayed_job start -n 2 --pid-dir=/var/run/aeolus-conductor (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/conductor-delayed_job.service
[root@dell-pe2900-02 deltacloud-core]# 


Not able to access the application and displays errors on page.see attached screen shot.

errors in dbomatic log : ex:/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:950:in `initialize'

Attaching debug logs.


Expected results:
Upgrade should be successful


Additional info:
Comment 1 Aziza Karol 2012-08-28 07:56:06 EDT
Created attachment 607494 [details]
err
Comment 2 Aziza Karol 2012-08-28 07:57:04 EDT
Created attachment 607495 [details]
logs
Comment 3 Dave Johnson 2012-08-29 16:36:46 EDT
This is most certainly because 'aeolus-services stop' in the upgrade process stops postgres and the subsequent configure does nothing with postgres and delayed_jobs?

On a fresh v1.1 install, configure does not start delayed_jobs either?  Postgres is started which is not the case in a upgrade and I assume it deals with initial database setup on a fresh install.
Comment 4 Dave Johnson 2012-08-31 12:08:27 EDT
And this is occuring on RHEL...  I tried to upgrade from CFCE v1.0.1 to CFCE v1.1 and the same thing occurred.
Comment 5 Steve Linabery 2012-09-06 11:49:52 EDT
The problem here is that the upgrade stage that calls aeolus-configure should really not leave services running at the end of its execution. All upgrade stages should reasonably expect services to be stopped prior to executing; otherwise each stage would need to check service statuses. The correct behavior is for aeolus-upgrade to stop all services and leave them stopped at the end of the upgrade process.

Patch forthcoming.
Comment 7 Steve Linabery 2012-09-07 14:19:36 EDT
cb643d581523d69acdac8d36d5d937edac0900fe on master, in nightly build
Comment 8 Steve Linabery 2012-09-07 18:01:23 EDT
in build aeolus-conductor-0.13.3-1.el6cf
Comment 9 Steve Linabery 2012-09-07 18:03:44 EDT
setting to MODIFIED so I can attach to ET
Comment 10 Giulio Fidente 2012-09-26 10:48:13 EDT
works on rhel with aeolus-conductor-0.13.14-1.el6cf
Comment 12 errata-xmlrpc 2012-12-04 10:18:01 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-1516.html

Note You need to log in before you can comment on or make changes to this bug.