Bug 785334

Summary:	error in aeolus-configure
Product:	[Retired] CloudForms Cloud Engine	Reporter:	Shveta <ssachdev>
Component:	aeolus-configure	Assignee:	Jan Provaznik <jprovazn>
Status:	CLOSED CURRENTRELEASE	QA Contact:	wes hayutin <whayutin>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	1.0.0	CC:	akarol, deltacloud-maint, jlaska, jprovazn, slinaber, ssachdev
Target Milestone:	beta6
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Shveta 2012-01-28 05:35:36 UTC

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
aeolus-configure -p mock
1. err: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: change from notrun to 0 failed: /usr/bin/rake db:migrate returned 1 instead of one of [0] at /usr/share/aeolus-configure/modules/aeolus/manifests/rails.pp:31

2.
3.
  
Actual results:


Expected results:


Additional info:

rpm -qa|grep aeolus
aeolus-conductor-doc-0.8.0-14.el6.noarch
aeolus-configure-2.5.0-11.el6.noarch
rubygem-aeolus-image-0.3.0-5.el6.noarch
rubygem-aeolus-cli-0.3.0-7.el6.noarch
aeolus-conductor-0.8.0-14.el6.noarch
aeolus-all-0.8.0-14.el6.noarch
aeolus-conductor-daemons-0.8.0-14.el6.noarch

Comment 1 Aziza Karol 2012-01-29 14:37:27 UTC

I am also getting the same error:

# aeolus-configure  -d -v -p ec2 
Launching aeolus configuration recipe...
info: Loading facts in oauth
info: Applying configuration version '1327847318'
notice: /File[/rsyslog]/seltype: seltype changed 'etc_runtime_t' to 'default_t'
notice: /File[/data]/seltype: seltype changed 'etc_runtime_t' to 'default_t'
notice: /Stage[main]/Aeolus::Deltacloud::Core/Exec[deltacloud-core-startup-wait]/returns: executed successfully
notice: /Stage[main]/Apache/Exec[permit-http-networking]/returns: executed successfully
info: FileBucket adding {md5}f33b6bc852ab262e583fa65e0cd89b49
info: /File[/usr/share/aeolus-conductor/config/initializers/secret_token.rb]: Filebucketed /usr/share/aeolus-conductor/config/initializers/secret_token.rb to puppet with sum f33b6bc852ab262e583fa65e0cd89b49
notice: /File[/usr/share/aeolus-conductor/config/initializers/secret_token.rb]/content: content changed '{md5}f33b6bc852ab262e583fa65e0cd89b49' to '{md5}012fd241d1f8d5440e4316d3f10070cf'
notice: /Stage[main]/Aeolus::Conductor/Rails::Create::Db[create_aeolus_database]/Exec[create_rails_database]/returns: conductor already exists
notice: /Stage[main]/Aeolus::Conductor/Rails::Create::Db[create_aeolus_database]/Exec[create_rails_database]/returns: Using gem require instead of bundler
notice: /Stage[main]/Aeolus::Conductor/Rails::Create::Db[create_aeolus_database]/Exec[create_rails_database]/returns: executed successfully
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: NOTICE:  CREATE TABLE will create implicit sequence "frontend_realms_id_seq" for serial column "frontend_realms.id"
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "frontend_realms_pkey" for table "frontend_realms"
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: NOTICE:  CREATE TABLE will create implicit sequence "realm_backend_targets_id_seq" for serial column "realm_backend_targets.id"
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "realm_backend_targets_pkey" for table "realm_backend_targets"
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: rake aborted!
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: An error has occurred, this and all later migrations canceled:
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: 
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: PGError: ERROR:  current transaction is aborted, commands ignored until end of transaction block
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: :             SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc, a.attnotnull
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:               FROM pg_attribute a LEFT JOIN pg_attrdef d
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:                 ON a.attrelid = d.adrelid AND a.attnum = d.adnum
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:              WHERE a.attrelid = '"realms"'::regclass
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:                AND a.attnum > 0 AND NOT a.attisdropped
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:              ORDER BY a.attnum
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: 
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: Tasks: TOP => db:migrate
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: (See full trace by running task with --trace)
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: Using gem require instead of bundler
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: ==  FrontendRealms: migrating =================================================
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: -- drop_table(:realm_map)
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:    -> 0.0011s
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: -- create_table(:frontend_realms)
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:    -> 0.0084s
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: -- create_table(:realm_backend_targets)
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:    -> 0.0031s
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: -- rename_column(:instances, :realm_id, :frontend_realm_id)
notice: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns:    -> 0.0005s
err: /Stage[main]/Aeolus::Conductor/Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: change from notrun to 0 failed: /usr/bin/rake db:migrate returned 1 instead of one of [0] at /usr/share/aeolus-configure/modules/aeolus/manifests/rails.pp:31
notice: /Stage[main]/Aeolus::Conductor/Rails::Seed::Db[seed_aeolus_database]/Exec[seed_rails_database]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Conductor/Rails::Seed::Db[seed_aeolus_database]/Exec[seed_rails_database]: Skipping because of failed dependencies
notice: /File[/var/lib/aeolus-conductor/production.seed]: Dependency Exec[migrate_rails_database] has failures: true
warning: /File[/var/lib/aeolus-conductor/production.seed]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Site_admin[admin]/Exec[create_site_admin_user]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Site_admin[admin]/Exec[create_site_admin_user]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Site_admin[admin]/Exec[grant_site_admin_privs]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Site_admin[admin]/Exec[grant_site_admin_privs]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Conductor/Service[aeolus-conductor]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Conductor/Service[aeolus-conductor]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Login[admin]/Web_request[admin-conductor-login]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Login[admin]/Web_request[admin-conductor-login]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Login[admin]/Exec[decrement_login_counter]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Login[admin]/Exec[decrement_login_counter]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-sa-east-1]/Web_request[provider-ec2-sa-east-1]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-sa-east-1]/Web_request[provider-ec2-sa-east-1]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Hwp[hwp1]/Web_request[hwp-hwp1]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Hwp[hwp1]/Web_request[hwp-hwp1]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-us-west-1]/Web_request[provider-ec2-us-west-1]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-us-west-1]/Web_request[provider-ec2-us-west-1]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-eu-west-1]/Web_request[provider-ec2-eu-west-1]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-eu-west-1]/Web_request[provider-ec2-eu-west-1]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-ap-southeast-1]/Web_request[provider-ec2-ap-southeast-1]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-ap-southeast-1]/Web_request[provider-ec2-ap-southeast-1]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-us-west-2]/Web_request[provider-ec2-us-west-2]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-us-west-2]/Web_request[provider-ec2-us-west-2]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-ap-northeast-1]/Web_request[provider-ec2-ap-northeast-1]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-ap-northeast-1]/Web_request[provider-ec2-ap-northeast-1]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-us-east-1]/Web_request[provider-ec2-us-east-1]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Provider[ec2-us-east-1]/Web_request[provider-ec2-us-east-1]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Logout[admin]/Web_request[admin-conductor-logout]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Conductor::Logout[admin]/Web_request[admin-conductor-logout]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Conductor/Service[conductor-dbomatic]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Conductor/Service[conductor-dbomatic]: Skipping because of failed dependencies
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Create_bucket[aeolus]/Exec[create-bucket-aeolus]/returns:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Create_bucket[aeolus]/Exec[create-bucket-aeolus]/returns:                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0create-bucket-aeolus]/returns: 
notice: /Stage[main]/Aeolus::Profiles::Ec2/Aeolus::Create_bucket[aeolus]/Exec[create-bucket-aeolus]/returns: executed successfully
notice: /Stage[main]/Aeolus::Image-factory/Service[imagefactory]: Dependency Exec[migrate_rails_database] has failures: true
warning: /Stage[main]/Aeolus::Image-factory/Service[imagefactory]: Skipping because of failed dependencies
notice: Finished catalog run in 27.76 seconds

[root@dell-pe2950-1 ~]# aeolus-check-services 

Checking mongod ...
 Success: mongod (pid 7384) is running...

Checking iwhd ...
 Success: iwhd (pid  7475) is running...

Checking postgresql ...
 Success: postmaster (pid  7661) is running...

Checking httpd ...
 Success: httpd (pid  7416) is running...

Checking qpidd ...
 Success: qpidd (pid  7334) is running...

Checking deltacloud-core ...
 Success: deltacloud-core (pid  7745) is running...

Checking libvirtd ...
 Success: libvirtd (pid  7816) is running...

Checking aeolus-conductor ...
 FAILURE: is stopped

Checking conductor-dbomatic ...
 FAILURE: dbomatic is stopped

Checking imagefactory ...
 FAILURE: imagefactory is stopped

Checking ntpd ...
 Success: ntpd (pid  7788) is running...


conductor=> select *  from   users;
 id | login | email | crypted_password | password_salt | persistence_token | single_access_token | perishable_token | first_name | last_name | quota_id | log
in_count | failed_login_count | last_request_at | current_login_at | last_login_at | current_login_ip | last_login_ip | created_at | updated_at 
----+-------+-------+------------------+---------------+-------------------+---------------------+------------------+------------+-----------+----------+----
---------+--------------------+-----------------+------------------+---------------+------------------+---------------+------------+------------
(0 rows)


Not able to login as no user is  created.


# rpm -qa |   grep aeolus
aeolus-conductor-daemons-0.8.0-14.el6.noarch
rubygem-aeolus-cli-0.3.0-7.el6.noarch
rubygem-aeolus-image-0.3.0-5.el6.noarch
aeolus-all-0.8.0-14.el6.noarch
aeolus-configure-2.5.0-11.el6.noarch
aeolus-conductor-0.8.0-14.el6.noarch
aeolus-conductor-doc-0.8.0-14.el6.noarch

Comment 2 wes hayutin 2012-01-29 14:44:25 UTC

Yikes.. Shveta, Aziza hopefully our qe wiki still has the rake command for adding a user.  It's worth a try but may also fail

Comment 3 Mike Orazi 2012-01-30 04:33:19 UTC

Looks like the underlying migrate is failing.

Comment 4 Jan Provaznik 2012-01-30 10:17:33 UTC

the problem is caused by commit d95ea3770bb0fb6126fb7eb9d2316d731516b2ad which adds new HABTM associoations to ProviderAccount and Realm models. Some of earlier migrations run a code related to these models -> models are loaded and try to access this habtm assoc table which doesn't exist yet - it's created later in the migration chain.

I don't think there is a clean solution for this . Least dirty hack I know about is to create a migration which creates missing HABTM assoc and prepend it at the beginning of the migration chain. Any better idea?

Comment 5 Jan Provaznik 2012-01-30 11:44:33 UTC

the patch sent: https://fedorahosted.org/pipermail/aeolus-devel/2012-January/008456.html

Comment 6 Jan Provaznik 2012-01-30 12:24:34 UTC

pushed, commit: 98969205af078341dc3eda362a8417fda78edeb8

Comment 7 James Laska 2012-01-30 14:07:01 UTC

(In reply to comment #6)
> pushed, commit: 98969205af078341dc3eda362a8417fda78edeb8

Tested this patch with aeolus-conductor-0.8.0-14.el6 and aeolus-configure-2.5.0-11.el6.  Verified fix.

Will wait until fix lands in an official build before marking as VERIFIED.

Comment 9 Steve Linabery 2012-01-30 16:14:57 UTC

9896920 in aeolus-conductor-0.8.0-15

Comment 10 Shveta 2012-01-31 11:14:22 UTC

Fixed ..Verified in 

aeolus-conductor-doc-0.8.0-16.el6.noarch
aeolus-configure-2.5.0-11.el6.noarch
aeolus-conductor-daemons-0.8.0-16.el6.noarch
rubygem-aeolus-image-0.3.0-6.el6.noarch
aeolus-all-0.8.0-16.el6.noarch
aeolus-conductor-0.8.0-16.el6.noarch
rubygem-aeolus-cli-0.3.0-7.el6.noarch