Description of problem: Run aeolus-configure, then aeolus-configure -p rhevm. Logged to Web UI (conductor) and default RHEVM provider was not listed. The file /etc/aeolus-configure/nodes/rhevm_configure had the correct parameters. Several errors were listed in /var/log/aeolus-configure/aeolus-configure.log but were not self explanatory (see below). Run aeolus-cleanup and detected an error related to an unsuccessful connection to the Postgres database in the /var/log/aeolus-configure/aeolus-cleanup.log (see below). Version-Release number of selected component (if applicable): aeolus-configure-2.5.10-10-1.el6cf.noarch How reproducible: Running aeolus-configure when the database is down or unreachable. Steps to Reproduce: 1. aoulus-configure 2. aeolus-configure -p rhevm 3. aeolus-cleanup 4. Repeat Actual results: RHEVM provider not available in Web UI under ADMINISTER > Cloud Resource Providers. Expected results: RHEVM provider available in Web UI under ADMINISTER > Cloud Resource Providers. Additional info: The problem was diagnosed as an issue with database seeding. Here's a transcript of an email thread with the Cloud Forms development team while troubleshooting this issue: The problem seemed to be an unresponsive Postgres database. This message appeared in aeolus-cleanup.log after running aeolus-cleanup: /var/log/aeolus-configure/aeolus-cleanup.log notice:/Stage[main]/Aeolus::Conductor::Disabled /Rails::Drop::Db[drop_aeolus_database]/Exec[drop_rails_database]/returns: Couldn't drop conductor : #<PGError: could not connect to server: Connection refused The aeulous-configure command that was run after the aeolus-cleanup showed the following errors: /var/log/aeolus-configure/aeolus-configure.log /Stage[main]/Aeolus::Profiles::Rhevm Aeolus::Profiles::Rhevm::Instance[default]/Aeolus::Conductor::Provider[rhevm-default]/Web_request[provider-rhevm-default] (notice): Dependency Exec[seed_rails_database] has failures: true /Stage[main]/Aeolus::Profiles::Rhevm Aeolus::Profiles::Rhevm::Instance[default]/Aeolus::Conductor::Provider[rhevm-default]/Web_request[provider-rhevm-default] (warning): Skipping because of failed dependencies Under the seeds part you can see the following: /Stage[main]/Aeolus::Conductor/Rails::Seed::Db[seed_aeolus_database]/Exec[seed_rails_database]/returns (notice): rake aborted! /Stage[main]/Aeolus::Conductor/Rails::Seed::Db[seed_aeolus_database]/Exec[seed_rails_database]/returns (notice): Validation failed: Cloud Name has already been taken So it looks like the db is already seeded, but for some reason configure is trying to re-seed it. aeolus-configure determines whether or not it needs to run the seeds by touching /var/lib/aeolus-conductor/production.seed, if that file exists, it doesn't try to run the seeds (in this case, the file didn't exist.) Bottom line: We should have a more robust way to determine if we need to run seeds or not. Quick fix: Make sure Postgres is running (and reachable) and run . aeolus-cleanup . aeolus-configure . aeolus-configure -p rhevm
Running aeolus-configure when the database is down or unreachable. Steps to Reproduce: 1. aoulus-configure 2. aeolus-configure -p rhevm 3. aeolus-cleanup 4. Repeat [root@dhcp201-113 ~]# service postgresql stop Stopping postgresql service: [ OK ] [root@dhcp201-113 ~]# aeolus-configure && aeolus-configure -p rhevm && aeolus-cleanup -y Launching aeolus configuration recipe... notice: /File[/usr/share/aeolus-conductor/config/initializers/secret_token.rb]/content: content changed '{md5}31d3bd26512a2ef60da464140b0b4d89' to '{md5}7fcf7618d7fdbc8090f0486d5ac7920b' notice: /Stage[main]/Postgres::Server/Service[postgresql]/ensure: ensure changed 'stopped' to 'running' notice: /Stage[main]/Aeolus::Conductor/Aeolus::Rails::Create::Db[create_aeolus_database]/Exec[create_rails_database]/returns: executed successfully notice: /Stage[main]/Aeolus::Conductor/Aeolus::Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: executed successfully notice: /Stage[main]/Aeolus::Conductor/Aeolus::Conductor::Destroy_temp_admins[before]/Exec[destroy_temp_admin-before]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Temp_admin[temporary-administrative-user-e89f5529e65fc542c0ccd991fa37dc01f745a7073051792c]/Exec[create_temp_admin]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Temp_admin[temporary-administrative-user-e89f5529e65fc542c0ccd991fa37dc01f745a7073051792c]/Exec[grant_temp_admin_privs]/returns: executed successfully notice: /Stage[main]/Aeolus::Deltacloud::Core/Exec[deltacloud-core-startup-wait]/returns: executed successfully notice: /Stage[main]/Aeolus::Conductor/Service[conductor-delayed_job]/ensure: ensure changed 'stopped' to 'running' notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Login[temporary-administrative-user-e89f5529e65fc542c0ccd991fa37dc01f745a7073051792c]/Web_request[temporary-administrative-user-e89f5529e65fc542c0ccd991fa37dc01f745a7073051792c-conductor-login]/post: post changed '' to 'https://localhost/conductor/user_session' notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Login[temporary-administrative-user-e89f5529e65fc542c0ccd991fa37dc01f745a7073051792c]/Exec[decrement_login_counter]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Hwp[small-x86_64]/Web_request[hwp-small-x86_64]/post: post changed '' to 'https://localhost/conductor/hardware_profiles' notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Logout[temporary-administrative-user-e89f5529e65fc542c0ccd991fa37dc01f745a7073051792c]/Web_request[temporary-administrative-user-e89f5529e65fc542c0ccd991fa37dc01f745a7073051792c-conductor-logout]/get: get changed '' to 'https://localhost/conductor/logout' notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Destroy_temp_admins[after]/Exec[destroy_temp_admin-after]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Create_bucket[aeolus]/Exec[create-bucket-aeolus]/returns: executed successfully notice: Finished catalog run in 34.67 seconds Launching aeolus configuration recipe... notice: /File[/usr/share/aeolus-conductor/config/initializers/secret_token.rb]/content: content changed '{md5}7fcf7618d7fdbc8090f0486d5ac7920b' to '{md5}8721e23fa99e879c48d0c6503451b2fd' notice: /Stage[main]/Aeolus::Conductor/Aeolus::Rails::Create::Db[create_aeolus_database]/Exec[create_rails_database]/returns: executed successfully notice: /Stage[main]/Aeolus::Conductor/Aeolus::Rails::Migrate::Db[migrate_aeolus_database]/Exec[migrate_rails_database]/returns: executed successfully notice: /Stage[main]/Aeolus::Conductor/Aeolus::Conductor::Destroy_temp_admins[before]/Exec[destroy_temp_admin-before]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Temp_admin[temporary-administrative-user-9cb58782e4439f510063ad71013113595ed2a22e4892070d]/Exec[create_temp_admin]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Temp_admin[temporary-administrative-user-9cb58782e4439f510063ad71013113595ed2a22e4892070d]/Exec[grant_temp_admin_privs]/returns: executed successfully notice: the RHEV NFS export is on the correct storage domain and has type 'export' => true notice: /Stage[main]/Aeolus::Profiles::Rhevm/Aeolus::Profiles::Rhevm::Instance[default]/Aeolus::Rhevm::Validate[RHEV NFS export validation for default]/Notify[RHEV NFS export validation for default]/message: defined 'message' as 'the RHEV NFS export is on the correct storage domain and has type 'export' => true' notice: /Stage[main]/Aeolus::Deltacloud::Core/Exec[deltacloud-core-startup-wait]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Login[temporary-administrative-user-9cb58782e4439f510063ad71013113595ed2a22e4892070d]/Web_request[temporary-administrative-user-9cb58782e4439f510063ad71013113595ed2a22e4892070d-conductor-login]/post: post changed '' to 'https://localhost/conductor/user_session' notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Login[temporary-administrative-user-9cb58782e4439f510063ad71013113595ed2a22e4892070d]/Exec[decrement_login_counter]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Hwp[small-x86_64]/Web_request[hwp-small-x86_64]/post: post changed '' to 'https://localhost/conductor/hardware_profiles' notice: /Stage[main]/Aeolus::Profiles::Rhevm/Aeolus::Profiles::Rhevm::Instance[default]/Aeolus::Conductor::Provider[rhevm-default]/Web_request[provider-rhevm-default]/post: post changed '' to 'https://localhost/conductor/providers.xml' notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Logout[temporary-administrative-user-9cb58782e4439f510063ad71013113595ed2a22e4892070d]/Web_request[temporary-administrative-user-9cb58782e4439f510063ad71013113595ed2a22e4892070d-conductor-logout]/get: get changed '' to 'https://localhost/conductor/logout' notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Conductor::Destroy_temp_admins[after]/Exec[destroy_temp_admin-after]/returns: executed successfully notice: /Stage[main]/Aeolus::Profiles::Common/Aeolus::Create_bucket[aeolus]/Exec[create-bucket-aeolus]/returns: executed successfully notice: Finished catalog run in 25.67 seconds notice: /Stage[main]/Aeolus::Image-factory::Disabled/Service[imagefactory]/ensure: ensure changed 'running' to 'stopped' notice: /File[/etc/rsyslog.d/aeolus.conf]/ensure: removed notice: /Stage[main]/Aeolus::Deltacloud::Disabled/Service[deltacloud-core]/ensure: ensure changed 'running' to 'stopped' notice: /Stage[main]/Aeolus::Iwhd::Disabled/Exec[clean_iwhd]/returns: executed successfully notice: /Stage[main]/Aeolus::Iwhd::Disabled/Service[iwhd]/ensure: ensure changed 'running' to 'stopped' notice: /Stage[main]/Aeolus::Iwhd::Disabled/Service[mongod]/ensure: ensure changed 'running' to 'stopped' notice: /Stage[main]/Aeolus::Conductor::Disabled/Service[conductor-delayed_job]/ensure: ensure changed 'running' to 'stopped' notice: /Stage[main]/Aeolus::Image-factory::Disabled/Exec[remove_imagefactory_tmp_files]/returns: executed successfully notice: /Stage[main]/Aeolus::Image-factory::Disabled/Exec[remove_aeolus_templates]/returns: executed successfully notice: /Stage[main]/Aeolus::Deltacloud::Disabled/Exec[remove_deltacloud_tmp_files]/returns: executed successfully notice: /File[/var/lib/aeolus-conductor]/ensure: removed notice: /Stage[main]/Aeolus::Conductor::Disabled/Service[conductor-dbomatic]/ensure: ensure changed 'running' to 'stopped' notice: /Stage[main]/Aeolus::Conductor::Disabled/Service[aeolus-conductor]/ensure: ensure changed 'running' to 'stopped' notice: /Stage[main]/Aeolus::Conductor::Disabled/Service[httpd]/ensure: ensure changed 'running' to 'stopped' notice: /Stage[main]/Aeolus::Conductor::Disabled/Aeolus::Rails::Drop::Db[drop_aeolus_database]/Exec[drop_rails_database]/returns: executed successfully notice: /Stage[main]/Aeolus::Conductor::Disabled/Postgres::User[aeolus]/Exec[drop_aeolus_postgres_user]/returns: executed successfully notice: Finished catalog run in 11.72 seconds Problem not observed.Verified on: [root@dhcp201-113 ~]# rpm -qa|grep aeolus rubygem-aeolus-cli-0.7.2-1.el6cf.noarch aeolus-configure-2.8.7-1.el6cf.noarch aeolus-conductor-daemons-0.13.14-1.el6cf.noarch rubygem-aeolus-image-0.3.0-12.el6.noarch aeolus-all-0.13.14-1.el6cf.noarch aeolus-conductor-doc-0.13.14-1.el6cf.noarch aeolus-conductor-0.13.14-1.el6cf.noarch