Description of problem: failed to depoly controller node. bluck at step 29 Actions::Staypuft::Host::AssertReportSuccess (error) [ 0.03s / 0.03s ] Error: Staypuft::Exception ERF42-7244 [Staypuft::Exception]: Latest Puppet run contains failures for host: 6 --- - /opt/rh/ruby193/root/usr/share/gems/gems/staypuft-0.5.9/app/lib/actions/staypuft/host/assert_report_success.rb:42:in `check_for_failures' - /opt/rh/ruby193/root/usr/share/gems/gems/staypuft-0.5.9/app/lib/actions/staypuft/host/assert_report_success.rb:31:in `assert_latest_report_success' - /opt/rh/ruby193/root/usr/share/gems/gems/staypuft-0.5.9/app/lib/actions/staypuft/host/assert_report_success.rb:17:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:443:in `block (3 levels) in execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/staypuft-0.5.9/app/lib/actions/staypuft/middleware/as_current_user.rb:14:in `block in run' - /opt/rh/ruby193/root/usr/share/gems/gems/staypuft-0.5.9/app/lib/actions/staypuft/middleware/as_current_user.rb:30:in `as_current_user' - /opt/rh/ruby193/root/usr/share/gems/gems/staypuft-0.5.9/app/lib/actions/staypuft/middleware/as_current_user.rb:14:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:30:in `with_progress_calculation' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:16:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/world.rb:30:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:442:in `block (2 levels) in execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in `catch' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in `block in execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `block in with_error_handling' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `catch' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `with_error_handling' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:436:in `execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:230:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in `with_meta_calculation' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:20:in `block in on_message' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in `block in assigns' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `tap' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `assigns' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in `match_value' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in `block in match' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `each' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `match' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:17:in `on_message' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:82:in `on_envelope' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:72:in `receive' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `block (2 levels) in run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `loop' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `block in run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `catch' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:13:in `block in initialize' and we also can see database error in /var/log/foreman-installer/foreman-installer.log Warning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false. (at /usr/share/ruby/vendor_ruby/puppet/type.rb:816:in `set_default') Error: /Stage[main]/Quickstack::Galera::Db/Mysql_database[neutron]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_database[nova]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_database[glance]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_database[heat]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_database[cinder]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_database[keystone]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_user[keystone@%]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_user[heat@%]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_user[glance@%]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_user[nova@%]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_user[cinder@%]: Provider mysql is not functional on this host Error: /Stage[main]/Quickstack::Galera::Db/Mysql_user[neutron@%]: Provider mysql is not functional on this host Error: Could not prefetch mysql_grant provider 'mysql': Command mysql is missing Error: Could not prefetch keystone_endpoint provider 'keystone': File: /etc/keystone/keystone.conf does not contain a section DEFAULT with the admin_token specified. Keystone types will not work if keystone is not correctly configured Error: Could not find a suitable provider for mysql_user Error: Could not find a suitable provider for vs_bridge Error: Could not find a suitable provider for mysql_database Error: Could not find a suitable provider for mongodb_replset [root@mac005056a42401 ~]# service mariadb status -l Redirecting to /bin/systemctl status -l mariadb.service mariadb.service - MariaDB database server Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled) Active: active (running) since Sun 2015-01-11 22:41:11 EST; 1min 36s ago Process: 24677 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID (code=exited, status=0/SUCCESS) Process: 24633 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n (code=exited, status=0/SUCCESS) Main PID: 24676 (mysqld_safe) CGroup: /system.slice/mariadb.service ├─24676 /bin/sh /usr/bin/mysqld_safe --basedir=/usr └─26457 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --wsrep-provider=/usr/lib64/galera/libgalera_smm.so --log-error=/var/log/mariadb/mariadb.log --open-files-limit=-1 --pid-file=/var/run/mariadb/mariadb.pid --socket=/var/lib/mysql/mysql.sock --port=3306 --wsrep_start_position=d1b0afe2-99b0-11e4-962a-d61cb23ff037:24225 Jan 11 22:41:11 mac005056a42401.rhops.com systemd[1]: Started MariaDB database server. Jan 11 22:41:12 mac005056a42401.rhops.com mysqld_safe[24676]: sed: -e expression #1, char 26: unknown option to `s' Jan 11 22:41:12 mac005056a42401.rhops.com mysqld_safe[24676]: sed: -e expression #1, char 25: unknown option to `s' Jan 11 22:41:16 mac005056a42401.rhops.com mysqld_safe[24676]: 150111 22:41:16 mysqld_safe Logging to '/var/log/mariadb/mariadb.log'. Jan 11 22:41:17 mac005056a42401.rhops.com mysqld_safe[24676]: 150111 22:41:17 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql Jan 11 22:41:17 mac005056a42401.rhops.com mysqld_safe[24676]: 150111 22:41:17 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.AzYpny' --pid-file='/var/lib/mysql/mac005056a42401.rhops.com-recover.pid' Jan 11 22:41:27 mac005056a42401.rhops.com mysqld_safe[24676]: 150111 22:41:27 mysqld_safe WSREP: Recovered position d1b0afe2-99b0-11e4-962a-d61cb23ff037:24225 [root@mac005056a42401 ~]# tail -200 /var/log/mariadb/mariadb.log 150111 22:41:29 Percona XtraDB (http://www.percona.com) 5.5.40-MariaDB-36.1 started; log sequence number 18999860 150111 22:41:29 [Note] Plugin 'FEEDBACK' is disabled. 150111 22:41:29 [Warning] Failed to setup SSL 150111 22:41:29 [Warning] SSL error: SSL_CTX_set_default_verify_paths failed
see multiple Errors for missing provider MySQL . "Provider mysql is not functional on this host". From checking the controllers it seems that mariadb was installed : [root@mac005056a42401 keystone(openstack_admin)]# rpm -qa | grep mariadb mariadb-libs-5.5.40-2.el7_0.x86_64 mariadb-galera-common-5.5.40-3.el7ost.x86_64 mariadb-galera-server-5.5.40-3.el7ost.x86_64 mariadb-5.5.40-2.el7_0.x86_64 test bed: rhel-osp-installer-client-0.5.4-1.el7ost.noarch puppet-3.6.2-2.el7.noarch /var/log/foreman-installer/foreman-installer.log error: [1;31mWarning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false. (at /usr/share/ruby/vendor_ruby/puppet/type.rb:816:in `set_default')[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_database[neutron]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_database[nova]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_database[glance]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_database[heat]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_database[cinder]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_database[keystone]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_user[keystone@%]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_user[heat@%]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_user[glance@%]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_user[nova@%]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_user[cinder@%]: Provider mysql is not functional on this host[0m [1;31mError: /Stage[main]/Quickstack::Galera::Db/Mysql_user[neutron@%]: Provider mysql is not functional on this host[0m [1;31mError: Could not prefetch mysql_grant provider 'mysql': Command mysql is missing[0m [1;31mError: Could not prefetch keystone_endpoint provider 'keystone': File: /etc/keystone/keystone.conf does not contain a section DEFAULT with the admin_token specified. Keystone types will not work if keystone is not correctly configured[0m [1;31mError: Could not find a suitable provider for mysql_user[0m database is correct.openstack service and endpoint has been created: [root@mac005056a42401 keystone(openstack_admin)]# mysql -u root keystone Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 1758 Server version: 5.5.40-MariaDB-wsrep MariaDB Server, wsrep_25.11.r4026 Copyright (c) 2000, 2014, Oracle, Monty Program Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [keystone]> show tables; +-----------------------+ | Tables_in_keystone | +-----------------------+ | assignment | | credential | | domain | | endpoint | | group | | id_mapping | | migrate_version | | policy | | project | | region | | revocation_event | | role | | service | | token | | trust | | trust_role | | user | | user_group_membership | +-----------------------+ 18 rows in set (0.00 sec) [root@mac005056a42401 keystone(openstack_admin)]# keystone service-list +----------------------------------+----------+---------------+---------------------------------+ | id | name | type | description | +----------------------------------+----------+---------------+---------------------------------+ | e787a2831469488396839b87e8dc661f | cinder | volume | Cinder Service | | 3934239094d749f9aec8db8f83fbcafa | cinderv2 | volumev2 | Cinder Service v2 | | 410f887fc27046908e8967b27d3710c7 | glance | image | Openstack Image Service | | bd79044e9f5340468e782761058648cc | heat | orchestration | Openstack Orchestration Service | | 23f98c21b31048ec81758ac4a90076f8 | keystone | identity | OpenStack Identity Service | | a39f9935c3424820bc3cffa74473c9b9 | neutron | network | Neutron Networking Service | | ce3df83ad88d4649baccd6551564c53f | nova | compute | Openstack Compute Service | | 944420bfe0f34af98c2a3719024f6951 | nova_ec2 | ec2 | EC2 Service | | 154b174cfab04b48811c0fef3407fbbf | novav3 | computev3 | Openstack Compute Service v3 | +----------------------------------+----------+---------------+---------------------------------+ [root@mac005056a42401 keystone(openstack_admin)]# keystone endpoint-list +----------------------------------+-----------+--------------------------------------------+--------------------------------------------+--------------------------------------------+----------------------------------+ | id | region | publicurl | internalurl | adminurl | service_id | +----------------------------------+-----------+--------------------------------------------+--------------------------------------------+--------------------------------------------+----------------------------------+ | 0b7ac12dafa8419e9b66d3fb26773bd0 | RegionOne | http://10.103.118.95:8773/services/Cloud | http://10.103.118.94:8773/services/Cloud | http://10.103.118.93:8773/services/Admin | 944420bfe0f34af98c2a3719024f6951 | | 23d5583240044ac9835c49bc44a8b57e | RegionOne | http://10.103.118.95:8774/v3 | http://10.103.118.94:8774/v3 | http://10.103.118.93:8774/v3 | 154b174cfab04b48811c0fef3407fbbf | | 319373c495f747dabaec0d8d257372aa | RegionOne | http://10.103.118.88:5000/v2.0 | http://10.103.118.87:5000/v2.0 | http://10.103.118.86:35357/v2.0 | 23f98c21b31048ec81758ac4a90076f8 | | 6993597bae094a4aacf9ec111dbda9aa | RegionOne | http://10.103.118.71:8776/v2/%(tenant_id)s | http://10.103.118.70:8776/v2/%(tenant_id)s | http://10.103.118.69:8776/v2/%(tenant_id)s | 3934239094d749f9aec8db8f83fbcafa | | 6b201c9b603f4bf2a312022ece3cf847 | RegionOne | http://10.103.118.71:8776/v1/%(tenant_id)s | http://10.103.118.70:8776/v1/%(tenant_id)s | http://10.103.118.69:8776/v1/%(tenant_id)s | e787a2831469488396839b87e8dc661f | | aaa163cf2a4c4992bc530145c6f8b3dc | RegionOne | http://10.103.118.78:8004/v1/%(tenant_id)s | http://10.103.118.77:8004/v1/%(tenant_id)s | http://10.103.118.76:8004/v1/%(tenant_id)s | bd79044e9f5340468e782761058648cc | | ab3d2db1de2e453f9cd6151180b73229 | RegionOne | http://10.103.118.95:8774/v2/%(tenant_id)s | http://10.103.118.94:8774/v2/%(tenant_id)s | http://10.103.118.93:8774/v2/%(tenant_id)s | ce3df83ad88d4649baccd6551564c53f | | dbfa725ca300459198831b636fde541d | RegionOne | http://10.103.118.92:9696/ | http://10.103.118.91:9696/ | http://10.103.118.90:9696/ | a39f9935c3424820bc3cffa74473c9b9 | | ee441728485d4c6eb789377912b097fc | RegionOne | http://10.103.118.75:9292 | http://10.103.118.74:9292 | http://10.103.118.73:9292 | 410f887fc27046908e8967b27d3710c7 | +----------------------------------+-----------+--------------------------------------------+--------------------------------------------+--------------------------------------------+----------------------------------+
Thanks for the BZ. This appears to be a duplicate of bug 1180322 *** This bug has been marked as a duplicate of bug 1180322 ***