Bug 1316588 - Upgrade undercloud fails on keystone error
Upgrade undercloud fails on keystone error
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud (Show other bugs)
8.0 (Liberty)
Unspecified Unspecified
unspecified Severity high
: ---
: 8.0 (Liberty)
Assigned To: Brad P. Crochet
Udi
: Automation, AutomationBlocker
: 1318029 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-10 09:37 EST by Adriano Petrich
Modified: 2016-04-07 17:49 EDT (History)
9 users (show)

See Also:
Fixed In Version: instack-undercloud-2.2.6-1.el7ost
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-07 17:49:13 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
httpd logs (1.55 KB, application/x-gzip)
2016-03-10 11:41 EST, Marios Andreou
no flags Details
/var/log/messages (1.67 MB, application/x-gzip)
2016-03-10 11:42 EST, Marios Andreou
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 291292 None None None 2016-03-10 12:14 EST
Red Hat Product Errata RHEA-2016:0604 normal SHIPPED_LIVE Red Hat OpenStack Platform 8 director Enhancement Advisory 2016-04-07 21:03:56 EDT

  None (edit)
Description Adriano Petrich 2016-03-10 09:37:27 EST
Description of problem:
When doing an upgrade from 7.3 to 8 the undercloud upgrade fails

Version-Release number of selected component (if applicable):
upgrade 7.3 to 8

How reproducible:
very

Steps to Reproduce:
1. do a full install on 7.3 (undercloud and overcloud
2. do a rhos-release 8 -d 
3. update the packages with yum upgrade -y
4. source stackrc
5. openstack undercloud install #upgrade the undercloud

Actual results:
It fails

Notice: /File[/etc/ironic/ironic.conf]/seluser: seluser changed 'unconfined_u' to 'system_u'
Notice: /Stage[main]/Keystone::Service/Service[keystone]/ensure: ensure changed 'running' to 'stopped'

Broadcast message from systemd-journald@instack.localdomain (Thu 2016-03-10 09:18:09 EST):

haproxy[9505]: proxy keystone_admin has no server available!


Broadcast message from systemd-journald@instack.localdomain (Thu 2016-03-10 09:18:09 EST):

haproxy[9505]: proxy keystone_public has no server available!

Error: Could not start Service[httpd]: Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
Wrapped exception:
Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
Error: /Stage[main]/Apache::Service/Service[httpd]/ensure: change from stopped to running failed: Could not start Service[httpd]: Execution of '/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
Notice: /Stage[main]/Apache::Service/Service[httpd]: Triggered 'refresh' from 3 events
Notice: /Stage[main]/Keystone/Anchor[keystone_started]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone/Anchor[keystone_started]: Skipping because of failed dependencies
Error: Could not prefetch keystone_user provider 'openstack': undefined method `collect' for nil:NilClass
Notice: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_user[neutron]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_user[neutron]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_user[heat]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_user[heat]: Skipping because of failed dependencies
Error: Could not prefetch keystone_service provider 'openstack': undefined method `collect' for nil:NilClass
Notice: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_service[glance::image]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_service[glance::image]: Skipping because of failed dependencies
Error: Could not prefetch keystone_role provider 'openstack': undefined method `collect' for nil:NilClass
Notice: /Stage[main]/Ceilometer::Keystone::Auth/Keystone_role[ResellerAdmin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ceilometer::Keystone::Auth/Keystone_role[ResellerAdmin]: Skipping because of failed dependencies
Notice: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_service[ironic::baremetal]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_service[ironic::baremetal]: Skipping because of failed dependencies
Notice: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_user[aodh]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_user[aodh]: Skipping because of failed dependencies
Notice: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_user[nova]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_user[nova]: Skipping because of failed dependencies
Notice: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_service[aodh::alarming]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_service[aodh::alarming]: Skipping because of failed dependencies
Notice: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_user[glance]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_user[glance]: Skipping because of failed dependencies
Notice: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova v3 service, user novav3]/Keystone_service[novav3::computev3]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova v3 service, user novav3]/Keystone_service[novav3::computev3]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Auth/Keystone_role[heat_stack_user]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Auth/Keystone_role[heat_stack_user]: Skipping because of failed dependencies
Notice: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_user[ironic]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_user[ironic]: Skipping because of failed dependencies
Notice: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_service[nova::compute]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_service[nova::compute]: Skipping because of failed dependencies
Notice: /Stage[main]/Swift::Keystone::Auth/Keystone_role[swiftoperator]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Swift::Keystone::Auth/Keystone_role[swiftoperator]: Skipping because of failed dependencies
Notice: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_user[ceilometer]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_user[ceilometer]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_service[neutron::network]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_service[neutron::network]: Skipping because of failed dependencies
Notice: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_service[ceilometer::metering]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_service[ceilometer::metering]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/notify_nova_on_port_status_changes]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/notify_nova_on_port_status_changes]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/region_name]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/region_name]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/project_name]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/project_name]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/auth_plugin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/auth_plugin]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/send_events_interval]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/send_events_interval]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/username]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/username]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/password]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/password]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/nova_region_name]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/nova_region_name]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/project_domain_id]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/project_domain_id]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/notify_nova_on_port_data_changes]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/notify_nova_on_port_data_changes]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/tenant_name]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/tenant_name]: Skipping because of failed dependencies
Notice: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_service[swift::object-store]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_service[swift::object-store]: Skipping because of failed dependencies
Notice: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_service[keystone::identity]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_service[keystone::identity]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/user_domain_id]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/user_domain_id]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_service[heat::orchestration]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_service[heat::orchestration]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/auth_url]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[nova/auth_url]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/nova_url]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server::Notifications/Neutron_config[DEFAULT/nova_url]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Agents::Dhcp/Service[neutron-dhcp-service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Agents::Dhcp/Service[neutron-dhcp-service]: Skipping because of failed dependencies
Notice: /Stage[main]/Main/Keystone_service[nova::computev3]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Main/Keystone_service[nova::computev3]: Skipping because of failed dependencies
Notice: Puppet::Provider::Openstack: endpoint service is unavailable. Will retry for up to 9 seconds.
Error: Could not prefetch keystone_endpoint provider 'openstack': undefined method `each' for nil:NilClass
Notice: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova v3 service, user novav3]/Keystone_endpoint[regionOne/novav3::computev3]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova v3 service, user novav3]/Keystone_endpoint[regionOne/novav3::computev3]: Skipping because of failed dependencies
Notice: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_endpoint[regionOne/aodh::alarming]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_endpoint[regionOne/aodh::alarming]: Skipping because of failed dependencies
Notice: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_endpoint[regionOne/keystone::identity]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_endpoint[regionOne/keystone::identity]: Skipping because of failed dependencies
Notice: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_endpoint[regionOne/nova::compute]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_endpoint[regionOne/nova::compute]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_endpoint[regionOne/heat::orchestration]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_endpoint[regionOne/heat::orchestration]: Skipping because of failed dependencies
Notice: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_endpoint[regionOne/glance::image]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_endpoint[regionOne/glance::image]: Skipping because of failed dependencies
Notice: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_endpoint[regionOne/swift::object-store]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_endpoint[regionOne/swift::object-store]: Skipping because of failed dependencies
Notice: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_endpoint[regionOne/ironic::baremetal]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_endpoint[regionOne/ironic::baremetal]: Skipping because of failed dependencies
Notice: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_endpoint[regionOne/ceilometer::metering]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_endpoint[regionOne/ceilometer::metering]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_endpoint[regionOne/neutron::network]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_endpoint[regionOne/neutron::network]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Db::Sync/Exec[neutron-db-sync]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Db::Sync/Exec[neutron-db-sync]: Skipping because of failed dependencies
Notice: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_user[swift]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_user[swift]: Skipping because of failed dependencies
Notice: Puppet::Provider::Openstack: project service is unavailable. Will retry for up to 9 seconds.
Error: Could not prefetch keystone_tenant provider 'openstack': undefined method `collect' for nil:NilClass
Notice: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[service]: Skipping because of failed dependencies
Notice: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[admin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[admin]: Skipping because of failed dependencies
Notice: /Stage[main]/Keystone::Roles::Admin/Keystone_role[admin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_role[admin]: Skipping because of failed dependencies
Notice: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_user_role[aodh@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Aodh::Keystone::Auth/Keystone::Resource::Service_identity[aodh]/Keystone_user_role[aodh@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_user_role[nova@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Nova::Keystone::Auth/Keystone::Resource::Service_identity[nova service, user nova]/Keystone_user_role[nova@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_user_role[heat@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Auth/Keystone::Resource::Service_identity[heat]/Keystone_user_role[heat@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_user_role[swift@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Swift::Keystone::Auth/Keystone::Resource::Service_identity[swift]/Keystone_user_role[swift@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_user_role[ceilometer@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_user_role[ceilometer@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_user_role[ironic@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Ironic::Keystone::Auth/Keystone::Resource::Service_identity[ironic]/Keystone_user_role[ironic@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Keystone::Roles::Admin/Keystone_user[admin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_user[admin]: Skipping because of failed dependencies
Notice: /Stage[main]/Keystone::Roles::Admin/Keystone_user_role[admin@admin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_user_role[admin@admin]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_user_domain_name]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_user_domain_name]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_domain_admin_password]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_domain_admin_password]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_domain_admin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_domain_admin]: Skipping because of failed dependencies
Notice: Puppet::Provider::Openstack: domain service is unavailable. Will retry for up to 9 seconds.
Error: Could not prefetch keystone_domain provider 'openstack': undefined method `collect' for nil:NilClass
Notice: /Stage[main]/Heat::Keystone::Domain/Keystone_domain[heat_stack]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Domain/Keystone_domain[heat_stack]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Domain/Keystone_user[heat_admin::heat_stack]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Domain/Keystone_user[heat_admin::heat_stack]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Keystone::Domain/Keystone_user_role[heat_admin::heat_stack@::heat_stack]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Keystone::Domain/Keystone_user_role[heat_admin::heat_stack@::heat_stack]: Skipping because of failed dependencies
Notice: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_user_role[glance@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Glance::Keystone::Auth/Keystone::Resource::Service_identity[glance]/Keystone_user_role[glance@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Glance::Registry/Service[glance-registry]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Glance::Registry/Service[glance-registry]: Skipping because of failed dependencies
Notice: /Stage[main]/Glance::Api/Service[glance-api]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Glance::Api/Service[glance-api]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_user_role[neutron@service]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_user_role[neutron@service]: Skipping because of failed dependencies
Notice: /Stage[main]/Neutron::Server/Service[neutron-server]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Neutron::Server/Service[neutron-server]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Deps/Anchor[heat::config::end]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Deps/Anchor[heat::config::end]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Deps/Anchor[heat::db::begin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Deps/Anchor[heat::db::begin]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Mysql_database[heat]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Mysql_database[heat]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_%]/Mysql_user[heat@%]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_%]/Mysql_user[heat@%]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_%]/Mysql_grant[heat@%/heat.*]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_%]/Mysql_grant[heat@%/heat.*]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_192.0.2.1]/Mysql_user[heat@192.0.2.1]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_192.0.2.1]/Mysql_user[heat@192.0.2.1]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_192.0.2.1]/Mysql_grant[heat@192.0.2.1/heat.*]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_192.0.2.1]/Mysql_grant[heat@192.0.2.1/heat.*]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Deps/Anchor[heat::db::end]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Deps/Anchor[heat::db::end]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Deps/Anchor[heat::dbsync::begin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Deps/Anchor[heat::dbsync::begin]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Db::Sync/Exec[heat-dbsync]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Db::Sync/Exec[heat-dbsync]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Deps/Anchor[heat::dbsync::end]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Deps/Anchor[heat::dbsync::end]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Deps/Anchor[heat::service::begin]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Deps/Anchor[heat::service::begin]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Api_cfn/Service[heat-api-cfn]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Api_cfn/Service[heat-api-cfn]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Engine/Service[heat-engine]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Engine/Service[heat-engine]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Api/Service[heat-api]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Api/Service[heat-api]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Api_cloudwatch/Service[heat-api-cloudwatch]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Api_cloudwatch/Service[heat-api-cloudwatch]: Skipping because of failed dependencies
Notice: /Stage[main]/Heat::Deps/Anchor[heat::service::end]: Dependency Service[httpd] has failures: true
Warning: /Stage[main]/Heat::Deps/Anchor[heat::service::end]: Skipping because of failed dependencies
Notice: Finished catalog run in 118.21 seconds
+ rc=6
+ set -e
+ echo 'puppet apply exited with exit code 6'
puppet apply exited with exit code 6
+ '[' 6 '!=' 2 -a 6 '!=' 0 ']'
+ exit 6
[2016-03-10 09:19:29,913] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 6]

[2016-03-10 09:19:29,913] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 807, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 691, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 366, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1




Expected results:
upgrade works

Additional info:


there's an warning up the install process that is

Warning: Scope(Class[Keystone::Endpoint]): In Mitaka, the default value of $keystone::endpoint::version will change to ''. To avoid this warning, please set the version parameter.

Not sure if that is relevant or not
Comment 2 Emilien Macchi 2016-03-10 11:32:23 EST
FYI the Warning Scope(Class[Keystone::Endpoint]) is not relevant. mmagr added this warning in puppet-keystone.
Comment 3 Adriano Petrich 2016-03-10 11:36:48 EST
# systemctl status httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/httpd.service.d
           └─openstack-dashboard.conf
   Active: failed (Result: exit-code) since Thu 2016-03-10 09:18:19 EST; 2h 16min ago
     Docs: man:httpd(8)
           man:apachectl(8)
  Process: 10502 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE)
  Process: 10499 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE)
  Process: 10399 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py compress --force (code=exited, status=0/SUCCESS)
  Process: 10370 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear (code=exited, status=0/SUCCESS)
 Main PID: 10499 (code=exited, status=1/FAILURE)

Mar 10 09:18:19 instack.localdomain httpd[10499]: (98)Address already in use: AH00072: make_sock: could not bind to address [::]:35357
Mar 10 09:18:19 instack.localdomain httpd[10499]: (98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:35357
Mar 10 09:18:19 instack.localdomain httpd[10499]: no listening sockets available, shutting down
Mar 10 09:18:19 instack.localdomain httpd[10499]: AH00015: Unable to open logs
Mar 10 09:18:19 instack.localdomain systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Mar 10 09:18:19 instack.localdomain kill[10502]: kill: cannot find process ""
Mar 10 09:18:19 instack.localdomain systemd[1]: httpd.service: control process exited, code=exited status=1
Mar 10 09:18:19 instack.localdomain systemd[1]: Failed to start The Apache HTTP Server.
Mar 10 09:18:19 instack.localdomain systemd[1]: Unit httpd.service entered failed state.
Mar 10 09:18:19 instack.localdomain systemd[1]: httpd.service failed.

# cat error_log 
[Thu Mar 10 05:18:05.876608 2016] [core:notice] [pid 32026] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Thu Mar 10 05:18:05.877725 2016] [suexec:notice] [pid 32026] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Mar 10 05:18:05.886882 2016] [auth_digest:notice] [pid 32026] AH01757: generating secret for digest authentication ...
[Thu Mar 10 05:18:05.892957 2016] [mpm_prefork:notice] [pid 32026] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Thu Mar 10 05:18:05.892999 2016] [core:notice] [pid 32026] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Mar 10 05:18:19.391863 2016] [mpm_prefork:notice] [pid 32026] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Mar 10 05:18:31.567266 2016] [core:notice] [pid 32548] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Thu Mar 10 05:18:31.568935 2016] [suexec:notice] [pid 32548] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Mar 10 05:18:31.578477 2016] [auth_digest:notice] [pid 32548] AH01757: generating secret for digest authentication ...
[Thu Mar 10 05:18:31.583899 2016] [mpm_prefork:notice] [pid 32548] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Thu Mar 10 05:18:31.583923 2016] [core:notice] [pid 32548] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Mar 10 05:25:12.616067 2016] [mpm_prefork:notice] [pid 32548] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Mar 10 05:25:24.030143 2016] [core:notice] [pid 6335] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Thu Mar 10 05:25:24.031137 2016] [suexec:notice] [pid 6335] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Mar 10 05:25:24.040256 2016] [auth_digest:notice] [pid 6335] AH01757: generating secret for digest authentication ...
[Thu Mar 10 05:25:24.045257 2016] [mpm_prefork:notice] [pid 6335] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Thu Mar 10 05:25:24.045287 2016] [core:notice] [pid 6335] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Mar 10 05:25:27.576710 2016] [mpm_prefork:notice] [pid 6335] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Mar 10 05:25:39.310434 2016] [core:notice] [pid 6566] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Thu Mar 10 05:25:39.312170 2016] [suexec:notice] [pid 6566] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Mar 10 05:25:39.326976 2016] [auth_digest:notice] [pid 6566] AH01757: generating secret for digest authentication ...
[Thu Mar 10 05:25:39.334254 2016] [mpm_prefork:notice] [pid 6566] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Thu Mar 10 05:25:39.334288 2016] [core:notice] [pid 6566] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Mar 10 06:58:19.462589 2016] [mpm_prefork:notice] [pid 6566] AH00170: caught SIGWINCH, shutting down gracefully
Comment 4 Marios Andreou 2016-03-10 11:41 EST
Created attachment 1134956 [details]
httpd logs
Comment 5 Marios Andreou 2016-03-10 11:42 EST
Created attachment 1134957 [details]
/var/log/messages
Comment 6 Marios Andreou 2016-03-10 11:43:39 EST
added httpd logs and /var/log/messages on request from irc
Comment 7 Alexander Chuzhoy 2016-03-10 18:47:26 EST
Reproduced the issue:
Environment: 
openstack-tripleo-heat-templates-0.8.10-2.el7ost.noarch
instack-undercloud-2.2.5-1.el7ost.noarch
openstack-puppet-modules-7.0.14-1.el7ost.noarch
Comment 8 Brad P. Crochet 2016-03-15 20:19:23 EDT
*** Bug 1318029 has been marked as a duplicate of this bug. ***
Comment 10 Udi 2016-04-03 09:56:09 EDT
With the latest puddles the upgrade is successful. Verified in python-tripleoclient-0.3.4-1.el7ost.noarch.
Comment 12 errata-xmlrpc 2016-04-07 17:49:13 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0604.html

Note You need to log in before you can comment on or make changes to this bug.