Bug 1397233

Summary: rhel-osp-director: OSP9.0 undercloud deployment doesn't complete: Error: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: Failed to call refresh: Command exceeded timeout
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: rhosp-directorAssignee: Angus Thomas <athomas>
Status: CLOSED WORKSFORME QA Contact: Omri Hochman <ohochman>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 9.0 (Mitaka)CC: dbecker, jslagle, mburns, morazi, rhel-osp-director-maint, sasha
Target Milestone: async   
Target Release: 9.0 (Mitaka)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-23 19:35:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Alexander Chuzhoy 2016-11-22 00:56:10 UTC
rhel-osp-director:   OSP9.0  undercloud deployment doesn't complete: Error: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: Failed to call refresh: Command exceeded timeout

Environment:
instack-undercloud-4.0.0-15.el7ost.noarch


Steps to reproduce:
Try to deploy undercloud


Result:

Notice: /Stage[main]/Apache/Apache::Vhost[default]/Concat[15-default.conf]/File[15-default.conf]/ensure: defined content as '{md5}a430bf4e003be964b419e7aea251c6c4'                                                  
Notice: /Stage[main]/Tripleo::Loadbalancer/Haproxy::Balancermember[swift_proxy_server]/Concat::Fragment[haproxy-swift_proxy_server_balancermember_swift_proxy_server]/File[/var/lib/puppet/concat/_etc_haproxy_haproxy.cfg/fragments/20-swift_proxy_server-01-swift_proxy_server_haproxy-swift_proxy_server_balancermember_swift_proxy_server]/ensure: defined content as '{md5}d09dacd99101419fdb870b1feed54c87'                         
Notice: /Stage[main]/Apache::Mod::Dav_fs/File[dav_fs.conf]/ensure: defined content as '{md5}899a57534f3d84efa81887ec93c90c9b'                                                                                        
Notice: /Stage[main]/Tripleo::Loadbalancer/Haproxy::Balancermember[glance_api]/Concat::Fragment[haproxy-glance_api_balancermember_glance_api]/File[/var/lib/puppet/concat/_etc_haproxy_haproxy.cfg/fragments/20-glance_api-01-glance_api_haproxy-glance_api_balancermember_glance_api]/ensure: defined content as '{md5}3ad3f624fc92e61a74a2990ac3486d57'                                                                                 
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/File[/var/www/cgi-bin/aodh]/ensure: created                                                                                            
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/File[aodh_wsgi]/ensure: defined content as '{md5}736d8e48b4142275f5515ded6ebdfd56'                                                     
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat[10-aodh_wsgi.conf]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf]/ensure: created                      
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat[10-aodh_wsgi.conf]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments.concat.out]/ensure: created 
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat[10-aodh_wsgi.conf]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments]/ensure: created            
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-file_footer]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/999_aodh_wsgi-file_footer]/ensure: defined content as '{md5}e27b2525783e590ca1820f1e2118285d'                                                                                                                                      
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-directories]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/60_aodh_wsgi-directories]/ensure: defined content as '{md5}aaf32c3d7774278408a3e9fd8cc691d0'                                                                                                                                       
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-access_log]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/100_aodh_wsgi-access_log]/ensure: defined content as '{md5}58043fc6adf64216e4c0779ec2116fad'                                                                                                                                        
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-docroot]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/10_aodh_wsgi-docroot]/ensure: defined content as '{md5}c614880518e83bfe6ca56aa2016a40d5'                                                                                                                                               
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-apache-header]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/0_aodh_wsgi-apache-header]/ensure: defined content as '{md5}514205d0d47eeafde304e6e2ed847765'                                                                                                                                    
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-logging]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/80_aodh_wsgi-logging]/ensure: defined content as '{md5}0e57a94cc02481de496d2b1c8d4c5b93'                                                                                                                                               
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Apache::Listen[192.0.2.1:8042]/Concat::Fragment[Listen 192.0.2.1:8042]/File[/var/lib/puppet/concat/_etc_httpd_conf_ports.conf/fragments/10_Listen 192.0.2.1_8042]/ensure: defined content as '{md5}1fe7d729b2dfeb31f00f7d8da35fa05c'                                                                                               
Notice: /Stage[main]/Apache/Concat[/etc/httpd/conf/ports.conf]/Exec[concat_/etc/httpd/conf/ports.conf]/returns: executed successfully                                                                                
Notice: /Stage[main]/Apache/Concat[/etc/httpd/conf/ports.conf]/Exec[concat_/etc/httpd/conf/ports.conf]: Triggered 'refresh' from 7 events                                                                            
Notice: /Stage[main]/Apache/Concat[/etc/httpd/conf/ports.conf]/File[/etc/httpd/conf/ports.conf]/ensure: defined content as '{md5}64d45d45277569f50ab8c7f1232ffeb6'                                                   
Notice: /Stage[main]/Apache/File[/etc/httpd/conf/httpd.conf]/content: content changed '{md5}f5e7449c0f17bc856e86011cb5d152ba' to '{md5}d5d339c65678a5c660174610b3ea4821'                                             
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-serversignature]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/90_aodh_wsgi-serversignature]/ensure: defined content as '{md5}9bf5a458783ab459e5043e1cdf671fa7'                                                                                                                               
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat[10-aodh_wsgi.conf]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments.concat]/ensure: created     
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat::Fragment[aodh_wsgi-wsgi]/File[/var/lib/puppet/concat/10-aodh_wsgi.conf/fragments/260_aodh_wsgi-wsgi]/ensure: defined content as '{md5}de870449510f4074ae0439b1a0591a00'                                                                                                                                                    
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat[10-aodh_wsgi.conf]/Exec[concat_10-aodh_wsgi.conf]/returns: executed successfully                       
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat[10-aodh_wsgi.conf]/Exec[concat_10-aodh_wsgi.conf]: Triggered 'refresh' from 10 events                  
Notice: /Stage[main]/Aodh::Wsgi::Apache/Openstacklib::Wsgi::Apache[aodh_wsgi]/Apache::Vhost[aodh_wsgi]/Concat[10-aodh_wsgi.conf]/File[10-aodh_wsgi.conf]/ensure: defined content as '{md5}bcef578f7ae21ae6faa18f3c14c2146a'                                                                                                                                                                                                               
Notice: /Stage[main]/Ceilometer/Ceilometer_config[publisher/metering_secret]/ensure: created                                                                                                                         
Notice: /Stage[main]/Swift::Storage::All/Swift::Storage::Server[6000]/Concat[/etc/swift/object-server.conf]/File[/var/lib/puppet/concat/_etc_swift_object-server.conf/fragments.concat]/ensure: created              
Notice: /Stage[main]/Swift::Storage::All/Swift::Storage::Server[6000]/Concat[/etc/swift/object-server.conf]/Exec[concat_/etc/swift/object-server.conf]/returns: executed successfully                                
Notice: /Stage[main]/Swift::Storage::All/Swift::Storage::Server[6000]/Concat[/etc/swift/object-server.conf]/Exec[concat_/etc/swift/object-server.conf]: Triggered 'refresh' from 5 events                            
Notice: /Stage[main]/Swift::Storage::All/Swift::Storage::Server[6000]/Concat[/etc/swift/object-server.conf]/File[/etc/swift/object-server.conf]/content: content changed '{md5}43f14d676b28bc8111d6100e06e9a8bf' to '{md5}f01fab0807c5b1395ff003394d7540d4'                                                                                                                                                                               
Notice: /Stage[main]/Swift::Storage::All/Swift::Storage::Server[6000]/Concat[/etc/swift/object-server.conf]/File[/etc/swift/object-server.conf]/owner: owner changed 'root' to 'swift'                               
Notice: /Stage[main]/Swift::Storage::All/Swift::Storage::Server[6000]/Concat[/etc/swift/object-server.conf]/File[/etc/swift/object-server.conf]/mode: mode changed '0640' to '0644'                                  
Notice: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Swift::Service[swift-object-server]/Service[swift-object-server]/ensure: ensure changed 'stopped' to 'running'                           
Notice: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Swift::Service[swift-object-replicator]/Service[swift-object-replicator]/ensure: ensure changed 'stopped' to 'running'                   
Notice: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Swift::Service[swift-object-auditor]/Service[swift-object-auditor]/ensure: ensure changed 'stopped' to 'running'                         
Notice: /Stage[main]/Swift::Storage::Object/Swift::Service[swift-object-updater]/Service[swift-object-updater]/ensure: ensure changed 'stopped' to 'running'                                                         
Notice: /Stage[main]/Keystone/Keystone_config[token/revoke_by_id]/ensure: created                                                                                                                                    
Notice: /Stage[main]/Tripleo::Loadbalancer/Keepalived::Instance[51]/Concat::Fragment[keepalived_51]/File[/var/lib/puppet/concat/_etc_keepalived_keepalived.conf/fragments/50_keepalived_51]/ensure: defined content as '{md5}d2378066152ff1a49697d06f5fbf7cda'                                                                                                                                                                            
Notice: /Stage[main]/Tripleo::Loadbalancer/Keepalived::Instance[52]/Concat::Fragment[keepalived_52]/File[/var/lib/puppet/concat/_etc_keepalived_keepalived.conf/fragments/50_keepalived_52]/ensure: defined content as '{md5}e2c5e7ff41c00a742177799547cf02de'                                                                                                                                                                            
Notice: /Stage[main]/Keepalived::Config/Concat[/etc/keepalived/keepalived.conf]/Exec[concat_/etc/keepalived/keepalived.conf]/returns: executed successfully                                                          
Notice: /Stage[main]/Keepalived::Config/Concat[/etc/keepalived/keepalived.conf]/Exec[concat_/etc/keepalived/keepalived.conf]: Triggered 'refresh' from 6 events                                                      
Notice: /Stage[main]/Keepalived::Config/Concat[/etc/keepalived/keepalived.conf]/File[/etc/keepalived/keepalived.conf]/content: content changed '{md5}e79dca9e58978e8035e7d4ad25e2ce67' to '{md5}a0c41b494beddc09dcd63c56d979cafa'                                                                                                                                                                                                         
Notice: /Stage[main]/Keepalived::Service/Service[keepalived]/ensure: ensure changed 'stopped' to 'running'                                                                                                           
Notice: /Stage[main]/Tripleo::Loadbalancer/Haproxy::Balancermember[rabbitmq]/Concat::Fragment[haproxy-rabbitmq_balancermember_rabbitmq]/File[/var/lib/puppet/concat/_etc_haproxy_haproxy.cfg/fragments/20-rabbitmq-01-rabbitmq_haproxy-rabbitmq_balancermember_rabbitmq]/ensure: defined content as '{md5}054ce1396ee30bcc15e78bec5dd7bb8d'                                                                                               
Notice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]/returns: executed successfully                               
Notice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]: Triggered 'refresh' from 31 events                          
Notice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/content: content changed '{md5}1f337186b0e1ba5ee82760cb437fb810' to '{md5}bca5f1228a1f0dcfdcfae1b2dac781b0'                                                                                                                                                                              
Notice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Service[haproxy]/Service[haproxy]/ensure: ensure changed 'stopped' to 'running'                                                                     
Notice: /Stage[main]/Swift::Storage::Container/Swift::Service[swift-container-updater]/Service[swift-container-updater]/ensure: ensure changed 'stopped' to 'running'                                                
Notice: /Stage[main]/Neutron::Server/Neutron_config[DEFAULT/l3_ha]/ensure: created                                                                                                                                   
Notice: /Stage[main]/Glance::Notify::Rabbitmq/Glance_api_config[oslo_messaging_rabbit/rabbit_host]/ensure: created                                                                                                   
Notice: /Stage[main]/Glance::Api/Glance_cache_config[DEFAULT/auth_url]/ensure: created                                                                                                                               
Error: /Stage[main]/Glance::Db::Sync/Exec[glance-manage db_sync]: Failed to call refresh: Command exceeded timeout                                                                                                   
Error: /Stage[main]/Glance::Db::Sync/Exec[glance-manage db_sync]: Command exceeded timeout                                                                                                                           
Wrapped exception:                                                                                                                                                                                                   
execution expired                                                                                                                                                                                                    
Notice: /Stage[main]/Keystone::Wsgi::Apache/Apache::Vhost[keystone_wsgi_main]/Concat::Fragment[keystone_wsgi_main-directories]/File[/var/lib/puppet/concat/10-keystone_wsgi_main.conf/fragments/60_keystone_wsgi_main-directories]/ensure: defined content as '{md5}cc81234a3bbf77f857ed3f11bb369e8c'                                                                                                                                     
Notice: /Stage[main]/Keystone::Wsgi::Apache/Apache::Vhost[keystone_wsgi_main]/Concat[10-keystone_wsgi_main.conf]/Exec[concat_10-keystone_wsgi_main.conf]/returns: executed successfully                              
Notice: /Stage[main]/Keystone::Wsgi::Apache/Apache::Vhost[keystone_wsgi_main]/Concat[10-keystone_wsgi_main.conf]/Exec[concat_10-keystone_wsgi_main.conf]: Triggered 'refresh' from 10 events                         
Notice: /Stage[main]/Keystone::Wsgi::Apache/Apache::Vhost[keystone_wsgi_main]/Concat[10-keystone_wsgi_main.conf]/File[10-keystone_wsgi_main.conf]/ensure: defined content as '{md5}aefb3afeb6ad674912b7be6c1b4b22fc' 
Notice: /Stage[main]/Apache::Mod::Cgi/Apache::Mod[cgi]/File[cgi.load]/ensure: defined content as '{md5}ac20c5c5779b37ab06b480d6485a0881'                                                                             
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/00-proxy.conf]/ensure: removed                                                                                                                            
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/01-cgi.conf]/ensure: removed                                                                                                                              
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/00-mpm.conf]/ensure: removed                                                                                                                              
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/00-systemd.conf]/ensure: removed                                                                                                                          
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/10-wsgi.conf]/ensure: removed                                                                                                                             
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/00-base.conf]/ensure: removed                                                                                                                             
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/00-lua.conf]/ensure: removed                                                                                                                              
Notice: /Stage[main]/Apache/File[/etc/httpd/conf.modules.d/00-dav.conf]/ensure: removed                                                                                                                              
Notice: /Stage[main]/Nova::Deps/Anchor[nova::install::end]: Triggered 'refresh' from 3 events                                                                                                                        
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/project_name]/ensure: created                                                                                                                        
Notice: /Stage[main]/Nova::Compute/Nova_config[DEFAULT/force_raw_images]/ensure: created                                                                                                                             
Notice: /Stage[main]/Nova::Logging/Nova_config[DEFAULT/debug]/ensure: created                                                                                                                                        
Notice: /Stage[main]/Nova::Api/Nova_config[neutron/service_metadata_proxy]/ensure: created                                                                                                                           
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[DEFAULT/use_neutron]/ensure: created                                                                                                                         
Notice: /Stage[main]/Nova::Logging/Nova_config[DEFAULT/log_dir]/ensure: created                                                                                                                                      
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/extension_sync_interval]/ensure: created                                                                                                             
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/password]/ensure: created                                                                                                                            
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/scheduler_use_baremetal_filters]/ensure: created                                                                                                    
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/notify_api_faults]/ensure: created                                                                                                                                     
Notice: /Stage[main]/Nova::Db/Nova_config[api_database/connection]/ensure: created                                                                                                                                   
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/osapi_compute_listen]/ensure: created                                                                                                                             
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/scheduler_default_filters]/ensure: created                                                                                                          
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/rabbit_userid]/ensure: created                                                                                                                           
Notice: /Stage[main]/Nova::Conductor/Nova_config[conductor/use_local]/ensure: created                                                                                                                                
Notice: /Stage[main]/Nova::Api/Nova_config[osapi_v3/enabled]/ensure: created                                                                                                                                         
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/notification_driver]/ensure: created                                                                                                                                   
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/max_io_ops_per_host]/ensure: created                                                                                                                
Notice: /Stage[main]/Main/Nova_config[DEFAULT/linuxnet_interface_driver]/ensure: created                                                                                                                             
Notice: /Stage[main]/Nova::Api/Nova_config[keystone_authtoken/admin_password]/ensure: created                                                                                                                        
Notice: /Stage[main]/Nova::Compute/Nova_config[DEFAULT/compute_manager]/ensure: created                                                                                                                              
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/rabbit_password]/ensure: created                                                                                                                         
Notice: /Stage[main]/Nova::Api/Nova_config[keystone_authtoken/admin_user]/ensure: created                                                                                                                            
Notice: /Stage[main]/Main/Nova_config[DEFAULT/sync_power_state_interval]/ensure: created                                                                                                                             
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/state_path]/ensure: created                                                                                                                                            
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/report_interval]/ensure: created                                                                                                                                       
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/scheduler_weight_classes]/ensure: created                                                                                                           
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/rabbit_ha_queues]/ensure: created                                                                                                                        
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/auth_plugin]/ensure: created                                                                                                                         
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[DEFAULT/security_group_api]/ensure: created                                                                                                                  
Notice: /Stage[main]/Nova::Ironic::Common/Nova_config[ironic/admin_tenant_name]/ensure: created                                                                                                                      
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/enabled_apis]/ensure: created                                                                                                                                     
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/auth_url]/ensure: created                                                                                                                            
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/user_domain_name]/ensure: created                                                                                                                    
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/osapi_compute_listen_port]/ensure: created                                                                                                                        
Notice: /Stage[main]/Nova::Ironic::Common/Nova_config[ironic/admin_username]/ensure: created                                                                                                                         
Notice: /Stage[main]/Nova::Ironic::Common/Nova_config[ironic/api_endpoint]/ensure: created                                                                                                                           
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/rabbit_virtual_host]/ensure: created                                                                                                                     
Notice: /Stage[main]/Nova/Nova_config[glance/api_servers]/ensure: created                                                                                                                                            
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/image_service]/ensure: created                                                                                                                                         
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/volume_api_class]/ensure: created                                                                                                                                 
Notice: /Stage[main]/Main/Nova_config[DEFAULT/rpc_response_timeout]/ensure: created                                                                                                                                  
Notice: /Stage[main]/Nova::Compute/Nova_config[DEFAULT/heal_instance_info_cache_interval]/ensure: created                                                                                                            
Notice: /Stage[main]/Main/Nova_config[DEFAULT/my_ip]/ensure: created                                                                                                                                                 
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/notify_on_state_change]/ensure: created                                                                                                                                
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[DEFAULT/firewall_driver]/ensure: created                                                                                                                     
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/scheduler_host_subset_size]/ensure: created                                                                                                         
Notice: /Stage[main]/Nova/Nova_config[cinder/catalog_info]/ensure: created                                                                                                                                           
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/kombu_reconnect_delay]/ensure: created                                                                                                                   
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/ram_allocation_ratio]/ensure: created                                                                                                               
Notice: /Stage[main]/Nova::Db/Nova_config[database/connection]/ensure: created                                                                                                                                       
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/default_floating_pool]/ensure: created                                                                                                                            
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/timeout]/ensure: created                                                                                                                             
Notice: /Stage[main]/Nova::Compute/Nova_config[DEFAULT/reserved_host_memory_mb]/ensure: created                                                                                                                      
Notice: /Stage[main]/Nova::Scheduler/Nova_config[DEFAULT/scheduler_driver]/ensure: created                                                                                                                           
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/amqp_durable_queues]/ensure: created                                                                                                                     
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/disk_allocation_ratio]/ensure: created                                                                                                              
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/heartbeat_rate]/ensure: created                                                                                                                          
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/api_paste_config]/ensure: created                                                                                                                                 
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/heartbeat_timeout_threshold]/ensure: created                                                                                                             
Notice: /Stage[main]/Nova::Ironic::Common/Nova_config[ironic/admin_password]/ensure: created                                                                                                                         
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/project_domain_name]/ensure: created                                                                                                                 
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/scheduler_max_attempts]/ensure: created                                                                                                             
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/cpu_allocation_ratio]/ensure: created                                                                                                               
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[DEFAULT/vif_plugging_is_fatal]/ensure: created                                                                                                               
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/metadata_listen_port]/ensure: created                                                                                                                             
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/rabbit_use_ssl]/ensure: created                                                                                                                          
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/max_instances_per_host]/ensure: created                                                                                                             
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/scheduler_available_filters]/ensure: created                                                                                                        
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/url]/ensure: created                                                                                                                                 
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/service_down_time]/ensure: created                                                                                                                                     
Notice: /Stage[main]/Nova::Api/Nova_config[keystone_authtoken/admin_tenant_name]/ensure: created                                                                                                                     
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/ovs_bridge]/ensure: created                                                                                                                          
Notice: /Stage[main]/Nova::Scheduler::Filter/Nova_config[DEFAULT/scheduler_host_manager]/ensure: created                                                                                                             
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/notification_topics]/ensure: created                                                                                                                                   
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/use_forwarded_for]/ensure: created                                                                                                                                
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/osapi_volume_listen]/ensure: created                                                                                                                              
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/metadata_listen]/ensure: created                                                                                                                                  
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/auth_strategy]/ensure: created                                                                                                                                         
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/region_name]/ensure: created                                                                                                                         
Notice: /Stage[main]/Nova::Ironic::Common/Nova_config[ironic/admin_url]/ensure: created                                                                                                                              
Notice: /Stage[main]/Nova::Compute/Nova_config[vnc/enabled]/ensure: created                                                                                                                                          
Notice: /Stage[main]/Nova::Compute::Ironic/Nova_config[DEFAULT/compute_driver]/ensure: created                                                                                                                       
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/osapi_compute_workers]/ensure: created                                                                                                                            
Notice: /Stage[main]/Nova::Api/Nova_config[keystone_authtoken/identity_uri]/ensure: created                                                                                                                          
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/rootwrap_config]/ensure: created                                                                                                                                       
Notice: /Stage[main]/Nova/Nova_config[oslo_messaging_rabbit/rabbit_hosts]/ensure: created                                                                                                                            
Notice: /Stage[main]/Nova/Nova_config[DEFAULT/rpc_backend]/ensure: created                                                                                                                                           
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[DEFAULT/vif_plugging_timeout]/ensure: created                                                                                                                
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/metadata_workers]/ensure: created                                                                                                                                 
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[DEFAULT/dhcp_domain]/ensure: created                                                                                                                         
Notice: /Stage[main]/Nova::Compute/Nova_config[DEFAULT/force_config_drive]/ensure: created                                                                                                                           
Notice: /Stage[main]/Nova::Compute/Nova_config[DEFAULT/allow_resize_to_same_host]/ensure: created                                                                                                                    
Notice: /Stage[main]/Nova/Nova_config[oslo_concurrency/lock_path]/ensure: created                                                                                                                                    
Notice: /Stage[main]/Nova::Api/Nova_config[keystone_authtoken/auth_uri]/ensure: created                                                                                                                              
Notice: /Stage[main]/Nova::Network::Neutron/Nova_config[neutron/username]/ensure: created                                                                                                                            
Notice: /Stage[main]/Nova::Api/Nova_config[DEFAULT/fping_path]/ensure: created                                                                                                                                       
Notice: /Stage[main]/Nova::Deps/Anchor[nova::config::end]: Triggered 'refresh' from 97 events                                                                                                                        
Notice: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Mysql_database[nova]/ensure: created                                                                                                              
Notice: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Openstacklib::Db::Mysql::Host_access[nova_%]/Mysql_user[nova@%]/ensure: created                                                                   
Notice: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Openstacklib::Db::Mysql::Host_access[nova_%]/Mysql_grant[nova@%/nova.*]/ensure: created                                                           
Notice: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Mysql_database[nova_api]/ensure: created                                                                                                  
Notice: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Openstacklib::Db::Mysql::Host_access[nova_api_%]/Mysql_user[nova_api@%]/ensure: created                                                   
Notice: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Openstacklib::Db::Mysql::Host_access[nova_api_192.0.2.1]/Mysql_user[nova_api.2.1]/ensure: created                                   
Notice: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Openstacklib::Db::Mysql::Host_access[nova_api_192.0.2.1]/Mysql_grant[nova_api.2.1/nova_api.*]/ensure: created                       
Notice: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Openstacklib::Db::Mysql::Host_access[nova_192.0.2.1]/Mysql_user[nova.2.1]/ensure: created                                                   
Notice: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Openstacklib::Db::Mysql::Host_access[nova_192.0.2.1]/Mysql_grant[nova.2.1/nova.*]/ensure: created                                           
Notice: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Openstacklib::Db::Mysql::Host_access[nova_api_%]/Mysql_grant[nova_api@%/nova_api.*]/ensure: created                                       
Notice: /Stage[main]/Nova::Deps/Anchor[nova::db::end]: Triggered 'refresh' from 2 events                                                                                                                             
Notice: /Stage[main]/Nova::Deps/Anchor[nova::dbsync::begin]: Triggered 'refresh' from 1 events                                                                                                                       
Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]: Failed to call refresh: Command exceeded timeout                                                                                                              
Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]: Command exceeded timeout                                                                                                                                      
Wrapped exception:                                                                                                                                                                                                   
execution expired                                                                                                                                                                                                    
Error: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: Failed to call refresh: Command exceeded timeout                                                                                                      
Error: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: Command exceeded timeout                                                                                                                              
Wrapped exception:                                                                                                                                                                                                   
execution expired                                                                                                                                                                                                    
Notice: /Stage[main]/Nova::Deps/Anchor[nova::service::begin]: Triggered 'refresh' from 2 events                                                                                                                      
Notice: /Stage[main]/Nova::Cert/Nova::Generic_service[cert]/Service[nova-cert]/ensure: ensure changed 'stopped' to 'running'                                                                                         
Notice: /Stage[main]/Nova::Api/Nova::Generic_service[api]/Service[nova-api]/ensure: ensure changed 'stopped' to 'running'  



Expected result:
Successfully deployed undercloud.

Comment 1 Lon Hohberger 2016-11-22 14:41:09 UTC
This is likely a nova bug. Nova is the thing that changed, not instack-undercloud.

Comment 2 James Slagle 2016-11-22 18:07:45 UTC
can you attach the entire undercloud installation log from ~/.instack/install-undercloud.log?

Comment 3 Alexander Chuzhoy 2016-11-23 19:35:52 UTC
I don't reproduce this issue now.
Closing.
Thanks.