| Summary: | rhosp 8 beta 7 deployment failed | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | bigswitch <rhosp-bugs-internal> | ||||||
| Component: | rhosp-director | Assignee: | Angus Thomas <athomas> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Arik Chernetsky <achernet> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | urgent | ||||||||
| Version: | 8.0 (Liberty) | CC: | dbecker, mburns, morazi, rhel-osp-director-maint, rhosp-bugs-internal, sbaker, shardy, yeylon | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | 8.0 (Liberty) | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2016-03-10 12:18:28 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Attachments: |
|
||||||||
|
Description
bigswitch
2016-03-01 18:21:31 UTC
Created attachment 1132006 [details]
sosreport-compute
sosreport is attached Added journalctl output from controller:
[root@overcloud-controller-0 tmp]# journalctl -fn 100000 | grep -ni error
1513:Mar 01 12:51:39 localhost.localdomain cloud-init[877]: 2016-03-01 12:51:39,266 - util.py[WARNING]: Route info failed: Unexpected error while running command.
1519:Mar 01 12:51:39 localhost.localdomain cloud-init[877]: [CLOUDINIT] util.py[WARNING]: Route info failed: Unexpected error while running command.
1525:Mar 01 12:51:39 localhost.localdomain cloud-init[877]: [CLOUDINIT] util.py[DEBUG]: Route info failed: Unexpected error while running command.
1538: ProcessExecutionError: Unexpected error while running command.
2485:Mar 01 12:52:16 overcloud-controller-0.localdomain dracut[3561]: -rwxr-xr-x 1 root root 145 Jan 15 03:06 usr/bin/kdump-error-handler.sh
2731:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: -rw-r--r-- 1 root root 993 Jan 15 03:06 usr/lib/systemd/system/kdump-error-handler.service
2884:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: lrwxrwxrwx 1 root root 22 Mar 1 12:52 usr/lib64/libgpg-error.so -> libgpg-error.so.0.10.0
2885:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: lrwxrwxrwx 1 root root 22 Mar 1 12:52 usr/lib64/libgpg-error.so.0 -> libgpg-error.so.0.10.0
2886:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: -rwxr-xr-x 1 root root 19384 Mar 1 12:52 usr/lib64/libgpg-error.so.0.10.0
5079:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Resource start-up disabled since no STONITH resources have been defined
5080:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Either configure some or disable STONITH with the stonith-enabled option
5081:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: NOTE: Clusters with shared data need STONITH to ensure data integrity
5084:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: notice: Configuration ERRORs found during PE processing. Please run "crm_verify -L" to identify issues.
5089:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Resource start-up disabled since no STONITH resources have been defined
5090:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Either configure some or disable STONITH with the stonith-enabled option
5091:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: NOTE: Clusters with shared data need STONITH to ensure data integrity
5094:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: notice: Configuration ERRORs found during PE processing. Please run "crm_verify -L" to identify issues.
5108:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: e]/returns: executed successfully\u001b[0m\n\u001b[mNotice: Pacemaker has reported quorum achieved\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Notify[pacemaker settled]/message: defined 'message' as 'Pacemaker has reported quorum achieved'\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Stonith/Exec[Disable STONITH]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Tripleo::Loadbalancer/Haproxy::Listen[redis]/Concat::Fragment[haproxy-redis_listen_block]/File[/var/lib/puppet/concat/_etc_haproxy_haproxy.cfg/fragments/20-redis-00_haproxy-redis_listen_block]/ensure: defined content as '{md5}f11dc73be476a8ce3267d965fedc63f1'\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]: Triggered 'refresh' from 41 events\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/content: content changed '{md5}1f337186b0e1ba5ee82760cb437fb810' to '{md5}40706bafaaa45c5cdc110ab5b717f865'\u001b[0m\n\u001b[mNotice: /File[/etc/haproxy/haproxy.cfg]/seluser: seluser changed 'unconfined_u' to 'system_u'\u001b[0m\n\u001b[mNotice: Finished catalog run in 43.66 seconds\u001b[0m\n", "deploy_stderr": "\u001b[1;31mWarning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.\u001b[0m\n\u001b[1;31mWarning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user'
5109:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0}
5213:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
5214:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
5817:Mar 01 12:57:42 overcloud-controller-0.localdomain galera(galera)[3738]: ERROR: MySQL is not running
5861:Mar 01 12:57:48 overcloud-controller-0.localdomain galera(galera)[4905]: ERROR: MySQL is not running
6115:Mar 01 12:58:14 overcloud-controller-0.localdomain os-collect-config[7424]: int::Base[storage_mgmt_vip-then-haproxy]/Exec[Creating order constraint storage_mgmt_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Colocation[control_vip-with-haproxy]/Pcmk_constraint[colo-ip-192.168.0.12-haproxy-clone]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[public_vip-then-haproxy]/Exec[Creating order constraint public_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[redis_vip-then-haproxy]/Exec[Creating order constraint redis_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[storage_vip-then-haproxy]/Exec[Creating order constraint storage_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: Finished catalog run in 35.84 seconds\u001b[0m\n", "deploy_stderr": "\u001b[1;31mWarning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.\u001b[0m\n\u001b[1;31mWarning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0}
6219:Mar 01 12:58:14 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
6220:Mar 01 12:58:14 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
9454:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: ]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Db/Ceilometer_config[database/retry_interval]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Agent::Notification/Ceilometer_config[notification/ack_on_event_error]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/rpc_backend]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Alarm::Evaluator/Ceilometer_config[alarm/evaluation_service]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[oslo_messaging_rabbit/rabbit_hosts]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Agent::Central/Ceilometer_config[coordination/backend_url]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/http_timeout]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/notification_topics]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Db/Ceilometer_config[database/max_retries]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Agent::Auth/Ceilometer_config[service_credentials/os_auth_url]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Logging/Ceilometer_config[DEFAULT/use_syslog]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Collector/Ceilometer_config[collector/udp_address]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Expirer/Ceilometer_config[database/time_to_live]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Alarm::Evaluator/Ceilometer_config[alarm/record_history]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Logging/Ceilometer_config[DEFAULT/use_stderr]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Alarm::Evaluator/Ceilometer_config[alarm/evaluation_interval]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[oslo_messaging_rabbit/rabbit_ha_queues]/ensure: created\u001
9489:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: /swift/object-server/]/seluser: seluser changed 'unconfined_u' to 'system_u'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments.concat]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments.concat.out]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat::Fragment[swift_proxy]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/00_swift_proxy]/ensure: defined content as '{md5}d40310054460199f880f40207d9a92f4'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Catch_errors/Concat::Fragment[swift_catch_errors]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/24_swift_catch_errors]/ensure: defined content as '{md5}e07f0e5b125db7d6c8b4724c1648bcd5'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Healthcheck/Concat::Fragment[swift_healthcheck]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/25_swift_healthcheck]/ensure: defined content as '{md5}8c92056c41082619d179f88ea15c5fc6'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Cache/Concat::Fragment[swift_cache]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/23_swift_cache]/ensure: defined content as '{md5}20006a9d2864c3a4f5d6b1df19f09fce'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Tempurl/Concat::Fragment[swift-proxy-tempurl]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/29_swift-proxy-tempurl]/ensure: defined content as '{md5}2fe004eae9f03fc684f9ed90044bd9c5'\u001b[0m\n\u001b[mNotice:
9630:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: Notice: /Stage[main]/Ceilometer::Agent::Notification/Ceilometer_config[notification/ack_on_event_error]/ensure: created
10179:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: Notice: /Stage[main]/Swift::Proxy::Catch_errors/Concat::Fragment[swift_catch_errors]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/24_swift_catch_errors]/ensure: defined content as '{md5}e07f0e5b125db7d6c8b4724c1648bcd5'
11233:Mar 01 13:03:31 overcloud-controller-0.localdomain crmd[30047]: warning: Action 121 (openstack-heat-engine:0_start_0) on overcloud-controller-0 failed (target: 0 vs. rc: 7): Error
11235:Mar 01 13:03:31 overcloud-controller-0.localdomain crmd[30047]: warning: Action 121 (openstack-heat-engine:0_start_0) on overcloud-controller-0 failed (target: 0 vs. rc: 7): Error
11807:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 31 (neutron-server:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error
11809:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 31 (neutron-server:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error
11810:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 32 (httpd:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error
11811:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 32 (httpd:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error
12032:Mar 01 13:03:48 overcloud-controller-0.localdomain object-server[8046]: Object audit (ZBF) "forever" mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0.00, Rate: 0.00
12034:Mar 01 13:03:48 overcloud-controller-0.localdomain object-server[8047]: Object audit (ALL) "forever" mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0.00, Rate: 0.00
12160:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8134]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server".
12171:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8169]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server".
12175:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8168]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server".
12185:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8170]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server".
12189:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8171]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server".
13110:Mar 01 13:03:58 overcloud-controller-0.localdomain ceilometer-alarm-evaluator[9169]: ImportError: No module named redis
13111:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl [req-57b7f775-9ae2-42d9-acc8-480165ed44f7 - - - - -] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=type', 'list', 'Interface', 'int-br-ex'].
13112:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
13113:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
13114:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip()
13115:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
13116:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m)
13117:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError:
13118:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=type', 'list', 'Interface', 'int-br-ex']
13119:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 1
13120:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl
13121:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl
13191:Mar 01 13:03:59 overcloud-controller-0.localdomain ceilometer-polling[9170]: ImportError: No module named redis
13194:Mar 01 13:03:59 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:59.652 9018 ERROR neutron.agent.linux.ovsdb_monitor [req-57b7f775-9ae2-42d9-acc8-480165ed44f7 - - - - -] Interface monitor is not active
13209:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg
13210:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last):
13211:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config
13212:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param)
13213:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get
13214:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section)
13215:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'database'
13216:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils
13217:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg
13218:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last):
13219:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config
13220:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param)
13221:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get
13222:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section)
13223:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'glance_store'
13224:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils
13225:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg
13226:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last):
13227:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config
13228:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param)
13229:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get
13230:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section)
13231:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'image_format'
13232:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils
13233:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg
13234:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last):
13235:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config
13236:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param)
13237:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get
13238:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section)
13239:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'keystone_authtoken'
13240:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils
The failure was in the ComputeAllNodesValidationDeployment script on Compute node 0, so to diagnose the problem we'd need to see the output of that. You can get it from the Heat API with the command: heat deployment-show `heat resource-show 43cf7036-2b47-4680-b243-9ec7091976cf 0 | grep physical_resource_id | cut -d'|' -f3` hi
this is the output
[stack@c5220-01 sosreport2]$ heat deployment-show `heat resource-show 43cf7036-2b47-4680-b243-9ec7091976cf 0 | grep physical_resource_id | cut -d'|' -f3`
{
"status": "FAILED",
"server_id": "2367a924-bff0-4e8a-befc-1c3e01a85dd5",
"config_id": "49025b5d-036e-49b4-8516-02cd00cfae3b",
"output_values": {
"deploy_stdout": "Trying to ping 172.16.0.4 for local network 172.16.0.0/24...FAILURE\n",
"deploy_stderr": "172.16.0.4 is not pingable. Local Network: 172.16.0.0/24\n",
"deploy_status_code": 1
},
"creation_time": "2016-03-01T17:54:54",
"updated_time": "2016-03-01T17:55:35",
"input_values": {},
"action": "CREATE",
"status_reason": "deploy_status_code : Deployment exited with non-zero status code: 1",
"id": "016747e7-896b-448b-a718-352dcafbac48"
}
hi
this is the output
[stack@c5220-01 sosreport2]$ heat deployment-show `heat resource-show 43cf7036-2b47-4680-b243-9ec7091976cf 0 | grep physical_resource_id | cut -d'|' -f3`
{
"status": "FAILED",
"server_id": "2367a924-bff0-4e8a-befc-1c3e01a85dd5",
"config_id": "49025b5d-036e-49b4-8516-02cd00cfae3b",
"output_values": {
"deploy_stdout": "Trying to ping 172.16.0.4 for local network 172.16.0.0/24...FAILURE\n",
"deploy_stderr": "172.16.0.4 is not pingable. Local Network: 172.16.0.0/24\n",
"deploy_status_code": 1
},
"creation_time": "2016-03-01T17:54:54",
"updated_time": "2016-03-01T17:55:35",
"input_values": {},
"action": "CREATE",
"status_reason": "deploy_status_code : Deployment exited with non-zero status code: 1",
"id": "016747e7-896b-448b-a718-352dcafbac48"
}
Ok I miss adding tenantNeworkVlanID to controller.yaml file. Will add it and redeploy Found one problem. Looks like beta7 is not using the stable/liberty tripleo-heat-templates
For example the upstream stable/liberty has following code:
if 'cisco_n1kv' in hiera('neutron::plugins::ml2::mechanism_drivers') {
at https://github.com/openstack/tripleo-heat-templates/blob/stable/liberty/puppet/manifests/overcloud_compute.pp#L123
However, the actual beta7 code is like following:
if 'cisco_n1kv' in hiera('neutron_mechanism_drivers') {
(In reply to bigswitch from comment #9) > Found one problem. Looks like beta7 is not using the stable/liberty > tripleo-heat-templates > > For example the upstream stable/liberty has following code: > if 'cisco_n1kv' in hiera('neutron::plugins::ml2::mechanism_drivers') { > at > https://github.com/openstack/tripleo-heat-templates/blob/stable/liberty/ > puppet/manifests/overcloud_compute.pp#L123 > > However, the actual beta7 code is like following: > if 'cisco_n1kv' in hiera('neutron_mechanism_drivers') { This is both true and not true. The build included in beta 7 was openstack-tripleo-heat-templates-0.8.7-12.el7ost. Since the 0.8.7 release upstream, upstream has moved along. The change that you're seeing was done in this review: https://review.openstack.org/#/c/269898/ The plan is that beta 8 will be on the 0.8.9 release which just happened this week and will contain that patch. (In reply to bigswitch from comment #8) > Ok I miss adding tenantNeworkVlanID to controller.yaml file. Will add it and > redeploy Did this help and resolve your problem? Thanks a lot Mike. Please close this bug :) |