Bug 1495224 - [osp12]Overcloud HA services isn't operable after reboot of overcloud nodes
Summary: [osp12]Overcloud HA services isn't operable after reboot of overcloud nodes
Keywords:
Status: CLOSED DUPLICATE of bug 1490281
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-containers
Version: 12.0 (Pike)
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ga
: 12.0 (Pike)
Assignee: Assaf Muller
QA Contact: Omri Hochman
Andrew Burden
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-25 14:40 UTC by Artem Hrechanychenko
Modified: 2017-09-27 09:17 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-09-27 09:17:35 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Artem Hrechanychenko 2017-09-25 14:40:29 UTC
Description of problem:

(overcloud) [stack@undercloud-0 ~]$ source overcloudrc
(overcloud) [stack@undercloud-0 ~]$ nova list
No handlers could be found for logger "keystoneauth.identity.generic.base"
ERROR (ConnectFailure): Unable to establish connection to http://10.0.0.104:5000/v2.0/tokens: HTTPConnectionPool(host='10.0.0.104', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x301ab90>: Failed to establish a new connection: [Errno 113] No route to host',))



[heat-admin@overcloud-controller-0 ~]$ sudo docker  ps -a
CONTAINER ID        IMAGE                                                                             COMMAND                  CREATED             STATUS                          PORTS               NAMES
080848b55fd5        192.168.24.1:8787/rhosp12/openstack-ceilometer-central-docker:2017-09-22.5        "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        ceilometer_gnocchi_upgrade
8ab5d31dc9f1        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        nova_api_discover_hosts
b2b239eefa4b        192.168.24.1:8787/rhosp12/openstack-gnocchi-api-docker:2017-09-22.5               "kolla_start"            About an hour ago   Up 58 minutes                                       gnocchi_api
2b4b8083c8b9        192.168.24.1:8787/rhosp12/openstack-panko-api-docker:2017-09-22.5                 "kolla_start"            About an hour ago   Up 58 minutes                                       panko_api
f1ef9f63c065        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Up 58 minutes (unhealthy)                           nova_metadata
48dadc7c7716        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Up 58 minutes (unhealthy)                           nova_api
258d99a7fc5e        192.168.24.1:8787/rhosp12/openstack-glance-api-docker:2017-09-22.5                "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             glance_api
6dbb46bc2592        192.168.24.1:8787/rhosp12/openstack-swift-account-docker:2017-09-22.5             "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_account_server
d37cb914e99a        192.168.24.1:8787/rhosp12/openstack-aodh-listener-docker:2017-09-22.5             "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             aodh_listener
6bd9813d8903        192.168.24.1:8787/rhosp12/openstack-swift-container-docker:2017-09-22.5           "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_container_auditor
0aa069d63444        192.168.24.1:8787/rhosp12/openstack-heat-api-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             heat_api_cron
41dee94b92c0        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server-docker:2017-09-22.5        "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_object_expirer
ae475746bc65        192.168.24.1:8787/rhosp12/openstack-swift-object-docker:2017-09-22.5              "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_object_updater
466c6c3eaa15        192.168.24.1:8787/rhosp12/openstack-swift-container-docker:2017-09-22.5           "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_container_replicator
ae6031e3404a        192.168.24.1:8787/rhosp12/openstack-swift-account-docker:2017-09-22.5             "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_account_auditor
df4c4cf11326        192.168.24.1:8787/rhosp12/openstack-cron-docker:2017-09-22.5                      "kolla_start"            About an hour ago   Up 58 minutes                                       logrotate_crond
1ff310206f90        192.168.24.1:8787/rhosp12/openstack-heat-api-cfn-docker:2017-09-22.5              "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             heat_api_cfn
db4c6d7a69fa        192.168.24.1:8787/rhosp12/openstack-nova-scheduler-docker:2017-09-22.5            "kolla_start"            About an hour ago   Up About a minute (unhealthy)                       nova_scheduler
d4c4888bb2ef        192.168.24.1:8787/rhosp12/openstack-gnocchi-metricd-docker:2017-09-22.5           "kolla_start"            About an hour ago   Up 58 minutes                                       gnocchi_metricd
c10285b34c4e        192.168.24.1:8787/rhosp12/openstack-swift-object-docker:2017-09-22.5              "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_object_replicator
bb8d35b69ae3        192.168.24.1:8787/rhosp12/openstack-swift-container-docker:2017-09-22.5           "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_container_server
43456bb93b40        192.168.24.1:8787/rhosp12/openstack-heat-engine-docker:2017-09-22.5               "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             heat_engine
4090c76370ca        192.168.24.1:8787/rhosp12/openstack-aodh-api-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Up 58 minutes                                       aodh_api
57554d295f55        192.168.24.1:8787/rhosp12/openstack-swift-object-docker:2017-09-22.5              "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_rsync
b22f73b718fc        192.168.24.1:8787/rhosp12/openstack-nova-novncproxy-docker:2017-09-22.5           "kolla_start"            About an hour ago   Up 58 minutes (unhealthy)                           nova_vnc_proxy
3e22c56eadfc        192.168.24.1:8787/rhosp12/openstack-gnocchi-api-docker:2017-09-22.5               "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        gnocchi_db_sync
59efb8fa78b3        192.168.24.1:8787/rhosp12/openstack-ceilometer-notification-docker:2017-09-22.5   "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             ceilometer_agent_notification
68c3c0aa57e2        192.168.24.1:8787/rhosp12/openstack-swift-account-docker:2017-09-22.5             "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_account_reaper
5490dcfabe50        192.168.24.1:8787/rhosp12/openstack-nova-consoleauth-docker:2017-09-22.5          "kolla_start"            About an hour ago   Up 58 minutes (unhealthy)                           nova_consoleauth
4b3e18eb0688        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             nova_api_cron
710277f0394f        192.168.24.1:8787/rhosp12/openstack-gnocchi-statsd-docker:2017-09-22.5            "kolla_start"            About an hour ago   Restarting (1) 2 minutes ago                        gnocchi_statsd
1d3017200b3f        192.168.24.1:8787/rhosp12/openstack-aodh-notifier-docker:2017-09-22.5             "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             aodh_notifier
2edc8209261c        192.168.24.1:8787/rhosp12/openstack-ceilometer-central-docker:2017-09-22.5        "kolla_start"            About an hour ago   Up 58 minutes (unhealthy)                           ceilometer_agent_central
f17735b9d634        192.168.24.1:8787/rhosp12/openstack-swift-account-docker:2017-09-22.5             "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_account_replicator
a8a694023d19        192.168.24.1:8787/rhosp12/openstack-swift-object-docker:2017-09-22.5              "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_object_auditor
0d5d4b9be9d1        192.168.24.1:8787/rhosp12/openstack-heat-api-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             heat_api
3ae69141d9c6        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server-docker:2017-09-22.5        "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_proxy
ce39853ce2fe        192.168.24.1:8787/rhosp12/openstack-swift-object-docker:2017-09-22.5              "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_object_server
73c24388bada        192.168.24.1:8787/rhosp12/openstack-nova-conductor-docker:2017-09-22.5            "kolla_start"            About an hour ago   Up 58 minutes (unhealthy)                           nova_conductor
85ebb454ef3d        192.168.24.1:8787/rhosp12/openstack-swift-container-docker:2017-09-22.5           "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             swift_container_updater
13cb1c1f97be        192.168.24.1:8787/rhosp12/openstack-aodh-evaluator-docker:2017-09-22.5            "kolla_start"            About an hour ago   Up 58 minutes (unhealthy)                           aodh_evaluator
5877f77ac544        192.168.24.1:8787/rhosp12/openstack-keystone-docker:2017-09-22.5                  "/bin/bash -c '/usr/l"   About an hour ago   Up 58 minutes (healthy)                             keystone_cron
3f49bd88c7d7        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        nova_db_sync
9a79a704b44c        192.168.24.1:8787/rhosp12/openstack-keystone-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Up 58 minutes (healthy)                             keystone
8c2e4e51f999        192.168.24.1:8787/rhosp12/openstack-iscsid-docker:2017-09-22.5                    "kolla_start"            About an hour ago   Up 58 minutes                                       iscsid
fcc739f95282        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        nova_api_create_default_cell
e4f90ad976d7        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        nova_api_map_cell0
e11059bf4cd9        192.168.24.1:8787/rhosp12/openstack-nova-placement-api-docker:2017-09-22.5        "kolla_start"            About an hour ago   Up 58 minutes                                       nova_placement
dce7fca7edd9        192.168.24.1:8787/rhosp12/openstack-swift-account-docker:2017-09-22.5             "chown -R swift: /srv"   About an hour ago   Exited (0) About an hour ago                        swift_setup_srv
f6108d2adccf        192.168.24.1:8787/rhosp12/openstack-horizon-docker:2017-09-22.5                   "kolla_start"            About an hour ago   Up 58 minutes                                       horizon
6404b747f7a4        192.168.24.1:8787/rhosp12/openstack-ceilometer-notification-docker:2017-09-22.5   "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        ceilometer_init_log
82d6cfd5eac6        192.168.24.1:8787/rhosp12/openstack-keystone-docker:2017-09-22.5                  "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        keystone_db_sync
43206056fbdb        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        nova_api_db_sync
9e1f3650a6dd        192.168.24.1:8787/rhosp12/openstack-panko-api-docker:2017-09-22.5                 "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        panko_db_sync
343fb64ca4b5        192.168.24.1:8787/rhosp12/openstack-swift-proxy-server-docker:2017-09-22.5        "/bin/bash -c 'cp -v "   About an hour ago   Exited (0) About an hour ago                        swift_copy_rings
10642b1cf056        192.168.24.1:8787/rhosp12/openstack-glance-api-docker:2017-09-22.5                "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        glance_api_db_sync
d0fb252faaa3        192.168.24.1:8787/rhosp12/openstack-heat-engine-docker:2017-09-22.5               "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        heat_engine_db_sync
1b362c5756a9        192.168.24.1:8787/rhosp12/openstack-aodh-api-docker:2017-09-22.5                  "/usr/bin/bootstrap_h"   About an hour ago   Exited (0) About an hour ago                        aodh_db_sync
de9cdce6d1ad        192.168.24.1:8787/rhosp12/openstack-haproxy-docker:pcmklatest                     "/bin/bash /usr/local"   About an hour ago   Exited (1) 58 minutes ago                           haproxy-bundle-docker-0
0717d8505b81        192.168.24.1:8787/rhosp12/openstack-redis-docker:pcmklatest                       "/bin/bash /usr/local"   About an hour ago   Exited (1) 58 minutes ago                           redis-bundle-docker-0
5615629b5140        192.168.24.1:8787/rhosp12/openstack-haproxy-docker:2017-09-22.5                   "/bin/bash -c 'cp -a "   About an hour ago   Exited (0) About an hour ago                        haproxy_init_bundle
1f42b88dbf80        192.168.24.1:8787/rhosp12/openstack-redis-docker:2017-09-22.5                     "/bin/bash -c 'cp -a "   About an hour ago   Exited (0) About an hour ago                        redis_init_bundle
e34ee525991a        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:pcmklatest                     "/bin/bash /usr/local"   About an hour ago   Exited (1) 58 minutes ago                           galera-bundle-docker-0
2d93bc713f10        192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:pcmklatest                    "/bin/bash /usr/local"   About an hour ago   Exited (1) 58 minutes ago                           rabbitmq-bundle-docker-0
072ec43b0626        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:2017-09-22.5                   "/bin/bash -c 'cp -a "   About an hour ago   Exited (0) About an hour ago                        mysql_init_bundle
a22652aad369        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:2017-09-22.5                   "kolla_start"            About an hour ago   Up 58 minutes                                       clustercheck
df5cc7b4d331        192.168.24.1:8787/rhosp12/openstack-aodh-api-docker:2017-09-22.5                  "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        aodh_init_log
3c2630455871        192.168.24.1:8787/rhosp12/openstack-panko-api-docker:2017-09-22.5                 "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        panko_init_log
bb6cb6d106bc        192.168.24.1:8787/rhosp12/openstack-horizon-docker:2017-09-22.5                   "/bin/bash -c 'touch "   About an hour ago   Exited (0) About an hour ago                        horizon_fix_perms
885596feff02        192.168.24.1:8787/rhosp12/openstack-keystone-docker:2017-09-22.5                  "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        keystone_init_log
4f76c8971d11        192.168.24.1:8787/rhosp12/openstack-glance-api-docker:2017-09-22.5                "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        glance_init_logs
f28820211151        192.168.24.1:8787/rhosp12/openstack-heat-engine-docker:2017-09-22.5               "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        heat_init_log
626c3d370af2        192.168.24.1:8787/rhosp12/openstack-nova-api-docker:2017-09-22.5                  "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        nova_init_logs
51b6793f32ca        192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:2017-09-22.5                  "/bin/bash -c 'cp -a "   About an hour ago   Exited (0) About an hour ago                        rabbitmq_init_bundle
bb25c9fc1ee5        192.168.24.1:8787/rhosp12/openstack-gnocchi-api-docker:2017-09-22.5               "/bin/bash -c 'chown "   About an hour ago   Exited (0) About an hour ago                        gnocchi_init_log
dc4a51d6dd7c        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:2017-09-22.5                   "/bin/bash -c '/usr/b"   About an hour ago   Exited (0) About an hour ago                        mysql_image_tag
9c621a774a0d        192.168.24.1:8787/rhosp12/openstack-memcached-docker:2017-09-22.5                 "/bin/bash -c 'source"   About an hour ago   Up 58 minutes                                       memcached
bb1f159c61e3        192.168.24.1:8787/rhosp12/openstack-haproxy-docker:2017-09-22.5                   "/bin/bash -c '/usr/b"   About an hour ago   Exited (0) About an hour ago                        haproxy_image_tag
bce6fa3ea4a4        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:2017-09-22.5                   "bash -ecx 'if [ -e /"   About an hour ago   Exited (0) About an hour ago                        mysql_bootstrap
7a8ebfe8d77b        192.168.24.1:8787/rhosp12/openstack-redis-docker:2017-09-22.5                     "/bin/bash -c '/usr/b"   About an hour ago   Exited (0) About an hour ago                        redis_image_tag
6951da07adf3        192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:2017-09-22.5                  "/bin/bash -c '/usr/b"   About an hour ago   Exited (0) About an hour ago                        rabbitmq_image_tag
ccd7dec8616f        192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:2017-09-22.5                  "kolla_start"            About an hour ago   Exited (0) About an hour ago                        rabbitmq_bootstrap
53d833309232        192.168.24.1:8787/rhosp12/openstack-memcached-docker:2017-09-22.5                 "/bin/bash -c 'source"   About an hour ago   Exited (0) About an hour ago                        memcached_init_logs
458111503cb6        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:2017-09-22.5                   "chown -R mysql: /var"   About an hour ago   Exited (0) About an hour ago                        mysql_data_ownership




[heat-admin@overcloud-controller-0 ~]$ sudo docker ps -a |grep "Exited (1)"
de9cdce6d1ad        192.168.24.1:8787/rhosp12/openstack-haproxy-docker:pcmklatest                     "/bin/bash /usr/local"   About an hour ago   Exited (1) 53 minutes ago                              haproxy-bundle-docker-0
0717d8505b81        192.168.24.1:8787/rhosp12/openstack-redis-docker:pcmklatest                       "/bin/bash /usr/local"   About an hour ago   Exited (1) 53 minutes ago                              redis-bundle-docker-0
e34ee525991a        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:pcmklatest                     "/bin/bash /usr/local"   About an hour ago   Exited (1) 53 minutes ago                              

2d93bc713f10        192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:pcmklatest                    "/bin/bash /usr/local"   About an hour ago   Exited (1) 53 minutes ago                              rabbitmq-bundle-docker-0


710277f0394f        192.168.24.1:8787/rhosp12/openstack-gnocchi-statsd-docker:2017-09-22.5            "kolla_start"            About an hour ago   Restarting (1) 24 minutes ago                          gnocchi_statsd

[heat-admin@overcloud-controller-1 ~]$ sudo docker ps -a |grep "(1)"
a0d589092751        192.168.24.1:8787/rhosp12/openstack-gnocchi-statsd-docker:2017-09-22.5            "kolla_start"            About an hour ago   Restarting (1) 3 minutes ago                               gnocchi_statsd
baba1e156202        192.168.24.1:8787/rhosp12/openstack-redis-docker:pcmklatest                       "/bin/bash /usr/local"   About an hour ago   Exited (1) 59 minutes ago                                  redis-bundle-docker-1
3e6224af6d46        192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:pcmklatest                    "/bin/bash /usr/local"   About an hour ago   Exited (1) 59 minutes ago                                  rabbitmq-bundle-docker-1

[heat-admin@overcloud-controller-2 ~]$ sudo docker ps -a |grep "(1)"
3e0fc8a87084        192.168.24.1:8787/rhosp12/openstack-gnocchi-statsd-docker:2017-09-22.5            "kolla_start"            About an hour ago   Restarting (1) 4 minutes ago                        gnocchi_statsd
ed2ed1320627        192.168.24.1:8787/rhosp12/openstack-haproxy-docker:pcmklatest                     "/bin/bash /usr/local"   About an hour ago   Exited (1) About an hour ago                        haproxy-bundle-docker-2
59abedb60dee        192.168.24.1:8787/rhosp12/openstack-redis-docker:pcmklatest                       "/bin/bash /usr/local"   About an hour ago   Exited (1) About an hour ago                        redis-bundle-docker-2
4668dfc0a4d7        192.168.24.1:8787/rhosp12/openstack-mariadb-docker:pcmklatest                     "/bin/bash /usr/local"   About an hour ago   Exited (1) About an hour ago                        galera-bundle-docker-2
6c8031d4c026        192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:pcmklatest                    "/bin/bash /usr/local"   About an hour ago   Exited (1) About an hour ago                        rabbitmq-bundle-docker-2

[heat-admin@overcloud-controller-0 ~]$ sudo pcs status
Cluster name: tripleo_cluster
Stack: corosync
Current DC: overcloud-controller-0 (version 1.1.16-12.el7_4.2-94ff4df) - partition WITHOUT quorum
Last updated: Mon Sep 25 14:39:26 2017
Last change: Mon Sep 25 13:24:51 2017 by root via cibadmin on overcloud-controller-0

12 nodes configured
37 resources configured

Online: [ overcloud-controller-0 ]
OFFLINE: [ overcloud-controller-1 overcloud-controller-2 ]

Full list of resources:

 Docker container set: rabbitmq-bundle [192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:pcmklatest]
   rabbitmq-bundle-0	(ocf::heartbeat:rabbitmq-cluster):	Stopped
   rabbitmq-bundle-1	(ocf::heartbeat:rabbitmq-cluster):	Stopped
   rabbitmq-bundle-2	(ocf::heartbeat:rabbitmq-cluster):	Stopped
 Docker container set: galera-bundle [192.168.24.1:8787/rhosp12/openstack-mariadb-docker:pcmklatest]
   galera-bundle-0	(ocf::heartbeat:galera):	Stopped
   galera-bundle-1	(ocf::heartbeat:galera):	Stopped
   galera-bundle-2	(ocf::heartbeat:galera):	Stopped
 Docker container set: redis-bundle [192.168.24.1:8787/rhosp12/openstack-redis-docker:pcmklatest]
   redis-bundle-0	(ocf::heartbeat:redis):	Stopped
   redis-bundle-1	(ocf::heartbeat:redis):	Stopped
   redis-bundle-2	(ocf::heartbeat:redis):	Stopped
 ip-192.168.24.12	(ocf::heartbeat:IPaddr2):	Stopped
 ip-10.0.0.104	(ocf::heartbeat:IPaddr2):	Stopped
 ip-172.17.1.13	(ocf::heartbeat:IPaddr2):	Stopped
 ip-172.17.1.18	(ocf::heartbeat:IPaddr2):	Stopped
 ip-172.17.3.12	(ocf::heartbeat:IPaddr2):	Stopped
 ip-172.17.4.10	(ocf::heartbeat:IPaddr2):	Stopped
 Docker container set: haproxy-bundle [192.168.24.1:8787/rhosp12/openstack-haproxy-docker:pcmklatest]
   haproxy-bundle-docker-0	(ocf::heartbeat:docker):	Stopped
   haproxy-bundle-docker-1	(ocf::heartbeat:docker):	Stopped
   haproxy-bundle-docker-2	(ocf::heartbeat:docker):	Stopped
 openstack-cinder-volume	(systemd:openstack-cinder-volume):	Stopped

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled


Logs:
gnocchi_statsd
http://pastebin.test.redhat.com/518984

haproxy-bundle-docker-0
http://pastebin.test.redhat.com/518985

galera:
http://pastebin.test.redhat.com/518988

rabbitmq
http://pastebin.test.redhat.com/518990

redis-logs:http://pastebin.test.redhat.com/518995

Version-Release number of selected component (if applicable):
OSP12
docker images - 2017-09-22.5

How reproducible:
openstack-swift-object-2.15.2-0.20170913222602.34601cc.el7ost.noarch
openstack-neutron-ml2-11.0.1-0.20170919192755.82db373.el7ost.noarch
openstack-heat-api-cfn-9.0.1-0.20170913170444.0c64134.el7ost.noarch
openstack-tripleo-image-elements-7.0.0-0.20170914203419.526772d.el7ost.noarch
openstack-zaqar-5.0.1-0.20170905222047.el7ost.noarch
openstack-nova-common-16.0.1-0.20170919190458.9ad426a.el7ost.noarch
openstack-neutron-11.0.1-0.20170919192755.82db373.el7ost.noarch
openstack-ironic-conductor-9.1.1-0.20170908114346.feb64c2.el7ost.noarch
openstack-tempest-17.0.0-0.20170901201711.ad75393.el7ost.noarch
openstack-tripleo-heat-templates-7.0.1-0.20170919183703.el7ost.noarch
openstack-mistral-executor-5.1.1-0.20170913191927.9e3b36b.el7ost.noarch
python-openstackclient-lang-3.12.0-0.20170821150739.f67ebce.el7ost.noarch
openstack-swift-account-2.15.2-0.20170913222602.34601cc.el7ost.noarch
openstack-neutron-common-11.0.1-0.20170919192755.82db373.el7ost.noarch
openstack-ironic-inspector-6.0.1-0.20170824132804.0e72dcb.el7ost.noarch
openstack-puppet-modules-11.0.0-0.20170828113154.el7ost.noarch
openstack-tripleo-common-7.6.1-0.20170913110336.el7ost.noarch
openstack-mistral-common-5.1.1-0.20170913191927.9e3b36b.el7ost.noarch
openstack-nova-api-16.0.1-0.20170919190458.9ad426a.el7ost.noarch
openstack-nova-conductor-16.0.1-0.20170919190458.9ad426a.el7ost.noarch
openstack-glance-15.0.0-0.20170830130905.9820166.el7ost.noarch
openstack-keystone-12.0.1-0.20170907172639.6a67918.el7ost.noarch
openstack-nova-compute-16.0.1-0.20170919190458.9ad426a.el7ost.noarch
openstack-heat-api-9.0.1-0.20170913170444.0c64134.el7ost.noarch
openstack-ironic-common-9.1.1-0.20170908114346.feb64c2.el7ost.noarch
openstack-mistral-api-5.1.1-0.20170913191927.9e3b36b.el7ost.noarch
openstack-nova-scheduler-16.0.1-0.20170919190458.9ad426a.el7ost.noarch
openstack-swift-proxy-2.15.2-0.20170913222602.34601cc.el7ost.noarch
openstack-ironic-api-9.1.1-0.20170908114346.feb64c2.el7ost.noarch
puppet-openstack_extras-11.3.1-0.20170906070209.b99c3a4.el7ost.noarch
openstack-mistral-engine-5.1.1-0.20170913191927.9e3b36b.el7ost.noarch
openstack-tripleo-common-containers-7.6.1-0.20170913110336.el7ost.noarch
openstack-swift-container-2.15.2-0.20170913222602.34601cc.el7ost.noarch
openstack-neutron-openvswitch-11.0.1-0.20170919192755.82db373.el7ost.noarch
openstack-heat-engine-9.0.1-0.20170913170444.0c64134.el7ost.noarch
openstack-tripleo-validations-7.4.0-0.20170918153806.efe8a72.el7ost.noarch
openstack-selinux-0.8.10-0.20170914195211.e16a8f8.2.el7ost.noarch
openstack-nova-placement-api-16.0.1-0.20170919190458.9ad426a.el7ost.noarch
openstack-tripleo-puppet-elements-7.0.0-0.20170914203705.2094778.el7ost.noarch
python-openstacksdk-0.9.17-0.20170821143340.7946243.el7ost.noarch
python-openstackclient-3.12.0-0.20170821150739.f67ebce.el7ost.noarch
openstack-heat-common-9.0.1-0.20170913170444.0c64134.el7ost.noarch
puppet-openstacklib-11.3.1-0.20170825142820.18ee919.el7ost.noarch
openstack-tripleo-ui-7.4.1-0.20170915125211.1031ea6.el7ost.noarch


Steps to Reproduce:
1.Deploy osp12
2.restart oc nodes
3.wait 5-10 minutes


Actual results:
overcloud ha services is down

Expected results:
overcloud in operable state

Additional info:

Comment 3 Michele Baldessari 2017-09-25 14:58:47 UTC
So the HA containers are down because the network is busted. You can tell that from the following initial lines in 'pcs status':
Online: [ overcloud-controller-0 ]
OFFLINE: [ overcloud-controller-1 overcloud-controller-2 ]

So node 0 cannot talk to node 1 and 2.

[root@overcloud-controller-0 ~]# ping overcloud-controller-1
PING overcloud-controller-1.localdomain (172.17.1.25) 56(84) bytes of data.
^C
--- overcloud-controller-1.localdomain ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1000ms

[root@overcloud-controller-0 ~]# ping overcloud-controller-2
PING overcloud-controller-2.localdomain (172.17.1.20) 56(84) bytes of data.
^C
--- overcloud-controller-2.localdomain ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms



Initially I thought this was a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1473763

But I see that you have:
[root@overcloud-controller-0 ~]# rpm -q os-net-config
os-net-config-7.3.0-0.20170910153345.77fe592.el7ost.noarch

And the above BZ (1473763) has fixed-in: os-net-config-7.2.1-0.20170825174722.77fe592.el7ost

And the patch from the BZ is in the package (I just checked). So not sure about this one. Maybe it is related to the '[rhos-dev] Overcloud Nodes Network Layout' thread on rhos-dev ?

Comment 4 Brian Haley 2017-09-25 22:02:40 UTC
I don't exactly know where to start looking on the test setup, just some things I noticed:

1. overcloud-controller-0 had a full set of iptables rules, overcloud-controller-1 has none

2. overcloud-controller-1 is flooding the vlan20 network with ARP frames, don't know why that is yet

I'll keep looking tomorrow, but if I can get more information on how things are configured here it might help.

Comment 5 Michele Baldessari 2017-09-26 09:55:30 UTC
Artem, this is likely a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1490281

Comment 6 Brian Haley 2017-09-26 13:27:07 UTC
The configuration does look just like the other bug looking at ovs-vsctl output, as well as the ARP storm on vlan20.

Comment 7 Jakub Libosvar 2017-09-27 09:17:35 UTC
Closing as duplicate as the reproducer steps and symptoms are the same.

*** This bug has been marked as a duplicate of bug 1490281 ***


Note You need to log in before you can comment on or make changes to this bug.