2269699 – failed to create instance with a time out error "oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID "

Bug 2269699 - failed to create instance with a time out error "oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID "

Summary: failed to create instance with a time out error "oslo_messaging.exceptions.Me...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	python-oslo-messaging
Sub Component:
Version:	17.1 (Wallaby)
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	high
Target Milestone:	---
Target Release:	---
Assignee:	dabarzil
QA Contact:	dabarzil
Docs Contact:
URL:
Whiteboard:
Depends On:	2253107
Blocks:
TreeView+	depends on / blocked

Reported:	2024-03-15 12:20 UTC by mbollo
Modified:	2024-11-21 09:39 UTC (History)
CC List:	18 users (show)
Fixed In Version:	python-oslo-messaging-12.7.3-17.1.20231219130800.5d6fd1a.el9ost
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	2253107
Environment:
Last Closed:	2024-11-21 09:39:50 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Issue Tracker	OSP-31624	0	None	None	None	2024-03-15 12:23:14 UTC
Red Hat Product Errata	RHBA-2024:9974	0	None	None	None	2024-11-21 09:39:52 UTC

Comment 6 dabarzil 2024-09-09 14:50:36 UTC

verified:
[tripleo-admin@controller-1 ~]$ sudo reboot

Node List:
  * Online: [ controller-0 controller-2 ]
  * OFFLINE: [ controller-1 ]
  * RemoteOnline: [ compute-0 compute-1 ]
  * GuestOnline: [ galera-bundle-0 galera-bundle-2 rabbitmq-bundle-0 rabbitmq-bundle-2 ]

Full List of Resources:
  * compute-0	(ocf:pacemaker:remote):	 Started controller-0
  * compute-1	(ocf:pacemaker:remote):	 Started controller-2
  * ip-192.168.24.44	(ocf:heartbeat:IPaddr2):	 Started controller-2
  * ip-10.0.0.107	(ocf:heartbeat:IPaddr2):	 Started controller-0
  * stonith-fence_ipmilan-5254007d4274	(stonith:fence_ipmilan):	 Started controller-2
  * stonith-fence_ipmilan-5254003d3886	(stonith:fence_ipmilan):	 Started controller-0
  * ip-172.17.1.81	(ocf:heartbeat:IPaddr2):	 Started controller-0
  * ip-172.17.3.10	(ocf:heartbeat:IPaddr2):	 Started controller-2
  * ip-172.17.4.97	(ocf:heartbeat:IPaddr2):	 Started controller-2
  * Container bundle set: haproxy-bundle [cluster.common.tag/haproxy:pcmklatest]:
    * haproxy-bundle-podman-0	(ocf:heartbeat:podman):	 Started controller-0
    * haproxy-bundle-podman-1	(ocf:heartbeat:podman):	 Stopped
    * haproxy-bundle-podman-2	(ocf:heartbeat:podman):	 Started controller-2
  * Container bundle set: galera-bundle [cluster.common.tag/mariadb:pcmklatest]:
    * galera-bundle-0	(ocf:heartbeat:galera):	 Promoted controller-0
    * galera-bundle-1	(ocf:heartbeat:galera):	 Stopped
    * galera-bundle-2	(ocf:heartbeat:galera):	 Promoted controller-2
  * Container bundle set: rabbitmq-bundle [cluster.common.tag/rabbitmq:pcmklatest]:
    * rabbitmq-bundle-0	(ocf:heartbeat:rabbitmq-cluster):	 Started controller-0
    * rabbitmq-bundle-1	(ocf:heartbeat:rabbitmq-cluster):	 Stopped
    * rabbitmq-bundle-2	(ocf:heartbeat:rabbitmq-cluster):	 Started controller-2
  * stonith-fence_ipmilan-52540024e4c4	(stonith:fence_ipmilan):	 Started controller-2
  * stonith-fence_compute-fence-nova	(stonith:fence_compute):	 Started controller-0
  * Clone Set: compute-unfence-trigger-clone [compute-unfence-trigger]:
    * Started: [ compute-0 compute-1 ]
    * Stopped: [ controller-0 controller-1 controller-2 ]
  * nova-evacuate	(ocf:openstack:NovaEvacuate):	 Started controller-2
  * stonith-fence_ipmilan-52540064e97d	(stonith:fence_ipmilan):	 Started controller-0
  * stonith-fence_ipmilan-5254009c1695	(stonith:fence_ipmilan):	 Started controller-0
  * Container bundle: openstack-cinder-volume [cluster.common.tag/cinder-volume:pcmklatest]:
    * openstack-cinder-volume-podman-0	(ocf:heartbeat:podman):	 Started controller-0

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled


and after reboot:

Node List:
  * Online: [ controller-0 controller-1 controller-2 ]
  * RemoteOnline: [ compute-0 compute-1 ]
  * GuestOnline: [ galera-bundle-0 galera-bundle-1 galera-bundle-2 rabbitmq-bundle-0 rabbitmq-bundle-1 rabbitmq-bundle-2 ]

Full List of Resources:
  * compute-0	(ocf:pacemaker:remote):	 Started controller-0
  * compute-1	(ocf:pacemaker:remote):	 Started controller-2
  * ip-192.168.24.44	(ocf:heartbeat:IPaddr2):	 Started controller-2
  * ip-10.0.0.107	(ocf:heartbeat:IPaddr2):	 Started controller-0
  * stonith-fence_ipmilan-5254007d4274	(stonith:fence_ipmilan):	 Started controller-2
  * stonith-fence_ipmilan-5254003d3886	(stonith:fence_ipmilan):	 Started controller-0
  * ip-172.17.1.81	(ocf:heartbeat:IPaddr2):	 Started controller-0
  * ip-172.17.3.10	(ocf:heartbeat:IPaddr2):	 Started controller-2
  * ip-172.17.4.97	(ocf:heartbeat:IPaddr2):	 Started controller-2
  * Container bundle set: haproxy-bundle [cluster.common.tag/haproxy:pcmklatest]:
    * haproxy-bundle-podman-0	(ocf:heartbeat:podman):	 Started controller-0
    * haproxy-bundle-podman-1	(ocf:heartbeat:podman):	 Started controller-1
    * haproxy-bundle-podman-2	(ocf:heartbeat:podman):	 Started controller-2
  * Container bundle set: galera-bundle [cluster.common.tag/mariadb:pcmklatest]:
    * galera-bundle-0	(ocf:heartbeat:galera):	 Promoted controller-0
    * galera-bundle-1	(ocf:heartbeat:galera):	 Promoted controller-1
    * galera-bundle-2	(ocf:heartbeat:galera):	 Promoted controller-2
  * Container bundle set: rabbitmq-bundle [cluster.common.tag/rabbitmq:pcmklatest]:
    * rabbitmq-bundle-0	(ocf:heartbeat:rabbitmq-cluster):	 Started controller-0
    * rabbitmq-bundle-1	(ocf:heartbeat:rabbitmq-cluster):	 Started controller-1
    * rabbitmq-bundle-2	(ocf:heartbeat:rabbitmq-cluster):	 Started controller-2
  * stonith-fence_ipmilan-52540024e4c4	(stonith:fence_ipmilan):	 Started controller-2
  * stonith-fence_compute-fence-nova	(stonith:fence_compute):	 Started controller-0
  * Clone Set: compute-unfence-trigger-clone [compute-unfence-trigger]:
    * Started: [ compute-0 compute-1 ]
    * Stopped: [ controller-0 controller-1 controller-2 ]
  * nova-evacuate	(ocf:openstack:NovaEvacuate):	 Started controller-2
  * stonith-fence_ipmilan-52540064e97d	(stonith:fence_ipmilan):	 Started controller-0
  * stonith-fence_ipmilan-5254009c1695	(stonith:fence_ipmilan):	 Started controller-0
  * Container bundle: openstack-cinder-volume [cluster.common.tag/cinder-volume:pcmklatest]:
    * openstack-cinder-volume-podman-0	(ocf:heartbeat:podman):	 Started controller-0

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

[tripleo-admin@controller-1 ~]$ sudo grep MessagingTimeout  /var/log/containers/nova/nova-conductor.log
[tripleo-admin@controller-1 ~]$
[tripleo-admin@controller-1 ~]$ sudo grep -i -E "ERROR.*oslo_messaging"  /var/log/containers/nova/nova-conductor.log
[tripleo-admin@controller-1 ~]$

Comment 11 errata-xmlrpc 2024-11-21 09:39:50 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (RHOSP 17.1.4 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:9974

Note You need to log in before you can comment on or make changes to this bug.