Bug 1734062

Summary: bundle resources restarting when probing podman containers takes too much time [rhel-8.0.0.z]
Product: Red Hat Enterprise Linux 8 Reporter: Oneata Mircea Teodor <toneata>
Component: resource-agentsAssignee: Oyvind Albrigtsen <oalbrigt>
Status: CLOSED ERRATA QA Contact: pkomarov
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.0CC: agk, cluster-maint, fdinitto, mjuricek, oalbrigt, pkomarov
Target Milestone: rcKeywords: ZStream
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: resource-agents-4.1.1-17.el8_0.4 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-09-10 13:13:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Oneata Mircea Teodor 2019-07-29 14:38:24 UTC 
This bug has been copied from bug#1718219 and has been proposed to be backported to 8.0.0 z-stream.
Devel and QA ack is needed for full approval of the zstream clone
Comment 4 pkomarov 2019-08-01 06:10:43 UTC 
Verified , 

[root@controller-2 ~]# rpm -qa|grep resource-agents
[...]
resource-agents-4.1.1-17.el8_0.4.x86_64

#start i/o load : 

[root@controller-2 ~]# cat /proc/sys/vm/drop_caches
2

[root@controller-2 ~]# time dd if=/dev/zero of=bigtestfile bs=1M
^C17832+0 records in
17832+0 records out
18698207232 bytes (19 GB, 17 GiB) copied, 80.3198 s, 233 MB/s


real	1m21.061s
user	0m0.028s
sys	1m6.617s


#during the load period resources stayed in healthy state : 
Stack: corosync
Current DC: controller-2 (version 2.0.1-4.el8_0.3-0eb7991564) - partition with quorum
Last updated: Thu Aug  1 06:09:22 2019
Last change: Thu Aug  1 06:06:30 2019 by hacluster via crmd on controller-2

17 nodes configured
72 resources configured

Online: [ controller-0 controller-1 controller-2 ]
RemoteOnline: [ overcloud-novacomputeiha-0 overcloud-novacomputeiha-1 ]
GuestOnline: [ galera-bundle-0@controller-2 galera-bundle-1@controller-0 galera-bundle-2@controller-1 ovn-dbs-bundle-0@
controller-2 ovn-dbs-bundle-1@controller-0 ovn-dbs-bundle-2@controller-1 rabbitmq-bundle-0@controller-2 rabbitmq-bundle
-1@controller-0 rabbitmq-bundle-2@controller-1 redis-bundle-0@controller-2 redis-bundle-1@controller-0 redis-bundle-2@c
ontroller-1 ]

Active resources:

overcloud-novacomputeiha-0	(ocf::pacemaker:remote):        Started controller-0
overcloud-novacomputeiha-1	(ocf::pacemaker:remote):        Started controller-1
 podman container set: galera-bundle [192.168.24.1:8787/rhosp15/openstack-mariadb:pcmklatest]
   galera-bundle-0	(ocf::heartbeat:galera):        Master controller-2
   galera-bundle-1	(ocf::heartbeat:galera):        Master controller-0
   galera-bundle-2	(ocf::heartbeat:galera):        Master controller-1
 podman container set: rabbitmq-bundle [192.168.24.1:8787/rhosp15/openstack-rabbitmq:pcmklatest]
   rabbitmq-bundle-0    (ocf::heartbeat:rabbitmq-cluster):	Started controller-2
   rabbitmq-bundle-1    (ocf::heartbeat:rabbitmq-cluster):	Started controller-0
   rabbitmq-bundle-2    (ocf::heartbeat:rabbitmq-cluster):	Started controller-1
 podman container set: redis-bundle [192.168.24.1:8787/rhosp15/openstack-redis:pcmklatest]
   redis-bundle-0	(ocf::heartbeat:redis): Master controller-2
   redis-bundle-1	(ocf::heartbeat:redis): Slave controller-0
   redis-bundle-2	(ocf::heartbeat:redis): Slave controller-1
ip-192.168.24.9 (ocf::heartbeat:IPaddr2):	Started controller-2
ip-10.0.0.149   (ocf::heartbeat:IPaddr2):	Started controller-0
ip-172.17.1.144 (ocf::heartbeat:IPaddr2):	Started controller-1
ip-172.17.1.118 (ocf::heartbeat:IPaddr2):	Started controller-2
ip-172.17.3.124 (ocf::heartbeat:IPaddr2):	Started controller-0
ip-172.17.4.117 (ocf::heartbeat:IPaddr2):	Started controller-1
 podman container set: haproxy-bundle [192.168.24.1:8787/rhosp15/openstack-haproxy:pcmklatest]
   haproxy-bundle-podman-0	(ocf::heartbeat:podman):        Started controller-2
   haproxy-bundle-podman-1	(ocf::heartbeat:podman):        Started controller-0
   haproxy-bundle-podman-2	(ocf::heartbeat:podman):        Started controller-1
 podman container set: ovn-dbs-bundle [192.168.24.1:8787/rhosp15/openstack-ovn-northd:pcmklatest]
   ovn-dbs-bundle-0     (ocf::ovn:ovndb-servers):	Master controller-2
   ovn-dbs-bundle-1     (ocf::ovn:ovndb-servers):	Slave controller-0
   ovn-dbs-bundle-2     (ocf::ovn:ovndb-servers):	Slave controller-1
stonith-fence_compute-fence-nova        (stonith:fence_compute):        Started controller-2
 Clone Set: compute-unfence-trigger-clone [compute-unfence-trigger]
     Started: [ overcloud-novacomputeiha-0 overcloud-novacomputeiha-1 ]
nova-evacuate   (ocf::openstack:NovaEvacuate):  Started controller-2
stonith-fence_ipmilan-525400e66aee	(stonith:fence_ipmilan):        Started controller-0
stonith-fence_ipmilan-52540022ec28	(stonith:fence_ipmilan):        Started controller-1
stonith-fence_ipmilan-5254009c0c88	(stonith:fence_ipmilan):        Started controller-0
stonith-fence_ipmilan-5254007c0d15	(stonith:fence_ipmilan):        Started controller-2
stonith-fence_ipmilan-525400917ae8	(stonith:fence_ipmilan):        Started controller-1
 podman container: openstack-cinder-volume [192.168.24.1:8787/rhosp15/openstack-cinder-volume:pcmklatest]
   openstack-cinder-volume-podman-0     (ocf::heartbeat:podman):        Started controller-0
Comment 6 errata-xmlrpc 2019-09-10 13:13:15 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2700