Bug 2209616

Summary:	test_rgw_kafka_notifications test fails with "RGW bucket notification is not working as expected." Error
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Abdul Kandathil (IBM) <akandath>
Component:	ceph	Assignee:	Yuval Lifshitz <ylifshit>
ceph sub component:	RGW	QA Contact:	Shrivaibavi Raghaventhiran <sraghave>
Status:	ON_QA ---	Docs Contact:
Severity:	medium
Priority:	unspecified	CC:	bniver, jthottan, mbenjamin, muagarwa, odf-bz-bot, sostapov, ylifshit
Version:	4.13	Flags:	muagarwa: needinfo? (mbenjamin)
Target Milestone:	---
Target Release:	ODF 4.14.0
Hardware:	s390x
OS:	Linux
Whiteboard:
Fixed In Version:	4.14.0-110	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Abdul Kandathil (IBM) 2023-05-24 09:02:43 UTC

Description of problem (please be detailed as possible and provide log
snippets):
Tier1 test "tests/e2e/workloads/app/amq/test_rgw_kafka_notifications.py::TestRGWAndKafkaNotifications::test_rgw_kafka_notifications" fails with below error.

>           raise Exception(
                "Error: Messages are not recieved from Kafka side."
                "RGW bucket notification is not working as expected."
            )
E           Exception: Error: Messages are not recieved from Kafka side.RGW bucket notification is not working as expected.



Version of all relevant components (if applicable):
OCP 4.13.0-rc.6
ODF 4.13.0-203


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?
yes

Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Install OCP and ODF 4.13
2. Execute tier1 test "tests/e2e/workloads/app/amq/test_rgw_kafka_notifications.py::TestRGWAndKafkaNotifications::test_rgw_kafka_notifications"
3.


Actual results:


Expected results:


Additional info:
Logs available in google drive : https://drive.google.com/file/d/180nzo7xi8cWU_wy3aVb1MteOPYI1AiNp/view?usp=sharing

Comment 2 Mudit Agarwal 2023-05-25 08:30:02 UTC

Jiffin, PTAL

Comment 4 Abdul Kandathil (IBM) 2023-05-30 11:57:40 UTC

Reran the test again after setting debug level as mentioned.

sh-5.1$ ceph config set client.rgw.ocs.storagecluster.cephobjectstore.a debug_rgw 20
sh-5.1$ ceph config get client.rgw.ocs.storagecluster.cephobjectstore.a debug_rgw
20/20
sh-5.1$


please find the logs here : https://drive.google.com/file/d/1em1jGjkYh3Gax6-JMP_4dGpMwbcNdu_n/view?usp=sharing