Bug 2303947

Summary: [8.0][rgw][notifications]: radosgw-admin notification get/list output json is incorrect if configured with metadata filter
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Hemanth Sai <hmaheswa>
Component: RGWAssignee: Yuval Lifshitz <ylifshit>
Status: CLOSED ERRATA QA Contact: Chaithra <ckulal>
Severity: low Docs Contact:
Priority: unspecified    
Version: 8.0CC: akraj, ceph-eng-bugs, cephqe-warriors, ckulal, mbenjamin, rpollack, ylifshit
Target Milestone: ---Keywords: Automation, Regression
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ceph-19.1.0-32 Doc Type: Bug Fix
Doc Text:
.Malformed JSON of radosgw-admin notification output is now corrected Previously, when bucket notifications were configured with metadata and tag filters, the output of `radosgw-admin` notification for the get/list output was malformed JSON. As a result, any JSON parser, such as jquery, reading the output would fail. With this fix, the JSON output for `radosgw-admin` is corrected.
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-11-25 09:05:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2317218    

Description Hemanth Sai 2024-08-10 11:51:18 UTC
Description of problem:
radosgw-admin notification get/list output json is missing closing brace at the end 
and S3Metadata and S3Tags are inside of S3Key.
if configured with metadata filter

[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ radosgw-admin notification list --bucket bobbyd.20-bucky-2877-0
{
    "notifications": [
        {
            "TopicArn": "arn:aws:sns:default::cephci-kafka-broker-ack-type-4143182e6a864a04",
            "Id": "notification-Delete",
            "Events": [
                "s3:ObjectCreated:*",
                "s3:ObjectRemoved:*"
            ],
            "Filter": {
                "S3Key": {
                    "FilterRules": [
                        {
                            "Name": "prefix",
                            "Value": "prefix1"
                        }
                    ],
                    "S3Metadata": {},
                    "S3Tags": {}
                }
            }
        }
    ][cephuser@ceph-ck-squid-8i54kq-node5 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ radosgw-admin notification get --bucket bobbyd.20-bucky-2877-0 --notification-id notification-Delete
{
    "TopicArn": "arn:aws:sns:default::cephci-kafka-broker-ack-type-4143182e6a864a04",
    "Id": "notification-Delete",
    "Events": [
        "s3:ObjectCreated:*",
        "s3:ObjectRemoved:*"
    ],
    "Filter": {
        "S3Key": {
            "FilterRules": [
                {
                    "Name": "prefix",
                    "Value": "prefix1"
                }
            ],
            "S3Metadata": {},
            "S3Tags": {}
        }
    }[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ radosgw-admin notification get --bucket bobbyd.20-bucky-2877-0 --notification-id notification-Delete | jq .
parse error: Unfinished JSON term at EOF at line 19, column 5
[cephuser@ceph-ck-squid-8i54kq-node5 ~]$ 



pass log on 7.1:
http://magna002.ceph.redhat.com/cephci-jenkins/hsm/PR_rgw_admin_notif_commands/test_bucket_notification_kafka_broker_copy.console.log


fail log on 8.0:
http://magna002.ceph.redhat.com/cephci-jenkins/hsm/TFA_squid_notif_topic_get/test_bucket_notification_kafka_broker_persistent_delete.console.log_notif_list_output_json_missing_close_bracket


Version-Release number of selected component (if applicable):
ceph version 19.1.0-21.el9cp

How reproducible:
always

Steps to Reproduce:
1.create a topic
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ aws --endpoint-url http://10.0.211.132:80 sns create-topic --name topic1
{
    "TopicArn": "arn:aws:sns:default::topic1"
}
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ 

2.put bucket notifications on the bucket with metadata filter
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ aws --endpoint-url http://10.0.211.132:80 s3api put-bucket-notification-configuration --bucket bkt1 --notification-configuration file://notif.json 
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ cat notif.json 
{
    "TopicConfigurations": [
        {
            "Id": "notif1",
            "TopicArn": "arn:aws:sns:default::topic1",
            "Events": [
                "s3:ObjectCreated:*"
            ],
            "Filter": {
                "Key": {
                    "FilterRules": [
                        {"Name": "prefix", "Value": "prefix1"}
                     ]
                 }
             }
        }
    ]
}
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ aws --endpoint-url http://10.0.211.132:80 s3api get-bucket-notification-configuration --bucket bkt1
{
    "TopicConfigurations": [
        {
            "Id": "notif1",
            "TopicArn": "arn:aws:sns:default::topic1",
            "Events": [
                "s3:ObjectCreated:*"
            ],
            "Filter": {
                "Key": {
                    "FilterRules": [
                        {
                            "Name": "prefix",
                            "Value": "prefix1"
                        }
                    ]
                }
            }
        }
    ]
}
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$

3.try radosgw-admin notification get/list. the output is missing end brace and S3Metadata, S3Tags are inside of S3Key.
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ radosgw-admin notification list --bucket bkt1
{
    "notifications": [
        {
            "TopicArn": "arn:aws:sns:default::topic1",
            "Id": "notif1",
            "Events": [
                "s3:ObjectCreated:*"
            ],
            "Filter": {
                "S3Key": {
                    "FilterRules": [
                        {
                            "Name": "prefix",
                            "Value": "prefix1"
                        }
                    ],
                    "S3Metadata": {},
                    "S3Tags": {}
                }
            }
        }
    ][cephuser@ceph-ck-squid-8i54kq-node6 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ radosgw-admin notification get --bucket bkt1 --notification-id notif1
{
    "TopicArn": "arn:aws:sns:default::topic1",
    "Id": "notif1",
    "Events": [
        "s3:ObjectCreated:*"
    ],
    "Filter": {
        "S3Key": {
            "FilterRules": [
                {
                    "Name": "prefix",
                    "Value": "prefix1"
                }
            ],
            "S3Metadata": {},
            "S3Tags": {}
        }
    }[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ 
[cephuser@ceph-ck-squid-8i54kq-node6 ~]$ 


Actual results:
the output json format is missing closing brace at the end. also S3Metadata and S3Tags are inside of S3Key.

Expected results:
expected notification get/list returns correct json syntax output

Additional info:

Comment 10 errata-xmlrpc 2024-11-25 09:05:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 8.0 security, bug fix, and enhancement updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:10216