Bug 2179130

Summary: [RFE] Filter ops in RGW Ops log prior to storage
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: James Poole <jpoole>
Component: RGWAssignee: Yuval Lifshitz <ylifshit>
Status: ASSIGNED --- QA Contact: Madhavi Kasturi <mkasturi>
Severity: low Docs Contact:
Priority: unspecified    
Version: 5.3CC: ceph-eng-bugs, cephqe-warriors, mbenjamin, ylifshit
Target Milestone: ---Keywords: FutureFeature
Target Release: 8.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description James Poole 2023-03-16 16:53:34 UTC
Description of problem:
When using a load balancer in front of RGWs all healthchecks are recorded in the ops log. This is unnecessary and causes a large build up of logs.

Version-Release number of selected component (if applicable):
16.2.10-94.el8cp
16.2.10-138.el8cp

How reproducible:
Consistently and easily.

Steps to Reproduce:

1. Deploy CEPH with RGWs and set "rgw_enable_ops_log" to true

2. Configure HAProxy with RGW backends:

frontend prod_rgws
    bind x.x.x.x:443 ssl crt /var/lib/haproxy/cert.pem
    http-request set-header rgwenv prod
    default_backend rgws_backend
  
backend rgws_backend
    server rgw0 127.0.0.1:8080 weight 100 check 
    balance static-rr
    option forwardfor
    option httpchk GET /swift/healthcheck

3. Observe logs in <zone name>.rgw.logs pool with "radosgw-admin log show --object [ object name from radosgw-admin log list]"

Actual results:
The healthcheck operations appearing in the ops log. Example message:
        {
            "bucket": "",
            "time": "2023-03-16T16:45:57.317366Z",
            "time_local": "2023-03-16T12:45:57.317366-0400",
            "remote_addr": "127.0.0.1",
            "user": "",
            "operation": "get_health_check",
            "uri": "GET /swift/healthcheck HTTP/1.0",
            "http_status": "200",
            "error_code": "",
            "bytes_sent": 0,
            "bytes_received": 0,
            "object_size": 0,
            "total_time": 0,
            "user_agent": "",
            "referrer": "",
            "trans_id": "tx0000008217415f8131a59-00641347c5-a4be9-wdc-prod",
            "temp_url": false
        }

Expected results:
This is expected but not desired. Ideally these healthchecks could be filtered out of the ops log prior to being stored in the log pool.

Additional info:
Providing a mechanism to exclude the "get_health_check" operation would prevent the ops log from growing extremely large.