Description of problem: https://tracker.ceph.com/issues/62558 Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Please specify the severity of this bug. Severity is defined here: https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.
Verified this BZ with # ceph --version ceph version 18.2.1-42.el9cp (99eeea5747b2d0f6873630efda2d6d2d50d7ae61) reef (stable) [ceph: root@argo016 home]# ceph mgr services { "dashboard": "https://10.8.128.216:8443/", "prometheus": "http://10.8.128.216:9283/" } ====== ceph metrics output --> URL -> http://10.8.128.199:9587/metrics ====== # HELP exposer_transferred_bytes_total Transferred bytes to metrics services # TYPE exposer_transferred_bytes_total counter exposer_transferred_bytes_total 264396 # HELP exposer_scrapes_total Number of times metrics were scraped # TYPE exposer_scrapes_total counter exposer_scrapes_total 619 # HELP exposer_request_latencies Latencies of serving scrape requests, in microseconds # TYPE exposer_request_latencies summary exposer_request_latencies_count 619 exposer_request_latencies_sum 140015 exposer_request_latencies{quantile="0.5"} 229 exposer_request_latencies{quantile="0.9"} 312 exposer_request_latencies{quantile="0.99"} 312 # HELP mdcache_cache_hits_total Counter for total cache hits in mdcache. # TYPE mdcache_cache_hits_total counter mdcache_cache_hits_total{operation="lookup"} 2 mdcache_cache_hits_total{operation="getattr"} 12 # HELP mdcache_cache_misses_total Counter for total cache misses in mdcache. # TYPE mdcache_cache_misses_total counter mdcache_cache_misses_total{operation="lookup"} 207 mdcache_cache_misses_total{operation="getattr"} 250291 # HELP mdcache_cache_misses_by_export_total Counter for total cache misses in mdcache, by export. # TYPE mdcache_cache_misses_by_export_total counter mdcache_cache_misses_by_export_total{export="export_id=1",operation="lookup"} 204 mdcache_cache_misses_by_export_total{export="export_id=1",operation="getattr"} 250288 # HELP rpcs_received_total Counter for total RPCs received. # TYPE rpcs_received_total counter rpcs_received_total 238566 # HELP rpcs_completed_total Counter for total RPCs completed. # TYPE rpcs_completed_total counter rpcs_completed_total 238566 # HELP client_bytes_received_total Total request bytes by client. # TYPE client_bytes_received_total counter client_bytes_received_total{client="10.8.128.222",operation="write"} 0 # HELP client_bytes_sent_total Total response bytes sent by client. # TYPE client_bytes_sent_total counter client_bytes_sent_total{client="10.8.128.222",operation="write"} 117440512 # HELP nfs_bytes_received_total Total request bytes. # TYPE nfs_bytes_received_total counter nfs_bytes_received_total{operation="write"} 0 # HELP nfs_bytes_sent_total Total response bytes. # TYPE nfs_bytes_sent_total counter nfs_bytes_sent_total{operation="write"} 117440512 # HELP nfs_bytes_received_by_export_total Total request bytes by export. # TYPE nfs_bytes_received_by_export_total counter nfs_bytes_received_by_export_total{export="export_id=1",operation="write"} 0 # HELP nfs_bytes_sent_by_export_total Total response bytes by export. # TYPE nfs_bytes_sent_by_export_total counter nfs_bytes_sent_by_export_total{export="export_id=1",operation="write"} 117440512 # HELP rpcs_in_flight Number of NFS requests received or in flight. # TYPE rpcs_in_flight gauge rpcs_in_flight 1 # HELP nfs_request_size_bytes Request size in bytes. # TYPE nfs_request_size_bytes histogram nfs_request_size_bytes_count{operation="write"} 112 nfs_request_size_bytes_sum{operation="write"} 117440512 nfs_request_size_bytes_bucket{operation="write",le="2"} 0 nfs_request_size_bytes_bucket{operation="write",le="4"} 0 nfs_request_size_bytes_bucket{operation="write",le="8"} 0 nfs_request_size_bytes_bucket{operation="write",le="16"} 0 nfs_request_size_bytes_bucket{operation="write",le="32"} 0 nfs_request_size_bytes_bucket{operation="write",le="64"} 0 nfs_request_size_bytes_bucket{operation="write",le="128"} 0 nfs_request_size_bytes_bucket{operation="write",le="256"} 0 nfs_request_size_bytes_bucket{operation="write",le="512"} 0 nfs_request_size_bytes_bucket{operation="write",le="1024"} 0 nfs_request_size_bytes_bucket{operation="write",le="2048"} 0 nfs_request_size_bytes_bucket{operation="write",le="4096"} 0 nfs_request_size_bytes_bucket{operation="write",le="8192"} 0 nfs_request_size_bytes_bucket{operation="write",le="16384"} 0 nfs_request_size_bytes_bucket{operation="write",le="32768"} 0 nfs_request_size_bytes_bucket{operation="write",le="65536"} 0 nfs_request_size_bytes_bucket{operation="write",le="131072"} 0 nfs_request_size_bytes_bucket{operation="write",le="262144"} 0 nfs_request_size_bytes_bucket{operation="write",le="524288"} 0 nfs_request_size_bytes_bucket{operation="write",le="1048576"} 112 nfs_request_size_bytes_bucket{operation="write",le="2097152"} 112 nfs_request_size_bytes_bucket{operation="write",le="4194304"} 112 nfs_request_size_bytes_bucket{operation="write",le="8388608"} 112 nfs_request_size_bytes_bucket{operation="write",le="16777216"} 112 nfs_request_size_bytes_bucket{operation="write",le="+Inf"} 112 # HELP nfs_response_size_bytes Response size in bytes. # TYPE nfs_response_size_bytes histogram nfs_response_size_bytes_count{operation="write"} 112 nfs_response_size_bytes_sum{operation="write"} 117440512 nfs_response_size_bytes_bucket{operation="write",le="2"} 0 nfs_response_size_bytes_bucket{operation="write",le="4"} 0 nfs_response_size_bytes_bucket{operation="write",le="8"} 0 nfs_response_size_bytes_bucket{operation="write",le="16"} 0 nfs_response_size_bytes_bucket{operation="write",le="32"} 0 nfs_response_size_bytes_bucket{operation="write",le="64"} 0 nfs_response_size_bytes_bucket{operation="write",le="128"} 0 nfs_response_size_bytes_bucket{operation="write",le="256"} 0 nfs_response_size_bytes_bucket{operation="write",le="512"} 0 nfs_response_size_bytes_bucket{operation="write",le="1024"} 0 nfs_response_size_bytes_bucket{operation="write",le="2048"} 0 nfs_response_size_bytes_bucket{operation="write",le="4096"} 0 nfs_response_size_bytes_bucket{operation="write",le="8192"} 0 nfs_response_size_bytes_bucket{operation="write",le="16384"} 0 nfs_response_size_bytes_bucket{operation="write",le="32768"} 0 nfs_response_size_bytes_bucket{operation="write",le="65536"} 0 nfs_response_size_bytes_bucket{operation="write",le="131072"} 0 nfs_response_size_bytes_bucket{operation="write",le="262144"} 0 nfs_response_size_bytes_bucket{operation="write",le="524288"} 0 nfs_response_size_bytes_bucket{operation="write",le="1048576"} 112 nfs_response_size_bytes_bucket{operation="write",le="2097152"} 112 nfs_response_size_bytes_bucket{operation="write",le="4194304"} 112 nfs_response_size_bytes_bucket{operation="write",le="8388608"} 112 nfs_response_size_bytes_bucket{operation="write",le="16777216"} 112 nfs_response_size_bytes_bucket{operation="write",le="+Inf"} 112 # HELP nfs_request_size_by_export_bytes Request size by export in bytes. # TYPE nfs_request_size_by_export_bytes histogram nfs_request_size_by_export_bytes_count{export="export_id=1",operation="write"} 112 nfs_request_size_by_export_bytes_sum{export="export_id=1",operation="write"} 117440512 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="2"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="4"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="8"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="16"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="32"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="64"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="128"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="256"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="512"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="1024"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="2048"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="4096"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="8192"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="16384"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="32768"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="65536"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="131072"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="262144"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="524288"} 0 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="1048576"} 112 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="2097152"} 112 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="4194304"} 112 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="8388608"} 112 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="16777216"} 112 nfs_request_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="+Inf"} 112 # HELP nfs_response_size_by_export_bytes Response size by export in bytes. # TYPE nfs_response_size_by_export_bytes histogram nfs_response_size_by_export_bytes_count{export="export_id=1",operation="write"} 112 nfs_response_size_by_export_bytes_sum{export="export_id=1",operation="write"} 117440512 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="2"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="4"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="8"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="16"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="32"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="64"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="128"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="256"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="512"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="1024"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="2048"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="4096"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="8192"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="16384"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="32768"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="65536"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="131072"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="262144"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="524288"} 0 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="1048576"} 112 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="2097152"} 112 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="4194304"} 112 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="8388608"} 112 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="16777216"} 112 nfs_response_size_by_export_bytes_bucket{export="export_id=1",operation="write",le="+Inf"} 112 ======= NFS Target are present under --> URL --> http://argo016.ceph.redhat.com:9095/targets?search= (Attached screenshot) ====== Delete the nfs cluster. NFS-Target and metrics on NFS-Ganesha node goes away. Moving this BZ to verified state
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Critical: Red Hat Ceph Storage 7.1 security, enhancements, and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:3925