Bug 1914169 - [DOC][RHCS4] Add steps for numa configurations and monitoring
Summary: [DOC][RHCS4] Add steps for numa configurations and monitoring
Keywords:
Status: NEW
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Documentation
Version: 4.1
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: Backlog
Assignee: Anjana Suparna Sriram
QA Contact: Pawan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-08 09:34 UTC by Deepu K S
Modified: 2023-07-25 18:12 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-2022 0 None None None 2021-10-12 13:22:48 UTC

Description Deepu K S 2021-01-08 09:34:41 UTC
Describe the issue:
With RHCS 4, we don't have a clear documentation for NUMA configurations.
For example : Pinning an OSD to specific CPU.

From the ceph OSD configs, auto-affinity is enabled but it couldn't be validated.
# ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config show | grep numa
    "osd_numa_auto_affinity": "true",
    "osd_numa_node": "-1",
    "osd_numa_prefer_iface": "true",

This article helped to understand the highlights in Nautilus,
https://ceph.io/community/new-in-nautilus-rados-highlights/

However the `ceph osd numa-status` comes empty.
# ceph osd numa-status
OSD HOST   NETWORK STORAGE AFFINITY CPUS 
  0 ceph81       -       -        - -    
  1 ceph80       -       -        - -    
  2 ceph82       -       -        - -    
  3 ceph81       -       -        - -    
  4 ceph80       -       -        - -    
  5 ceph82       -       -        - -    

Also a runtime change is not possible.
# ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set osd_numa_node 0
{
    "error": "error setting 'osd_numa_node' to '0': (38) Function not implemented"
}

# ceph tell osd.1 config set osd_numa_node 0
Error ENOSYS: Configuration option 'osd_numa_node' may not be modified at runtime

We manually added the osd numa node in ceph.conf and restarted osd, which does apply the changes. But not able to validate it properly.

Document URL:
RHCS 4 Administration Guide
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/4/html-single/administration_guide/index

Chapter/Section Number and Title:

Product Version: RHCS 4.1

We need a document for configuring and validation of NUMA settings for OSD's.


Note You need to log in before you can comment on or make changes to this bug.