Bug 2156919

Summary: public_network is not set as global while bootsraping via cephadm
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Sufiyan Siddiqui <ssiddiqui>
Component: CephadmAssignee: Redouane Kachach Elhichou <rkachach>
Status: CLOSED ERRATA QA Contact: Aditya Ramteke <aramteke>
Severity: medium Docs Contact: Akash Raj <akraj>
Priority: unspecified    
Version: 5.2CC: adking, akraj, cephqe-warriors, hklein, msaini, rkachach, rsachere, saraut, tserlin, vereddy
Target Milestone: ---   
Target Release: 6.1z1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: ceph-17.2.6-84.el9cp Doc Type: Enhancement
Doc Text:
._public_network_ parameter can now have configuration options, such as `global` or `mon` Previously, in `cephadm`, the `public_network` parameter was always set as a part of the `mon` configuration section during a cluster bootstrap without providing any configuration option to alter this behavior. With this enhancement, you can specify the configuration options, such as `global` or `mon` for the `public_network` parameter during cluster bootstrap by utilizing the Ceph configuration file.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-03 16:45:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2221020    

Description Sufiyan Siddiqui 2022-12-29 14:26:39 UTC 
Description of problem:

We don't have any option too make sure public network is set as global(both for mon and osds) while bootsraping ceph cluster with cephadm. After the ceph bootsraping process public_network is generally set for mon only but not for osds.

~~~
[root@ceph-1 ceph-exec]# ceph config dump | grep -i public_network
  mon                advanced  public_network                         192.168.xx.yy/27                                                                                                 *
~~~

- osds were only listening to cluster_network

~~~
[root@ceph-1 ceph-exec]# netstat -tulpn  | grep ceph
tcp        0      0 192.168.aa.bb:6827      0.0.0.0:*             LISTEN      6879/ceph-osd
tcp        0      0 0.0.0.0:6828            0.0.0.0:*             LISTEN      6879/ceph-osd
tcp        0      0 0.0.0.0:6829            0.0.0.0:*             LISTEN      6879/ceph-osd
tcp        0      0 192.168.aa.bb:6830      0.0.0.0:*             LISTEN      6879/ceph-osd
tcp        0      0 192.168.aa.bb:6831      0.0.0.0:*             LISTEN      6879/ceph-osd
tcp        0      0 0.0.0.0:6832            0.0.0.0:*             LISTEN      6911/ceph-osd
tcp        0      0 0.0.0.0:6800            0.0.0.0:*             LISTEN      6680/ceph-osd
tcp        0      0 0.0.0.0:6833            0.0.0.0:*             LISTEN      6911/ceph-osd
~~~

This will create issue when we integrate the external ceph with OpenStack environment as client(nova,cinder,glance etc) will not be able to connect to osd via public_network.

>> To Overcome this issue we have to update the ceph config and set public_network to global.

~~~
# ceph config set global public_network 192.168.xx.yy/27
# ceph orch daemon reconfig mon.host
# ceph orch daemon reconfig osd.host
~~~

~~~
[root@ceph-1 ceph-exec]# ceph config dump | grep -i public_network
global               advanced  public_network                         192.168.xx.yy/27                                                                                                *
[root@ceph-1 ceph-exec]# netstat -nltp | grep -i ceph
tcp        0      0 192.168.aa.bb:6800    0.0.0.0:*               LISTEN      2250215/ceph-osd
tcp        0      0 192.168.xx.yy:6800    0.0.0.0:*               LISTEN      2250215/ceph-osd
tcp        0      0 192.168.aa.bb:6801    0.0.0.0:*               LISTEN      2250215/ceph-osd
tcp        0      0 192.168.xx.yy:6801    0.0.0.0:*               LISTEN      2250215/ceph-osd
tcp        0      0 192.168.aa.bb:6802    0.0.0.0:*               LISTEN      2250215/ceph-osd
tcp        0      0 192.168.xx.yy:6802    0.0.0.0:*               LISTEN      2250215/ceph-osd
tcp        0      0 192.168.aa.bb:6803    0.0.0.0:*               LISTEN      2250215/ceph-osd
~~~


In our opinion it seems to be a bug as bootstrap process(via cephadm) should be capable of handing public_network properly(i.e, marking it global instead of restricting it to just mon only)

Version : Red Hat Ceph Storage 5.0
Comment 12 errata-xmlrpc 2023-08-03 16:45:09 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 6.1 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:4473