Bug 2142651

Summary:	RFE: OSDs need ability to bind to a service IP instead of the pod IP to support RBD mirroring in OCP clusters
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Travis Nielsen <tnielsen>
Component:	ceph	Assignee:	Santosh Pillai <sapillai>
ceph sub component:	RADOS	QA Contact:	Pratik Surve <prsurve>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	unspecified
Priority:	unspecified	CC:	bniver, muagarwa, nberry, ocs-bugs, odf-bz-bot, pdhiran, rcyriac, sapillai, sheggodu
Version:	4.12	Keywords:	FutureFeature
Target Milestone:	---
Target Release:	ODF 4.13.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	2143285 (view as bug list)		Environment:
Last Closed:	2023-06-21 15:22:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2143285
Bug Blocks:

Description Travis Nielsen 2022-11-14 18:16:41 UTC

Description of problem (please be detailed as possible and provide log
snippests):

To support RBD mirroring between OCP clusters with overlapping networks (see https://issues.redhat.com/browse/RHSTOR-3866), the OSDs need to support binding to a service IP instead of the pod IP.

The mons already support this behavior with two command line parameters:
  --public-addr=<serviceIP>
  --public-bind-addr=<podIP>

The --public-bind-addr parameter was added for the mons to support Rook scenarios way back in 2017 with this PR: https://github.com/ceph/ceph/pull/16189

In the prototyping for RBD mirroring, the mons are responding on the global IPs both for traffic internal to the cluster and from external to the cluster. For more details, see the prototyping doc:
https://docs.google.com/document/d/1C5y8bX77B7YQUX4XSCKA2Gncg_BE_DViWVN83BUtPe4/edit#heading=h.dv90pv56ycn

For the global IPs to work with OSDs, now Rook needs to create a service with a global IP in front of the OSD, while the OSD daemon needs to still bind to the pod IP. Therefore, similarly to the mons, the OSDs also need to implement the flag:
 --public-bind-addr=<podIP> 


Version of all relevant components (if applicable):

This feature is in the 4.13 planning with: 
https://issues.redhat.com/browse/RHSTOR-3866


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

This is necessary for the RBD mirroring scenario across OCP clusters with overlapping networks.


Is there any workaround available to the best of your knowledge?

No


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?

3

Can this issue reproducible?

Yes, RBD mirroring doesn't work without this feature when there are overlapping networks.

Comment 8 Travis Nielsen 2023-03-06 18:44:24 UTC

This is already done for 4.13, just waiting for it to be in the build.

Comment 11 Mudit Agarwal 2023-04-03 10:57:31 UTC

Fixed in version: 4.13.0-121

Comment 18 errata-xmlrpc 2023-06-21 15:22:18 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenShift Data Foundation 4.13.0 enhancement and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:3742

Comment 19 Red Hat Bugzilla 2023-12-08 04:31:20 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days