Bug 2231452 - RFE: CU can not deploy multiple RGW ingress service with different virtual IPs and the same port
Summary: RFE: CU can not deploy multiple RGW ingress service with different virtual IP...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Cephadm
Version: 5.3
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 7.0
Assignee: Adam King
QA Contact: Sayalee
Rivka Pollack
URL:
Whiteboard:
Depends On:
Blocks: 2237662
TreeView+ depends on / blocked
 
Reported: 2023-08-11 15:06 UTC by Raimund Sacherer
Modified: 2025-02-21 10:05 UTC (History)
10 users (show)

Fixed In Version: ceph-18.2.0-6.el9cp
Doc Type: Bug Fix
Doc Text:
.HA proxy now binds to the frontend port on the VIP Previously, in Cephadm, multiple ingress services could not be deployed on the same host with the same frontend port as the port binding occurred across all host networks. With this fix, multiple ingress services can now be present on the same host with the same frontend port as long as the services use different VIPs and different monitoring ports are set for the ingress service in the specification.
Clone Of:
Environment:
Last Closed: 2023-12-13 15:21:54 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-7200 0 None None None 2023-08-11 15:07:16 UTC
Red Hat Product Errata RHBA-2023:7780 0 None None None 2023-12-13 15:21:59 UTC

Description Raimund Sacherer 2023-08-11 15:06:48 UTC
Description of problem:
CU try to setup ingress services with multiple virtual IPs configured in the spec file, but with the same port. When trying to deploy the second ingress service, deployment failes with the following error: 

```
Cannot bind to IP 0.0.0.0 port 443: [Errno 98] Address already in use
Verifying port 19679 ...
ERROR: TCP Port(s) '443,19679' required for haproxy already in use
    Failed while placing haproxy.<name> on <server>: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --$ormat {{.State.Status}} ceph-<fsid>-haproxy-<name>
```


This error is caused by cephadm. As it is presumably not aware about the virtual (or any) service IP the service want's to use it only checks if the port is already used by binding to the port on `0.0.0.0`. This naturally fails if we have already a service bound to this port, even if it is on a different IP than the configuration in the spec calls for. 

There are a couple of related upstream trackers:
https://tracker.ceph.com/issues/45876
https://tracker.ceph.com/issues/45565

If cephadm could be made aware of the service URL the ingress service wants, it could then try to bind on it (e.g. the virtual IP + Port) and we would not fail if this is actually free, even if there are other ingress services using the same port, but bound to different virtual IPs. 


This is the 2nd time this issues comes up in support for me during the last couple of month.

I also think it should not be that hard to implement. When deploying a service the yaml spec already contains port and IP information, cephadm should be able to extract this when looking if the port is actually free and use the virtual ip from the spec instead of '0.0.0.0'.



Version-Release number of selected component (if applicable):


How reproducible:
every time

Steps to Reproduce:
1. deploy an ingress service with a virtual IP and port 443
2. deploy another ingress service with a different virtual IP and port 443 on the same host

Actual results:
2nd deployment fails with above error message.

Expected results:
service should be deployed.

Additional info:
N/A

Comment 10 errata-xmlrpc 2023-12-13 15:21:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 7.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:7780


Note You need to log in before you can comment on or make changes to this bug.