.HA proxy now binds to the frontend port on the VIP
Previously, in Cephadm, multiple ingress services could not be deployed on the same host with the same frontend port as the port binding occurred across all host networks.
With this fix, multiple ingress services can now be present on the same host with the same frontend port as long as the services use different VIPs and different monitoring ports are set for the ingress service in the specification.
DescriptionRaimund Sacherer
2023-08-11 15:06:48 UTC
Description of problem:
CU try to setup ingress services with multiple virtual IPs configured in the spec file, but with the same port. When trying to deploy the second ingress service, deployment failes with the following error:
```
Cannot bind to IP 0.0.0.0 port 443: [Errno 98] Address already in use
Verifying port 19679 ...
ERROR: TCP Port(s) '443,19679' required for haproxy already in use
Failed while placing haproxy.<name> on <server>: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --$ormat {{.State.Status}} ceph-<fsid>-haproxy-<name>
```
This error is caused by cephadm. As it is presumably not aware about the virtual (or any) service IP the service want's to use it only checks if the port is already used by binding to the port on `0.0.0.0`. This naturally fails if we have already a service bound to this port, even if it is on a different IP than the configuration in the spec calls for.
There are a couple of related upstream trackers:
https://tracker.ceph.com/issues/45876https://tracker.ceph.com/issues/45565
If cephadm could be made aware of the service URL the ingress service wants, it could then try to bind on it (e.g. the virtual IP + Port) and we would not fail if this is actually free, even if there are other ingress services using the same port, but bound to different virtual IPs.
This is the 2nd time this issues comes up in support for me during the last couple of month.
I also think it should not be that hard to implement. When deploying a service the yaml spec already contains port and IP information, cephadm should be able to extract this when looking if the port is actually free and use the virtual ip from the spec instead of '0.0.0.0'.
Version-Release number of selected component (if applicable):
How reproducible:
every time
Steps to Reproduce:
1. deploy an ingress service with a virtual IP and port 443
2. deploy another ingress service with a different virtual IP and port 443 on the same host
Actual results:
2nd deployment fails with above error message.
Expected results:
service should be deployed.
Additional info:
N/A
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Red Hat Ceph Storage 7.0 Bug Fix update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2023:7780
Description of problem: CU try to setup ingress services with multiple virtual IPs configured in the spec file, but with the same port. When trying to deploy the second ingress service, deployment failes with the following error: ``` Cannot bind to IP 0.0.0.0 port 443: [Errno 98] Address already in use Verifying port 19679 ... ERROR: TCP Port(s) '443,19679' required for haproxy already in use Failed while placing haproxy.<name> on <server>: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --$ormat {{.State.Status}} ceph-<fsid>-haproxy-<name> ``` This error is caused by cephadm. As it is presumably not aware about the virtual (or any) service IP the service want's to use it only checks if the port is already used by binding to the port on `0.0.0.0`. This naturally fails if we have already a service bound to this port, even if it is on a different IP than the configuration in the spec calls for. There are a couple of related upstream trackers: https://tracker.ceph.com/issues/45876 https://tracker.ceph.com/issues/45565 If cephadm could be made aware of the service URL the ingress service wants, it could then try to bind on it (e.g. the virtual IP + Port) and we would not fail if this is actually free, even if there are other ingress services using the same port, but bound to different virtual IPs. This is the 2nd time this issues comes up in support for me during the last couple of month. I also think it should not be that hard to implement. When deploying a service the yaml spec already contains port and IP information, cephadm should be able to extract this when looking if the port is actually free and use the virtual ip from the spec instead of '0.0.0.0'. Version-Release number of selected component (if applicable): How reproducible: every time Steps to Reproduce: 1. deploy an ingress service with a virtual IP and port 443 2. deploy another ingress service with a different virtual IP and port 443 on the same host Actual results: 2nd deployment fails with above error message. Expected results: service should be deployed. Additional info: N/A