Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 2139484

Summary: Unable to bind Ingress to port 2049
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Goutham Pacha Ravi <gouthamr>
Component: CephadmAssignee: Adam King <adking>
Status: CLOSED NOTABUG QA Contact: Manasa <mgowri>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.0CC: adking, cephqe-warriors, fpantano, gouthamr
Target Milestone: ---   
Target Release: 6.1   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-02-09 14:57:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Goutham Pacha Ravi 2022-11-02 16:46:11 UTC
Description of problem:

When Ceph NFS is consumed via the ingress service, binding the ingress to port 2049 would allow clients to mount exports directly without having to know/specify a different port. However, we're unable to do so - "TCP Port(s) '2049,9049' required for haproxy already in use"


   # ceph nfs cluster create cephfs --placement=devstack.localdomain --ingress  --virtual-ip 192.168.24.75/24 --port 2049
    NFS Cluster Created Successfully

  # ceph orch ps
    NAME                             HOST                  PORTS    STATUS        REFRESHED  AGE  MEM USE  MEM LIM  VERSION  IMAGE ID      CONTAINER ID
    crash.devstack                   devstack.localdomain           running (2h)     4m ago   2h    7201k        -  17.2.3   0912465dcea5  b79b84f7c463
    mgr.devstack.jykurw              devstack.localdomain           running (2h)     4m ago   2h     432M        -  17.2.3   0912465dcea5  e70c1bd70f43
    mgr.devstack.localdomain.yworig  devstack.localdomain  *:9283   running (2h)     4m ago   2h     539M        -  17.2.3   0912465dcea5  4b0232569f76
    mon.devstack.localdomain         devstack.localdomain           running (2h)     4m ago   2h     440M    2048M  17.2.3   0912465dcea5  1e53d1de366a
    nfs.cephfs.0.0.devstack.kfvcel   devstack.localdomain  *:12049  running (4m)     4m ago   4m    9353k        -  4.0      0912465dcea5  dab8ed3fd5bb
    osd.0                            devstack.localdomain           running (2h)     4m ago   2h     101M    4096M  17.2.3   0912465dcea5  88a226cee3e7


  # ceph -W cephadm --watch-debug
      cluster:
        id:     15b994ed-4341-4522-94e9-56e75279659a
        health: HEALTH_WARN
                Failed to place 2 daemon(s)

  data:
    pools:   2 pools, 9 pgs
    objects: 5 objects, 449 KiB
    usage:   21 MiB used, 10 GiB / 10 GiB avail
    pgs:     9 active+clean

  io:
    client:   767 B/s rd, 511 B/s wr, 0 op/s rd, 0 op/s wr


Failure logs:

  2022-09-20 07:42:53,430 7faa31849740 INFO Deploy daemon haproxy.nfs.cephfs.devstack.ultjer ...
  2022-09-20 07:42:53,751 7faa31849740 DEBUG stat: 0 0
  2022-09-20 07:42:53,906 7faa31849740 INFO Verifying port 2049 ...
  2022-09-20 07:42:53,907 7faa31849740 WARNING Cannot bind to IP 0.0.0.0 port 2049: [Errno 98] Address already in use
  2022-09-20 07:42:53,907 7faa31849740 INFO Verifying port 9049 ...
  2022-09-20 07:42:53,907 7faa31849740 ERROR ERROR: TCP Port(s) '2049,9049' required for haproxy already in use



  netstat shows the following:

  LISTEN     0       64                  0.0.0.0:2049               0.0.0.0:*
  LISTEN     0       128                       :12049                    *:      users:(("ganesha.nfsd",pid=611540,fd=35))
  LISTEN     0       64                     [::]:2049                  [::]:*



1. a process is bound on 2049, and it's not haproxy
2. ganesha, which is bound on $port + [1], is bound on '*', which is a limitation for the "ceph nfs cluster" cli
3. Problem occurs even when using a spec instead of the CLI:

  service_type: ingress
  service_id: nfs.cephfs
  placement:
    count: 1
  spec:
    backend_service: nfs.cephfs
    frontend_port: 2049
    monitor_port: 8000
    virtual_ip: 192.168.24.75/24"



Version-Release number of selected component (if applicable): RHCS 6.x+