Bug 2355899

Summary: ceph nfs IPv6 exports not working
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Goutham Pacha Ravi <gouthamr>
Component: NFS-GaneshaAssignee: Sachin Punadikar <spunadik>
NFS-Ganesha sub component: Manila QA Contact:
Status: CLOSED WORKSFORME Docs Contact:
Severity: high    
Priority: unspecified CC: cardasil, cephqe-warriors, ffilz, kkeithle, ltoscano, msaini, spunadik, vhariria
Version: 7.2   
Target Milestone: ---   
Target Release: 8.1z4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-10-15 05:11:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Goutham Pacha Ravi 2025-03-28 23:55:36 UTC
Description of problem:

I'm using rhceph:7-493 (Reef) orchestrated by cephadm; and using OpenStack Manila to create an NFS export with CephFS. My setup includes a 3-node ceph nfs cluster, front-ended by the Ingress service ( with "haproxy-protocol" as ingress mode ). The setup has a Virtual IP using an IPv6 address, and the client/s specified in the export have IPv6 addresses as well. 

I'm unsure if this problem is in the ingress service (nothing relevant in the logging) or in NFS-Ganesha (I've added verbose/FULL_DEBUG logging to this report).

OpenStack users have long used IPv6 with NFS-Ganesha, however, not with the latest version of NFS-Ganesha, and not when it is setup as a "ceph nfs" cluster service.


Version-Release number of selected component (if applicable): 

- Ceph Version rhceph:7-493
- NFS-Ganesha package versions:
nfs-utils-2.5.4-27.el9.x86_64
nfs-ganesha-selinux-5.7-5.el9cp.noarch
nfs-ganesha-5.7-5.el9cp.x86_64
nfs-ganesha-rgw-5.7-5.el9cp.x86_64
nfs-ganesha-ceph-5.7-5.el9cp.x86_64
nfs-ganesha-rados-grace-5.7-5.el9cp.x86_64
nfs-ganesha-rados-urls-5.7-5.el9cp.x86_64




Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create a Manila share (CephFS subvolume)
2. Create a Manila access rule on the share with IPv6 client ("ceph nfs export apply") 
3. Attempt to mount share on the client

Actual results:

Mount fails with mount(2): Input/output error


Expected results:

Mount succeeds


Additional info:


Manila share details
=====================


$ openstack share create nfs 1 --name share1
$ openstack share access create share1 ip 2620:cf:cf:aaaa::6b
+--------------+--------------------------------------+
| Field        | Value                                |
+--------------+--------------------------------------+
| id           | d47920bf-3a29-4a8c-9e2c-63d672e58785 |
| share_id     | 4159a122-bc1d-49b3-bbb4-d11ee5f66dd4 |
| access_level | rw                                   |
| access_to    | 2620:cf:cf:aaaa::6b                  |
| access_type  | ip                                   |
| state        | queued_to_apply                      |
| access_key   | None                                 |
| created_at   | 2025-03-28T23:21:03.300483           |
| updated_at   | None                                 |
| properties   |                                      |
+--------------+--------------------------------------+


$ openstack share access list share1
+--------------------------------------+-------------+---------------------+--------------+--------+------------+----------------------------+----------------------------+
| ID                                   | Access Type | Access To           | Access Level | State  | Access Key | Created At                 | Updated At                 |
+--------------------------------------+-------------+---------------------+--------------+--------+------------+----------------------------+----------------------------+
| d47920bf-3a29-4a8c-9e2c-63d672e58785 | ip          | 2620:cf:cf:aaaa::6b | rw           | active | None       | 2025-03-28T23:21:03.300483 | 2025-03-28T23:21:03.745510 |
+--------------------------------------+-------------+---------------------+--------------+--------+------------+----------------------------+----------------------------+

$ openstack share export location list share1
+--------------------------------------+------------------------------------------------------------------------------------------------------------------+-----------+
| ID                                   | Path                                                                                                             | Preferred |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------+-----------+
| 851bbcef-3273-442b-b1a3-fbbcbb992f36 | [2620:cf:cf:aaaa::2]:/volumes/_nogroup/97ea173d-f9fe-41ec-abb8-72518acd5585/b025b9ac-cd4a-4c00-b47a-4faa24dacb4b | True      |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------+-----------+



Ceph Details:
=============


# ceph orch ps | grep nfs
haproxy.nfs.cephfs.ceph-r1wfskb8-0.dbjetm      ceph-r1wfskb8-0  *:2049,9049               running (31h)     3m ago  31h    11.5M        -  2.4.22-f8e3218    ff05a790b4e9  336fca2a766d
haproxy.nfs.cephfs.ceph-r1wfskb8-1.qkkbfz      ceph-r1wfskb8-1  *:2049,9049               running (31h)     3m ago  31h    12.4M        -  2.4.22-f8e3218    ff05a790b4e9  7a483dcb904b
haproxy.nfs.cephfs.ceph-r1wfskb8-2.lnzmst      ceph-r1wfskb8-2  *:2049,9049               running (31h)     3m ago  31h    13.0M        -  2.4.22-f8e3218    ff05a790b4e9  f86848797a67
keepalived.nfs.cephfs.ceph-r1wfskb8-0.becivx   ceph-r1wfskb8-0                            running (31h)     3m ago  31h    1828k        -  2.2.8             617308907a17  b88da97a3b3f
keepalived.nfs.cephfs.ceph-r1wfskb8-1.tvdbbj   ceph-r1wfskb8-1                            running (31h)     3m ago  31h    1828k        -  2.2.8             617308907a17  f9e06c8633fc
keepalived.nfs.cephfs.ceph-r1wfskb8-2.eqdyqa   ceph-r1wfskb8-2                            running (31h)     3m ago  31h    1832k        -  2.2.8             617308907a17  f3382352a78d
nfs.cephfs.0.0.ceph-r1wfskb8-1.cfnejd          ceph-r1wfskb8-1  *:12049                   running (33m)     3m ago  31h    83.7M        -  5.7               11fbfe5b88bc  81d63c7af4d1
nfs.cephfs.1.0.ceph-r1wfskb8-2.raroai          ceph-r1wfskb8-2  *:12049                   running (33m)     3m ago  31h    82.2M        -  5.7               11fbfe5b88bc  b995553191c1
nfs.cephfs.2.0.ceph-r1wfskb8-0.qedgcv          ceph-r1wfskb8-0  *:12049                   running (33m)     3m ago  31h    81.2M        -  5.7               11fbfe5b88bc  bb990f3a2bee



Ceph NFS cluster:


$ ceph nfs cluster info cephfs
{
  "cephfs": {
    "backend": [
      {
        "hostname": "ceph-r1wfskb8-0",
        "ip": "2620:cf:cf:aaaa::6a",
        "port": 12049
      },
      {
        "hostname": "ceph-r1wfskb8-1",
        "ip": "2620:cf:cf:aaaa::6b",
        "port": 12049
      },
      {
        "hostname": "ceph-r1wfskb8-2",
        "ip": "2620:cf:cf:aaaa::6c",
        "port": 12049
      }
    ],
    "monitor_port": 9049,
    "port": 2049,
    "virtual_ip": "2620:cf:cf:aaaa::2"
  }
}

$ ceph nfs export info cephfs /volumes/_nogroup/97ea173d-f9fe-41ec-abb8-72518acd5585/b025b9ac-cd4a-4c00-b47a-4faa24dacb4b
{
  "access_type": "RO",
  "clients": [
    {
      "access_type": "rw",
      "addresses": [
        "2620:cf:cf:aaaa::6b"
      ],
      "squash": "none"
    }
  ],
  "cluster_id": "cephfs",
  "export_id": 1,
  "fsal": {
    "cmount_path": "/",
    "fs_name": "cephfs",
    "name": "CEPH",
    "user_id": "nfs.cephfs.cephfs"
  },
  "path": "/volumes/_nogroup/97ea173d-f9fe-41ec-abb8-72518acd5585/b025b9ac-cd4a-4c00-b47a-4faa24dacb4b",
  "protocols": [
    3,
    4
  ],
  "pseudo": "/volumes/_nogroup/97ea173d-f9fe-41ec-abb8-72518acd5585/b025b9ac-cd4a-4c00-b47a-4faa24dacb4b",
  "security_label": true,
  "squash": "none",
  "transports": [
    "TCP"
  ]
}


From the client
===============

[zuul@ceph-r1wfskb8-1 ~]$ rpcinfo -T tcp6 -s 2620:cf:cf:aaaa::2
   program version(s) netid(s)                         service     owner
    100000  2,3,4     local,udp,tcp,udp6,tcp6          portmapper  superuser
    100003  4,3       tcp6,tcp,udp6,udp                nfs         superuser
    100005  3,1       tcp6,tcp,udp6,udp                mountd      superuser


[zuul@ceph-r1wfskb8-1 ~]$ sudo mount -t nfs [2620:cf:cf:aaaa::2]:/volumes/_nogroup/97ea173d-f9fe-41ec-abb8-72518acd5585/b025b9ac-cd4a-4c00-b47a-4faa24dacb4b test -vvv
mount.nfs: timeout set for Fri Mar 28 19:27:06 2025
mount.nfs: trying text-based options 'vers=4.2,addr=2620:cf:cf:aaaa::2,clientaddr=2620:cf:cf:aaaa::6b'
mount.nfs: mount(2): Input/output error
mount.nfs: mount system call failed

Comment 1 Storage PM bot 2025-03-28 23:55:46 UTC
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.

Comment 12 vhariria 2025-05-18 09:40:09 UTC
Hi Sachin, 
Goutham provided access information out of band, however the env is no longer accessible today. 
Please let us know if further triage in needed.

Comment 20 Goutham Pacha Ravi 2025-10-15 05:11:12 UTC
(In reply to Sachin Punadikar from comment #19)
> As per discussion with Goutham, with latest code, he was not able to
> reproduce this issue.
> Waiting for more details on this from him.

Thank you Sachin!

Apologies for the late response here. Yes, I'm unable to reproduce this issue with RHOSO 18 and RHCS 8. Originally, the issue was observed with RHCS 7 (container tag: 7-493), I'll try to reproduce this with RHCS 7. However, I'll close this bug and report a new one if the issue does occur with RHCS 7. 


Ceph Image tag: '8-492'

ceph (client)  version:
ceph version 19.2.1-222.el9cp (f2cd71cc2f7b46709c2351134ac89ea3e9f609b6) squid (stable)

Version of NFS-Ganesha:

# rpm -qa | grep ganesha
nfs-ganesha-selinux-6.5-18.el9cp.noarch
nfs-ganesha-6.5-18.el9cp.x86_64
nfs-ganesha-ceph-6.5-18.el9cp.x86_64
nfs-ganesha-rados-grace-6.5-18.el9cp.x86_64
nfs-ganesha-rados-urls-6.5-18.el9cp.x86_64
nfs-ganesha-rgw-6.5-18.el9cp.x86_64
nfs-ganesha-utils-6.5-18.el9cp.x86_64

Version of haproxy:

# rpm -qa | grep haproxy
haproxy-2.4.22-4.el9.x86_64



[zuul@ceph-4b8r13sm-0 ~]$ sudo netstat -anp | grep ganesha
tcp6       0      0 :::50051                :::*                    LISTEN      674756/ganesha.nfsd
tcp6       0      0 :::9587                 :::*                    LISTEN      674756/ganesha.nfsd
tcp6       0      0 :::12049                :::*                    LISTEN      674756/ganesha.nfsd
tcp6       0      0 :::35843                :::*                    LISTEN      674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::48316 2620:cf:cf:aaaa::6:6800 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::48284 2620:cf:cf:aaaa::6:6804 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::37740 2620:cf:cf:aaaa::6:3300 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::37720 2620:cf:cf:aaaa::6:3300 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::38992 2620:cf:cf:aaaa::6:6808 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::34506 2620:cf:cf:aaaa::6:6800 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::46436 2620:cf:cf:aaaa::6:6812 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::37706 2620:cf:cf:aaaa::6:3300 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::34498 2620:cf:cf:aaaa::6:6800 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::37324 2620:cf:cf:aaaa::6:6800 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::48302 2620:cf:cf:aaaa::6:6800 ESTABLISHED 674756/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::44354 2620:cf:cf:aaaa::6:6802 ESTABLISHED 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17506330 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17506331 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17506333 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17506332 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17502972 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17502970 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17502981 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17502982 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17502971 674756/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     17502969 674756/ganesha.nfsd


ceph-1

[zuul@ceph-4b8r13sm-1 ~]$ sudo netstat -anp | grep ganesha
tcp6       0      0 :::9587                 :::*                    LISTEN      691585/ganesha.nfsd
tcp6       0      0 :::50051                :::*                    LISTEN      691585/ganesha.nfsd
tcp6       0      0 :::12049                :::*                    LISTEN      691585/ganesha.nfsd
tcp6       0      0 :::43483                :::*                    LISTEN      691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::47312 2620:cf:cf:aaaa::6:6804 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::56904 2620:cf:cf:aaaa::6:6800 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::33774 2620:cf:cf:aaaa::6:6800 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::57410 2620:cf:cf:aaaa::6:3300 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::33764 2620:cf:cf:aaaa::6:6800 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::34792 2620:cf:cf:aaaa::6:6802 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::37048 2620:cf:cf:aaaa::6:6808 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::56912 2620:cf:cf:aaaa::6:6800 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::40768 2620:cf:cf:aaaa::6:3300 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::60898 2620:cf:cf:aaaa::6:6812 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::45504 2620:cf:cf:aaaa::6:6806 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::40756 2620:cf:cf:aaaa::6:3300 ESTABLISHED 691585/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::59378 2620:cf:cf:aaaa::6:6800 ESTABLISHED 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15663086 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15663085 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15663088 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15663087 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15662055 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15662054 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15662056 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15662057 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15662066 691585/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     15662067 691585/ganesha.nfsd



ceph-2

[zuul@ceph-4b8r13sm-2 ~]$ sudo netstat -anp | grep ganesha
tcp6       0      0 :::9587                 :::*                    LISTEN      967256/ganesha.nfsd
tcp6       0      0 :::33849                :::*                    LISTEN      967256/ganesha.nfsd
tcp6       0      0 :::50051                :::*                    LISTEN      967256/ganesha.nfsd
tcp6       0      0 :::12049                :::*                    LISTEN      967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::51848 2620:cf:cf:aaaa::6:6800 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::45662 2620:cf:cf:aaaa::6:6800 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::45894 2620:cf:cf:aaaa::6:6800 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::56328 2620:cf:cf:aaaa::6:3300 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::45130 2620:cf:cf:aaaa::6:6808 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::55542 2620:cf:cf:aaaa::6:6802 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::37670 2620:cf:cf:aaaa::6:3300 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::45880 2620:cf:cf:aaaa::6:6800 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::33644 2620:cf:cf:aaaa::6:6806 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::37682 2620:cf:cf:aaaa::6:3300 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::36392 2620:cf:cf:aaaa::6:6800 ESTABLISHED 967256/ganesha.nfsd
tcp6       0      0 2620:cf:cf:aaaa:::35836 2620:cf:cf:aaaa::6:6812 ESTABLISHED 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19131074 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19131073 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19131072 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19131075 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19132465 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19132463 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19132464 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19132475 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19132474 967256/ganesha.nfsd
unix  3      [ ]         STREAM     CONNECTED     19132462 967256/ganesha.nfsd



On Ceph-0:

[ceph: root@ceph-4b8r13sm-0 /]# ceph orch ps | grep nfs
haproxy.nfs.cephfs.ceph-4b8r13sm-0.avbymj      ceph-4b8r13sm-0  *:2049,9049               running (5d)      9m ago   5d    13.7M        -  2.4.22-f8e3218    9a50affc328d  49f12de5e9c7
haproxy.nfs.cephfs.ceph-4b8r13sm-1.fyoctw      ceph-4b8r13sm-1  *:2049,9049               running (5d)      9m ago   5d    13.5M        -  2.4.22-f8e3218    9a50affc328d  3f524765fa81
haproxy.nfs.cephfs.ceph-4b8r13sm-2.dhrwvp      ceph-4b8r13sm-2  *:2049,9049               running (5d)      9m ago   5d    15.6M        -  2.4.22-f8e3218    9a50affc328d  1edaac2a2572
keepalived.nfs.cephfs.ceph-4b8r13sm-0.qhyesk   ceph-4b8r13sm-0                            running (5d)      9m ago   5d    1559k        -  2.2.8             65ab4b13c05e  f404ffddd7b3
keepalived.nfs.cephfs.ceph-4b8r13sm-1.ptcdap   ceph-4b8r13sm-1                            running (5d)      9m ago   5d    1551k        -  2.2.8             65ab4b13c05e  4d4d5cb12837
keepalived.nfs.cephfs.ceph-4b8r13sm-2.rqynkl   ceph-4b8r13sm-2                            running (5d)      9m ago   5d    1555k        -  2.2.8             65ab4b13c05e  b1e44a52e9a5
nfs.cephfs.0.0.ceph-4b8r13sm-1.ruinft          ceph-4b8r13sm-1  *:12049                   running (83m)     9m ago   5d    98.0M        -  6.5               62a14a43dc5b  fdfb12d9b712
nfs.cephfs.1.0.ceph-4b8r13sm-2.lkpgpj          ceph-4b8r13sm-2  *:12049                   running (10m)     9m ago   5d    73.8M        -  6.5               62a14a43dc5b  1dede608d005
nfs.cephfs.2.0.ceph-4b8r13sm-0.vdxryv          ceph-4b8r13sm-0  *:12049                   running (13m)     9m ago   5d    74.1M        -  6.5               62a14a43dc5b  8e9161072e81



[ceph: root@ceph-4b8r13sm-0 /]# ceph nfs cluster info cephfs
{
  "cephfs": {
    "backend": [
      {
        "hostname": "ceph-4b8r13sm-0",
        "ip": "2620:cf:cf:aaaa::6a",
        "port": 12049
      },
      {
        "hostname": "ceph-4b8r13sm-1",
        "ip": "2620:cf:cf:aaaa::6b",
        "port": 12049
      },
      {
        "hostname": "ceph-4b8r13sm-2",
        "ip": "2620:cf:cf:aaaa::6c",
        "port": 12049
      }
    ],
    "ingress_mode": "haproxy-protocol",
    "monitor_port": 9049,
    "port": 2049,
    "virtual_ip": "2620:cf:cf:aaaa::2"
  }
}

[zuul@ceph-4b8r13sm-2 ~]$ rpcinfo -T tcp6 -s 2620:cf:cf:aaaa::2
   program version(s) netid(s)                         service     owner
    100000  2,3,4     local,udp,tcp,udp6,tcp6          portmapper  superuser
    100003  4,3       tcp6,tcp                         nfs         superuser
    100005  3,1       tcp6,tcp                         mountd      superuser


[ceph: root@ceph-4b8r13sm-0 /]# ceph nfs export ls cephfs
[
  "/volumes/_nogroup/60b76a8e-2380-43f4-ac4a-96cf7ea06ffe/1ef19db7-d593-4a30-9cde-4a3c9c14f7a1"
]
[ceph: root@ceph-4b8r13sm-0 /]# ceph nfs export info cephfs /volumes/_nogroup/60b76a8e-2380-43f4-ac4a-96cf7ea06ffe/1ef19db7-d593-4a30-9cde-4a3c9c14f7a1
{
  "access_type": "RW",
  "clients": [],
  "cluster_id": "cephfs",
  "export_id": 1,
  "fsal": {
    "cmount_path": "/",
    "fs_name": "cephfs",
    "name": "CEPH",
    "user_id": "nfs.cephfs.cephfs.fc95c256"
  },
  "path": "/volumes/_nogroup/60b76a8e-2380-43f4-ac4a-96cf7ea06ffe/1ef19db7-d593-4a30-9cde-4a3c9c14f7a1",
  "protocols": [
    3,
    4
  ],
  "pseudo": "/volumes/_nogroup/60b76a8e-2380-43f4-ac4a-96cf7ea06ffe/1ef19db7-d593-4a30-9cde-4a3c9c14f7a1",
  "security_label": true,
  "squash": "none",
  "transports": [
    "TCP"
  ]
}