Bug 2351558

Summary: [NFS-Ganesha] NFS-Ganesha cluster deployment is failing with 8.1 builds
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Manisha Saini <msaini>
Component: CephadmAssignee: Adam King <adking>
Status: CLOSED ERRATA QA Contact: Manisha Saini <msaini>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 8.1CC: akane, cephqe-warriors, kkeithle, tserlin, vdas
Target Milestone: ---Keywords: Automation, Regression, Reopened, TestBlocker
Target Release: 8.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: nfs-ganesha-6.5-9.el9cp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-06-26 12:28:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2343149    
Bug Blocks:    

Description Manisha Saini 2025-03-12 10:55:07 UTC
Description of problem:
===========
NFS-Ganesha cluster is failing with latest builds of RHCS 8.1

[ceph: root@ceph-manisaini-uemu57-node1-installer /]# ceph nfs cluster create cephfs-nfs 'ceph-manisaini-uemu57-node2'

[ceph: root@ceph-manisaini-uemu57-node1-installer /]# ceph nfs cluster info cephfs-nfs
{
  "cephfs-nfs": {
    "backend": [
      {
        "hostname": "ceph-manisaini-uemu57-node2",
        "ip": "10.0.66.40",
        "port": 2049
      }
    ],
    "virtual_ip": null
  }
}

[ceph: root@ceph-manisaini-uemu57-node1-installer /]# ceph orch ps | grep nfs
nfs.cephfs-nfs.0.0.ceph-manisaini-uemu57-node2.bwnwzx  ceph-manisaini-uemu57-node2            *:2049            unknown          22s ago  25s        -        -  <unknown>        <unknown>     <unknown>

Version-Release number of selected component (if applicable):
=============
[ceph: root@ceph-manisaini-uemu57-node1-installer /]# ceph --version
ceph version 19.2.1-31.el9cp (83b1ebad0c3f84a8bd7a12b6b20f84f1fc691f08) squid (stable)

[ceph: root@ceph-manisaini-uemu57-node1-installer /]# rpm -qa | grep nfs
libnfsidmap-2.5.4-27.el9.x86_64
nfs-utils-2.5.4-27.el9.x86_64
nfs-ganesha-selinux-6.5-5.el9cp.noarch
nfs-ganesha-6.5-5.el9cp.x86_64
nfs-ganesha-rgw-6.5-5.el9cp.x86_64
nfs-ganesha-ceph-6.5-5.el9cp.x86_64
nfs-ganesha-rados-grace-6.5-5.el9cp.x86_64
nfs-ganesha-rados-urls-6.5-5.el9cp.x86_64
nfs-ganesha-utils-6.5-5.el9cp.x86_64


How reproducible:
==========
2/2


Steps to Reproduce:
==========
1.Create NFS ganesha cluster on ceph


Actual results:
==========
NFS-Ganesha cluster fails to comeup. NFS Daemon is in failed state


Expected results:
=========
NFS Cluster deplyment should be successful


Additional info:
=============

esha.nfsd-2[main] init_logging :LOG :NULL :LOG: Setting log level for all components to NIV_EVENT
esha.nfsd-2[main] SetComponentLogLevel :LOG :NULL :LOG: Changing log level of (null) from (null) to NIV_EVENT
esha.nfsd-2[main] main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version 6.5
esha.nfsd-2[main] nfs_set_param_from_conf :NFS STARTUP :WARN :domainname in NFSv4 config section will soon be deprecated, define it under DIRECTORY_SERVICES section
esha.nfsd-2[main] nfs_set_param_from_conf :NFS STARTUP :WARN :Use idmapped_user_time_validity under DIRECTORY_SERVICES section to configure time validity of idmapped users
esha.nfsd-2[main] nfs_set_param_from_conf :NFS STARTUP :WARN :Use idmapped_group_time_validity under DIRECTORY_SERVICES section to configure time validity of idmapped groups
esha.nfsd-2[main] rados_load_config_from_parse :NFS STARTUP :CRIT :Error while parsing RadosKV specific configuration
esha.nfsd-2[main] main :NFS STARTUP :CRIT :Error setting parameters from configuration file.
esha.nfsd-2[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:21): Expected a number, got a double quoted string
esha.nfsd-2[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:19): 1 errors while processing parameters for RADOS_KV
esha.nfsd-2[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:19): Errors processing block (RADOS_KV)
esha.nfsd-2[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:31): 1 (invalid param value) errors found block RADOS_KV
esha.nfsd-2[main] main :NFS STARTUP :FATAL :Fatal errors.  Server exiting...
esha.nfsd-2[main] gsh_backtrace :NFS STARTUP :MAJ :/lib64/libganesha_nfsd.so.6.5(+0x91e91) [0x7f87e9181e91]
esha.nfsd-2[main] gsh_backtrace :NFS STARTUP :MAJ :/lib64/libganesha_nfsd.so.6.5(+0x9230d) [0x7f87e918230d]
esha.nfsd-2[main] gsh_backtrace :NFS STARTUP :MAJ :/lib64/libganesha_nfsd.so.6.5(DisplayLogComponentLevel+0x8b) [0x7f87e91826fb]
esha.nfsd-2[main] gsh_backtrace :NFS STARTUP :MAJ :/usr/bin/ganesha.nfsd(main+0x71a) [0x5566c4fe9eaa]
esha.nfsd-2[main] gsh_backtrace :NFS STARTUP :MAJ :/lib64/libc.so.6(+0x295d0) [0x7f87e8ef25d0]
esha.nfsd-2[main] gsh_backtrace :NFS STARTUP :MAJ :/lib64/libc.so.6(__libc_start_main+0x80) [0x7f87e8ef2680]
esha.nfsd-2[main] gsh_backtrace :NFS STARTUP :MAJ :/usr/bin/ganesha.nfsd(_start+0x25) [0x5566c4fea635]

Comment 11 errata-xmlrpc 2025-06-26 12:28:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat Ceph Storage 8.1 security, bug fix, and enhancement updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2025:9775