Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 2418562

Summary: [NFS-Ganesha][Upgrade Test] In the 8.0x–8.1x–9.0 upgrade path, one NFS cluster encountered I/O failures (“stale file handle”) and could not be remounted.
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Manisha Saini <msaini>
Component: CephadmAssignee: Shweta Bhosale <shbhosal>
Status: CLOSED WORKSFORME QA Contact: Manisha Saini <msaini>
Severity: high Docs Contact:
Priority: unspecified    
Version: 9.0CC: akane, cephqe-warriors, ffilz, kkeithle, nchillar, ngangadh, nojha, prprakas, rpollack, shbhosal, spunadik, vdas
Target Milestone: ---   
Target Release: 9.0z1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
.Ceph NFS exports missing after upgrade In some cases, Ceph NFS services fail to export data after upgrading. This occurs because during the upgrade the RADOS object that stores export information can, occasionally, become empty. As a result, the following can occur: * No exports are available from the NFS service, resulting in loss of access (data remains intact). * NFS clients receive `Stale File Handle` errors when accessing exports or mounting new ones. * Ganesha logs display: `main :NFS STARTUP :WARN :No export entries found in configuration file !!!` As a workaround, recreate the missing exports to repopulate the RADOS object. 1. Delete the NFS export. `ceph nfs export delete CLUSTER_NAME EXPORT_PATH` 2. Recreate the export using the same subvolume path. `ceph nfs export create cephfs CLUSTER_NAME BIND_PATH cephfs --path=SUBVOLUME_PATH` 3. Verify the RADOS object storing export information. `rados -p '.nfs' -N CLUSTER_NAME get conf-nfs.CLUSTER_NAME /conf-nfs.CLUSTER_NAME` 4. Restart the NFS service: `ceph orch restart nfs.CLUSTER_NAME` After restarting the NFS service, clients can mount and access exports normally.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2026-01-21 12:06:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2388233    

Description Manisha Saini 2025-12-03 05:17:56 UTC 
Description of problem:
======================
While testing the upgrade path - Upgrade 8.0x --> 8.1x--> 9.0,
On 1 of the NFS cluster, the mount and IO's failed with "stale file handle". After the failure, attempts to remount the export on a new client also did not succeed.



Version-Release number of selected component (if applicable):
====================================
Upgrade was performed for path -
ceph version 19.2.0-139.el9cp --> ceph version 19.2.1-292.el9cp --> ceph version 20.1.0-115.el9cp

How reproducible:
=============
1/1


Steps to Reproduce:
===================
1. Deployed the ceph cluster on 8.0 builds
2. Create 2 NFS clusters on 2 diff nodes
3. Create 1-1 export from each NFS cluster and run IO's
4. Upgrade the ceph to 8.1 builds while IO's are running
5. Create 1 more NFS cluster on another node. Create NFS export from that cluster and start IO's
6. Upgrade the cluster to 9.0


Actual results:
===============
On the 1 out of 2 NFS cluster which was created at step 2, IO's failed with "stale file handle" while upgrade was in progress and the export could not be remounted.


Expected results:
===============
IO's should work seamlessly when upgrade is ongoing and the cluster is upgraded


Additional info:
================

Client 2:
—--

tar: linux-6.4/arch/arm64: Cannot stat: Stale file handle
tar: linux-6.4/arch: Cannot stat: Stale file handle
tar: linux-6.4: Cannot stat: Stale file handle
tar: Error is not recoverable: exiting now
[root@client1 nfsganesha2]# ls
ls: cannot open directory '.': Stale file handle
[root@client1 nfsganesha2]# ls
ls: cannot open directory '.': Stale file handle
[root@client1 nfsganesha2]# ls
ls: cannot open directory '.': Stale file handle
[root@client1 nfsganesha2]# ls
ls: cannot open directory '.': Stale file handle

[root@client1 ~]# df
df: /mnt/nfsganesha2: Stale file handle
Filesystem                        1K-blocks     Used Available Use% Mounted on
devtmpfs                               4096        0      4096   0% /dev
tmpfs                              24534540       84  24534456   1% /dev/shm
tmpfs                               9813820    98120   9715700   1% /run
/dev/mapper/rhel_dhcp46--151-root  36678148 23970472  12707676  66% /
/dev/sda1                           1038336   410748    627588  40% /boot
tmpfs                               4906908        0   4906908   0% /run/user/0

[root@client1 ~]# mount | grep ganesha
10.70.46.79:/ganesha1 on /mnt/nfsganesha2 type nfs4 (rw,relatime,seclabel,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.70.45.216,local_lock=none,addr=10.70.46.79)

[root@client1 ~]# cd /mnt/nfsganesha2
-bash: cd: /mnt/nfsganesha2: Stale file handle

[root@client1 mnt]# mount -t nfs 10.70.46.79:/ganesha1 /mnt/test/
mount.nfs: mounting 10.70.46.79:/ganesha1 failed, reason given by server: No such file or directory
Comment 16 Red Hat Bugzilla 2026-02-06 04:27:01 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days or the product is inactive and locked