Bug 2238623

Summary: Failing to AddExport or UpdateExport over dbus calls
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Carlos da Silva <cardasil>
Component: NFS-GaneshaAssignee: Frank Filz <ffilz>
Status: CLOSED ERRATA QA Contact: Manisha Saini <msaini>
Severity: high Docs Contact:
Priority: unspecified    
Version: 5.1CC: cephqe-warriors, ffilz, gouthamr, kdreyer, kkeithle, lkuchlan, msaini, ppitonak, racpatel, saraut, sostapov, tserlin, vdas, vereddy
Target Milestone: ---Keywords: Rebase
Target Release: 6.1z2   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: nfs-ganesha-5.5-0.1.el9cp, rhceph-container-6-238 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-10-12 16:34:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Carlos da Silva 2023-09-12 20:57:01 UTC
Description of problem:

We are running into an issue in OpenStack Manila while attempting to send dbus calls to AddExport or UpdateExport. Sometimes, we get an exception.

Command: dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/etc/ganesha/export.d/share-ee5daf66-0eab-494e-8cf6-01aafa00fbe6.conf.nFf2EQ string:EXPORT(Export_Id=1146)

Stderr: 'Error org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatable\n'.

The NFS Ganesha logs show that the service apparently had an error and restarted right after it [1].

Version-Release number of selected component (if applicable):
We're using NFS Ganesha 5.1


How reproducible:
This issue is currently being reproduced in our internal testing, while running various OpenStack Manila scenario tests in parallel. When we run the scenario tests limiting it to one thread, it works just fine.

Steps to Reproduce:


Actual results:
Manila receives an error when it tries to add/update exports.

Expected results:
Manila should be able to add/update exports.


Additional info:
[1] https://paste.opendev.org/show/bktNx5M1Anfq6r1QSWem/

Comment 10 Scott Ostapovicz 2023-09-14 19:02:18 UTC
As long as QE can accommodate this, I would say lets get it done as soon as possible.

Comment 17 Manisha Saini 2023-09-18 17:19:19 UTC
Hi Frank,

Could you provide the steps to verify this fix on containerised NFS deployment over CEPH?

As per update from Goutham and  Carlous, They have  a service (OpenStack Manila) making the calls to dbus on their behalf, they are not doing the calls manually.

I tried to run the dbus command from inside the NFS pod as well as on the Node itself hosting the NFS pod.But was unable to export via dbus manually


[root@ceph-mani-nr48e0-node2 /]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:'/cephfs2' string:/etc/ganesha/ganesha.conf string:'EXPORT(Export_Id=4)'
Failed to open connection to "system" message bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory[root@ceph-mani-nr48e0-node2 /]#

I also tried running the same on the Node itself where ganesha pod is running

[root@ceph-mani-nr48e0-node2 ~]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:'/cephfs2' string:/etc/ganesha/ganesha.conf string:'EXPORT(Export_Id=4)'
Error org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatable
[root@ceph-mani-nr48e0-node2 ~]#

In both cases, the command is unable to export via dbus. Could you please let me know how to verify this BZ in containerised environment running NFS?


Or is it possible to share the  RHCS container image build on RHOSBS for their team to verify this issue in their environment?

Comment 18 Frank Filz 2023-09-18 21:49:24 UTC
I'm not at all familiar with the Ganesha for Ceph deployment, maybe someone else can help?

I am pretty sure containerized Ganesha can not be updated with dBus, as that requires a privileged container.

I'm not sure who to hand the ball off to for verification...

Comment 30 errata-xmlrpc 2023-10-12 16:34:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat Ceph Storage 6.1 security, enhancement, and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:5693