Description of problem: The cmake commands for building downstream nfs-ganesha need to have rados based featured enabled. The following upstream PR's enable them: https://github.com/nfs-ganesha/nfs-ganesha-debian/pull/12 https://github.com/ceph/ceph-build/pull/888 All that needs to be changed is the downstream nfs-ganesha .spec file. Version-Release number of selected component (if applicable): How reproducible: Easily Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Testing Ganesha fetching exports from RADOS objects *************************************************** Prerequisites ============= * A Ceph cluster with a Ceph filesystem * 'nfs-ganesha' and 'nfs-ganesha-ceph' packages installed in client machine * Network connectivity between Ceph cluster and NFS-Ganesha machine Assuming ganesha host IP is 10.70.43.205, and Ceph MOn IP is 10.70.42.111 Steps ===== In the ganesha host do the following, 1. Set /etc/ceph/ceph.conf to be able to connect to Ceph cluster $ sudo cat /etc/ceph/ceph.conf [global] mon_host = 10.70.42.111:6789 2. Create an export block and store it as a RADOS object in CephFS's data pool (here named 'cephfs_data') $ cat export.conf EXPORT { EXPORT_ID = 100; Path = /; Pseudo = /; Protocols = 4; Transports = TCP; Squash = No_root_squash; FSAL { Name = Ceph; } CLIENT { Clients = 10.70.43.205; Access_Type = rw; } } $ sudo rados -p cephfs_data put ganesha export.conf 2. Setup ganesha.conf to allow Ganesha to read from RADOS object $ sudo cat /etc/ganesha/ganesha.conf %url rados://cephfs_data/ganesha 3. Start the NFS-Ganesha server. Ganesha server should now export the Ceph File System to 10.70.43.205, IP of Ganesha host $ sudo systemctl start nfs-ganesha 4. NFS mount the Ceph FileSystem and try accessing it $ sudo mount.nfs4 10.70.43.205:/ /mnt/nfs4/ $ cd /mnt/nfs4/
this is missing a QE ack.
(In reply to Ram Raja from comment #9) Adding a step > Testing Ganesha fetching exports from RADOS objects > *************************************************** > > Prerequisites > ============= > * A Ceph cluster with a Ceph filesystem > * 'nfs-ganesha' and 'nfs-ganesha-ceph' packages installed in client machine > * Network connectivity between Ceph cluster and NFS-Ganesha machine > > Assuming ganesha host IP is 10.70.43.205, and Ceph MOn IP is 10.70.42.111 > Steps > ===== > In the ganesha host do the following, > 1. Set /etc/ceph/ceph.conf to be able to connect to Ceph cluster > $ sudo cat /etc/ceph/ceph.conf > [global] > mon_host = 10.70.42.111:6789 Make sure that the Ceph 'admin' auth ID's keyring file is at the default location, e.g., /etc/ceph/ceph.client.admin.keyring Ganesha's librados client uses the 'admin' ID to fetch the export RADOS object. > > > 2. Create an export block and store it as a RADOS object in > CephFS's data pool (here named 'cephfs_data') > $ cat export.conf > EXPORT > { > EXPORT_ID = 100; > Path = /; > Pseudo = /; > Protocols = 4; > Transports = TCP; > Squash = No_root_squash; > FSAL { > Name = Ceph; > } > CLIENT { > Clients = 10.70.43.205; > Access_Type = rw; > } > } > $ sudo rados -p cephfs_data put ganesha export.conf > > 2. Setup ganesha.conf to allow Ganesha to read from RADOS object > $ sudo cat /etc/ganesha/ganesha.conf > %url rados://cephfs_data/ganesha > > 3. Start the NFS-Ganesha server. Ganesha server should now export > the Ceph File System to 10.70.43.205, IP of Ganesha host > $ sudo systemctl start nfs-ganesha > > 4. NFS mount the Ceph FileSystem and try accessing it > $ sudo mount.nfs4 10.70.43.205:/ /mnt/nfs4/ > $ cd /mnt/nfs4/
Testing Ganesha storing client recovery data in RADOS OMAP key-value ******************************************************************** Prerequisites ============= * A Ceph cluster with a Ceph filesystem * 'nfs-ganesha' and 'nfs-ganesha-ceph' packages installed in client machine * Network connectivity between Ceph cluster and NFS-Ganesha machines Assuming ganesha host IP is 10.70.43.205, and Ceph mon IP is 10.70.42.111 Steps ===== In the ganesha host do the following, 1. Set /etc/ceph/ceph.conf to be able to connect to Ceph cluster $ sudo cat /etc/ceph/ceph.conf [global] mon_host = 10.70.42.111:6789 2. Ensure that Ceph 'admin' auth ID is in the default location, e.g., /etc/ceph/ceph.client.admin.keyring 3. Create an export block and store it as a RADOS object in CephFS's data pool (here 'cephfs_data') $ cat export.conf EXPORT { EXPORT_ID = 100; Path = /; Pseudo = /; Protocols = 4; Transports = TCP; Squash = No_root_squash; FSAL { Name = Ceph; } CLIENT { Clients = 10.70.43.205; Access_Type = rw; } } $ sudo rados -p cephfs_data put ganesha export.conf 4. Setup ganesha.conf to allow Ganesha to read from RADOS object, and store client IDs as RADOS OMAP key-values in CephFS's data pool, 'cephfs_data' $ cat /etc/ganesha/ganesha.conf %url rados://cephfs_data/ganesha NFSv4 { RecoveryBackend = 'rados_kv'; } RADOS_KV { ceph_conf = '/etc/ceph/ceph.conf'; userid = 'admin'; pool = 'cephfs_data'; } 5. Start the NFS-Ganesha server. Ganesha server should now export the Ceph File System to 10.70.43.205, IP of Ganesha host $ sudo systemctl start nfs-ganesha 6. NFS mount the Ceph FileSystem and try accessing it $ sudo mount.nfs4 10.70.43.205:/ /mnt/nfs4/ $ cd /mnt/nfs4/ 7. Check for OMAP key values of objects with prefixes '_old' and '_recov' in CephFS's data pool. NFS client IP, hostname is stored among them. In the following case only in key-value of 'node0_recov' object. $ sudo rados -p cephfs_data listomapvals node0_recov - 6480837291517411329 value (72 bytes) : 00000000 3a 3a 66 66 66 66 3a 31 30 2e 37 30 2e 34 33 2e |::ffff:10.70.43.| 00000010 32 30 35 2d 28 34 37 3a 4c 69 6e 75 78 20 4e 46 |205-(47:Linux NF| 00000020 53 76 34 2e 31 20 64 68 63 70 34 33 2d 32 30 35 |Sv4.1 dhcp43-205| 00000030 2e 6c 61 62 2e 65 6e 67 2e 62 6c 72 2e 72 65 64 |.lab.eng.blr.red| 00000040 68 61 74 2e 63 6f 6d 29 |hat.com)| 00000048 $ sudo rados -p cephfs_data listomapvals node0_old -
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3387