Description of problem: Snapshot restores fails, when nfs ganesha was enabled while taking snapshot, and is disabled during snapshot restore and shared storage is down. Version-Release number of selected component (if applicable): glusterfs-3.8.4-7.el7rhgs.x86_64 How reproducible: 100% Steps to Reproduce: 1. Create 2*2 distribute replicate volume 2. Enable cluster.enable-shared-storage 3. enable nfs ganesha 4. create snapshot 5. disable nfs ganesha 6. bring gluster-shared-storage volume 7. Restore snapshot Actual results: Snapshot restore will fail Expected results: Snapshot restore should not fail Additional info: ============================================================= [2016-12-09 09:40:36.760986] E [MSGID: 106029] [glusterd-snapshot-utils.c:3412:glusterd_copy_file] 0-management: Unble to open a file /var/run/gluster/shared_storage/nfs-ganesha/exports/export.test-volume.conf [2016-12-09 09:40:36.761015] E [MSGID: 106021] [glusterd-snapshot-utils.c:3957:glusterd_restore_nfs_ganesha_file] 0-management: Failed to copy /var/lib/glusterd/snaps/snap1/export.snap1.conf in /var/run/gluster/shared_storage/nfs-ganesha/exports/export.test-volume.conf [Cannot allocate memory] [2016-12-09 09:40:36.761032] W [MSGID: 106043] [glusterd-snapshot.c:9946:gd_restore_snap_volume] 0-management: Failed to restore nfs-ganesha export file for snap snap1 [2016-12-09 09:40:36.761051] E [MSGID: 106043] [glusterd-snapshot.c:885:glusterd_snapshot_restore] 0-management: Failed to restore snap for snap1 [2016-12-09 09:40:36.761069] W [MSGID: 106043] [glusterd-snapshot.c:8433:glusterd_snapshot] 0-management: Failed to restore snapshot
RCA: Essentially a snapshot of a volume, which was taken when the volume had nfs.ganesha option enabled, has a export.conf file for it. This file will be restored to the Ganesha Export Directory, which has been moved to the shared storage. Currently we fail the snapshot restore in a scenario, where the snapshot has the said conf file, but the shared storage is not available. This behaviour is expected. However there is no option for the user to proceed with the snapshot at this point in time. We will introduce a force option for snapshot restore, which will enable the user to restore this particular snapshot in the above explained scenario, thereby abandoning the export.conf. The reason for introducing the force option is to make the user explicitly ask for the saved export.conf to be abandoned in such a scenario. Since this fix involves introducing CLI change, we propose this to be deferred from 3.2.0
This issue is not considered for releases in last 2 years, RCA available. We are not focusing on fixing the issue in any immediate timeline. Do raise priority on this, if required in any immediate releases!