Bug 1161413

The feature is under review for Kilo milestone 2 
https://review.openstack.org/#/c/133074/
Sean

*** Bug 1155504 has been marked as a duplicate of this bug. ***

Full Spec available at: http://specs.openstack.org/openstack/cinder-specs/specs/liberty/nfs-snapshots.html

Eric, is leading this effort upstream. The spec file is already merged, but it probably will be implemented only in the M cycle.

This is currently blocking on a required fix in Nova:
  https://bugs.launchpad.net/nova/+bug/1416132

Upstream patches merged, moving from ON_DEV to POST.

Adding a needinfo - need to document some config change and limitations for anyone to actually be able to use it there too

Deployments must set "nfs_snapshot_support = True" in the volume driver backend section of cinder.conf for this feature to be enabled.

Eric, need you insight on this. 
There is a temp issue with some undercloud bug can't install with Infra, started testing this with Packstack. 

Simple AIO server 
openstack-cinder-10.0.1-0.20170222155052.ea70d55.el7ost.noarch
puppet-cinder-10.3.0-0.20170220170133.0192e52.el7ost.noarch
python-cinderclient-1.11.0-0.20170208174522.7d140d0.el7ost.noarch
python-cinder-10.0.1-0.20170222155052.ea70d55.el7ost.noarch


On packstack answer file section:
CONFIG_CINDER_BACKEND=nfs
CONFIG_CINDER_NFS_MOUNTS=XXXXXXXX:/export/cinder

On cinder.conf also enabled
# grep nfs_snapshot /etc/cinder/cinder.conf 
nfs_snapshot_support = True

Cinder create works fine, vol created on nfs backend
# cinder list
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| ID                                   | Status    | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| e84c299d-fafd-4c4f-9a23-5ec3f1337057 | available | -    | 1    | nfs         | false  


Cinder snapshot create fails

[root@panther13 ~(keystone_admin)]# cinder snapshot-list
+--------------------------------------+--------------------------------------+--------+-----------+------+
| ID                                   | Volume ID                            | Status | Name      | Size |
+--------------------------------------+--------------------------------------+--------+-----------+------+
| e1c82a7c-3729-4098-998f-aeb620b27bce | e84c299d-fafd-4c4f-9a23-5ec3f1337057 | error  | snapshot1 | 1    |
| f3da162b-5c51-425b-afa8-874a14928ec6 | e84c299d-fafd-4c4f-9a23-5ec3f1337057 | error  | snap2     | 1    |


I've since added another volume/snapshots none work. 
Shouldn't I see some errors on volume log (Debug is set to true) ?

Created attachment 1260780 [details]
Packstack answer file ocata.conf, cinder.conf, cinder logs

Forgot to add before, logs files of my system

You need to set nfs_snapshot_support=True in the [nfs] backend section of cinder.conf instead of in [DEFAULT].

You do have a good point, sorry my bad. 
So i fixed it restarted services. 

[nfs]
volume_backend_name=nfs
volume_driver=cinder.volume.drivers.nfs.NfsDriver
nfs_shares_config=/etc/cinder/nfs_shares.conf
nfs_snapshot_support=True     ( #out the one I touched om  default section)


Now hitting something else, can't even create a volume



lib/python2.7/site-packages/cinder/volume/drivers/nfs.py:431
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.drivers.nfs [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] Snapshots are not supported with nas_secure_file_operations enabled ('true' or 'auto'). Please set it to 'false' if you intend to  have it enabled.
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] Failed to initialize driver.
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager Traceback (most recent call last):
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 431, in init_host
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager     self.driver.do_setup(ctxt)
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/nfs.py", line 199, in do_setup
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager     self._check_snapshot_support(setup_checking=True)
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/nfs.py", line 551, in _check_snapshot_support
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager     raise exception.VolumeDriverException(message=msg)
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager VolumeDriverException: Volume driver reported an error: Snapshots are not supported with nas_secure_file_operations enabled ('true' or 'auto'). Please set it to 'false' if you intend to  have it enabled.
2017-03-07 18:25:16.772 17511 ERROR cinder.volume.manager 
2017-03-07 18:25:16.773 17511 DEBUG cinder.service [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] Creating RPC server for service cinder-volume start /usr/lib/python2.7/site-packages/cinder/service.py:243
2017-03-07 18:25:16.776 17511 DEBUG oslo_db.sqlalchemy.engines [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:261
2017-03-07 18:25:16.780 17511 DEBUG cinder.service [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] Pinning object versions for RPC server serializer to 1.21 start /usr/lib/python2.7/site-packages/cinder/service.py:250
2017-03-07 18:25:16.780 17511 WARNING py.warnings [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] /usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py:200: FutureWarning: The access_policy argument is changing its default value to <class 'oslo_messaging.rpc.dispatcher.DefaultRPCAccessPolicy'> in version '?', please update the code to explicitly set None as the value: access_policy defaults to LegacyRPCAccessPolicy which exposes private methods. Explicitly set access_policy to DefaultRPCAccessPolicy or ExplicitRPCAccessPolicy.
  access_policy)

2017-03-07 18:25:16.810 17511 INFO cinder.volume.manager [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] Initializing RPC dependent components of volume driver NfsDriver (1.4.0)
2017-03-07 18:25:16.811 17511 ERROR cinder.utils [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] Volume driver NfsDriver not initialized
2017-03-07 18:25:16.811 17511 ERROR cinder.volume.manager [req-44f4716d-9400-413f-997a-b26c737b1da6 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.
2017-03-07 18:25:26.812 17511 ERROR cinder.service [-] Manager for service cinder-volume panther13.qa.lab.tlv.redhat.com@nfs is reporting problems, not sending heartbeat. Service will appear "down".


I'm guessing this isn't good
 Snapshots are not supported with nas_secure_file_operations enabled ('true' or 'auto'). Please set it to 'false' if you intend to  have it enabled.


While looking at cinder.conf 
nas_secure_file_operations 
Also noticed this one #nas_secure_file_permissions = auto

Guess needs a bit more fiddling around  
Thanks Eric.

(In reply to Tzach Shefi from comment #17)

> 
> While looking at cinder.conf 
> nas_secure_file_operations 
> Also noticed this one #nas_secure_file_permissions = auto
> 
> Guess needs a bit more fiddling around  
> Thanks Eric.

Correct, you'll have to set "nas_secure_file_permissions = False" for this feature to work.

Verified on version:
openstack-cinder-10.0.1-0.20170222155052.ea70d55.el7ost.noarch

This what I had set on cinder.conf:
enabled_backend=nfs

[nfs]
volume_backend_name=nfs
volume_driver=cinder.volume.drivers.nfs.NfsDriver
nfs_shares_config=/etc/cinder/nfs_shares.conf
nfs_snapshot_support=True
nas_secure_file_operations=False  (without which driver fails, default -> true)
nas_secure_file_permissions=False 

Run this twice on a packstack as well as OPSD deployment. 
There is an open OSPD NFS bug bz1416356, had to configure NFS manually post deploy. 

Tempest filtered out 60 snapshot related tests, 47 of them passed. 
Failing tests are same ones for v1 v2, they failed cause I had no connectivity to instance which was expected. 

Tempest summery: 
Ran 60 (+47) tests in 639.747s (+598.497s)
FAILED (id=2, failures=7 (+7), skips=3)

FAIL: tempest.api.volume.test_volumes_snapshots.VolumesV1SnapshotTestJSON.test_snapshot_create_offline_delete_online[compute,id-5210a1de-85a0-11e6-bb21-641c676a5d61]
FAIL: tempest.api.volume.test_volumes_snapshots.VolumesV2SnapshotTestJSON.test_snapshot_create_offline_delete_online[compute,id-5210a1de-85a0-11e6-bb21-641c676a5d61]

FAIL: tempest.api.volume.test_volumes_snapshots.VolumesV1SnapshotTestJSON.test_snapshot_create_with_volume_in_use[compute,id-b467b54c-07a4-446d-a1cf-651dedcc3ff1]
FAIL: tempest.api.volume.test_volumes_snapshots.VolumesV2SnapshotTestJSON.test_snapshot_create_with_volume_in_use[compute,id-b467b54c-07a4-446d-a1cf-651dedcc3ff1]

FAIL: tempest.api.volume.test_volumes_snapshots.VolumesV1SnapshotTestJSON.test_snapshot_delete_with_volume_in_use[compute,id-8567b54c-4455-446d-a1cf-651ddeaa3ff2]
FAIL: tempest.api.volume.test_volumes_snapshots.VolumesV2SnapshotTestJSON.test_snapshot_delete_with_volume_in_use[compute,id-8567b54c-4455-446d-a1cf-651ddeaa3ff2]

FAIL: tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern[compute,id-608e604b-1d63-4a82-8e3e-91bc665c90b4,image,network]

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245