Bug 1388056

Summary:	Nfs issues openstack newton
Product:	[Community] RDO	Reporter:	Andrei Ghiciac <andrei.ghiciac>
Component:	openstack-cinder	Assignee:	Tzach Shefi <tshefi>
Status:	CLOSED UPSTREAM	QA Contact:	nlevinki <nlevinki>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	andrei.ghiciac, eharney, srevivo
Target Milestone:	---
Target Release:	trunk
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-10-24 12:30:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Andrei Ghiciac 2016-10-24 10:36:54 UTC

Description of problem:

After the release of "Newton", we have reinstalled the Centos 7 servers with it and it seems there is a bug on creating snapshots which was not present on the previous version.

We are using a NFS and everything works except the snapshot creation of an instance.

The snapshot is created in the dashboard UI but it has 0 bytes.



Cinder versions :

openstack-cinder-9.0.0-1.el7.noarch
python-cinder-9.0.0-1.el7.noarch
puppet-cinder-9.4.1-1.el7.noarch
python2-cinderclient-1.9.0-1.el7.noarch



Errors in the logs :

As you can see, everything works until a snapshot creation is issues :

2016-10-24 12:58:45.614 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 12:59:45.610 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:00:45.610 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:01:45.605 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:02:45.606 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:03:45.604 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:04:45.607 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:05:45.603 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:06:45.605 864 INFO os_brick.remotefs.remotefs [req-89cc603d-924e-4792-b4d0-7bf2e414d64f - - - - -] Already mounted: /var/lib/cinder/mnt/cbeb33711a066b92c69af055294191cb
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server [req-c3b6d825-8f16-4d89-8c42-61e93c8163d1 bb753d3055be44e7a94a8cf3ace3dde4 e0546923a5544605bed01036f20dff37 - default default] Exception during message handling
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 4376, in create_snapshot
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     snapshot)
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 850, in create_snapshot
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     snapshot.save()
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     self.force_reraise()
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 842, in create_snapshot
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server     model_update = self.driver.create_snapshot(snapshot)
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server AttributeError: 'NfsDriver' object has no attribute 'create_snapshot'
2016-10-24 13:07:20.047 864 ERROR oslo_messaging.rpc.server 



How reproducible:

Openstack newton with NFS as a storage.

Steps to Reproduce:
1. Create an instance
2. Issue a create snapshot
3. Snapshot created with 0 bytes

Actual results:

No snapshot and log errors ( posted above )

Expected results:

Obtain a usable snapshot

Additional info:

Please let me know if more info is needed.

Thank you guys for all the great work you have done with the project.

Comment 1 Eric Harney 2016-10-24 12:30:55 UTC

Cinder's snapshot functionality is not yet supported for the NFS driver.  Previous versions of OpenStack would experience the same failure when using the NFS driver.

Work on this feature is being tracked upstream at:

https://blueprints.launchpad.net/cinder/+spec/nfs-snapshots