Red Hat Bugzilla – Bug 1023958
netapp NetAppNFSDriver fails at creating new snapshots
Last modified: 2016-04-26 19:16:53 EDT
Description of problem:
NetAppNFSDriver does not create new snapshots and returns error, the volume.log traceback is:
2013-10-28 14:45:57 ERROR [cinder.openstack.common.rpc.amqp] Exception during message handling
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/amqp.py", line 430, in _process_data
rval = self.proxy.dispatch(ctxt, version, method, **args)
File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/dispatcher.py", line 133, in dispatch
return getattr(proxyobj, method)(ctxt, **kwargs)
File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 482, in create_snapshot
File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 473, in create_snapshot
model_update = self.driver.create_snapshot(snapshot_ref)
File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/netapp/nfs.py", line 87, in create_snapshot
File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/netapp/nfs.py", line 132, in _clone_volume
host_id = self._get_host_id(volume_id)
File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/netapp/nfs.py", line 203, in _get_host_id
resp = server.HostListInfoIterStart(ObjectNameOrId=host_ip)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 542, in __call__
return client.invoke(args, kwargs)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 602, in invoke
result = self.send(soapenv)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 657, in send
result = self.failed(binding, e)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 718, in failed
raise Exception((status, reason))
Exception: (404, u'Not Found')
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. configure Cinder with NetAppNFSDriver and the NetApp software accordingly
2. create a new volume
3. create a volume snapshot
snapshot creation fails and the snapshot is found in "error" state
It looks like this may happen if the volume's provider_location doesn't match the IP address of the NetApp machine. Have you reconfigured networking parameters etc., lately for the NetApp machine?
# mysql cinder -e "select id,provider_location from volumes"
show, versus "netapp_server_hostname" in cinder.conf (or the known address).
that shouldn't be the case as the netapp vfiler didn't change nor I changed the hostname/dns/ip of it but I noticed the table has some NULL values:
| 07374fd2-af24-4b17-930a-1cd0cc33477c | vfiler02:/vol/vfilers/vfiler02 |
| 07bcd6e8-d681-4456-b8fb-08eab138b4f6 | vfiler02:/vol/vfilers/vfiler02 |
| 07d557b5-fd0b-4875-ab59-321ba131c65f | NULL |
| 09d2fe05-c727-42be-8255-7e3b9bf5374a | NULL |
| 09e3eaf0-e68e-4c83-9698-625d7e4554e5 | vfiler02:/vol/vfilers/vfiler02 |
| 0a0fcbe4-6e74-4a94-ab43-c94a5e760c41 | NULL |
| 0c26216b-e6ea-414f-8d6f-d925778dcc5c | vfiler02:/vol/vfilers/vfiler02 |
does that help?
I assume that one of those showing NULL is the volume that snapshot creation is failing on?
_get_host_id() calls _get_host_ip() which does:
So, host_id will be NULL in that case which is probably why the server returns a 404 for locating that object.
actually no, snapshotted volumes (volumes for which the snapshot creation failed) do report correctly the remote share path
they could probably be volumes where creation with type and extra spec failed, but this is probably not relevant to this bug
Giulio: Have you tested for or seen this issue on RHOS 4 at all?
We have not tested this with RHOS4 as we are waiting on hardware to use the newer unified driver from NetApp shipped in Havana.
I tried this out briefly w/ 3.0.z and this NetApp server and wasn't able to produce any problem -- need to dig more.
I haven't been able to reproduce this issue... can you please send it back my way if you can still produce it somehow?
In accordance with the Red Hat Enterprise Linux OpenStack Platform Support
Policy, the one-year life cycle of Production Support for version 3 will
end on July 31, 2014. On August 1, 2014, Red Hat Enterprise Linux OpenStack
Platform version 3 will enter an inactive state and will no longer receive
updated packages, including Critical-impact security patches or
urgent-priority bug fixes. In addition, technical support through Red Hat's
Global Support Services will no longer be provided after this date.
We encourage customers to plan their migration from Red Hat Enterprise
Linux OpenStack Platform 3.0 to a supported version of Red Hat Enterprise
Linux OpenStack Platform. To upgrade to Red Hat Enterprise Linux OpenStack
Platform version 4, see Chapter "Upgrading" in the Release Notes document
linked to in the References section.
Full details of the Red Hat Enterprise Linux OpenStack Platform Life Cycle
can be found at