Bug 1023958 - netapp NetAppNFSDriver fails at creating new snapshots
netapp NetAppNFSDriver fails at creating new snapshots
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder (Show other bugs)
Unspecified Unspecified
unspecified Severity medium
: ---
: 3.0
Assigned To: Eric Harney
Dafna Ron
: ZStream
Depends On:
  Show dependency treegraph
Reported: 2013-10-28 08:51 EDT by Giulio Fidente
Modified: 2016-04-26 19:16 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-08-01 08:52:21 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Giulio Fidente 2013-10-28 08:51:11 EDT
Description of problem:
NetAppNFSDriver does not create new snapshots and returns error, the volume.log traceback is:

2013-10-28 14:45:57    ERROR [cinder.openstack.common.rpc.amqp] Exception during message handling
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/amqp.py", line 430, in _process_data
    rval = self.proxy.dispatch(ctxt, version, method, **args)
  File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/dispatcher.py", line 133, in dispatch
    return getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 482, in create_snapshot
    {'status': 'error'})
  File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
  File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 473, in create_snapshot
    model_update = self.driver.create_snapshot(snapshot_ref)
  File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/netapp/nfs.py", line 87, in create_snapshot
  File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/netapp/nfs.py", line 132, in _clone_volume
    host_id = self._get_host_id(volume_id)
  File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/netapp/nfs.py", line 203, in _get_host_id
    resp = server.HostListInfoIterStart(ObjectNameOrId=host_ip)
  File "/usr/lib/python2.6/site-packages/suds/client.py", line 542, in __call__
    return client.invoke(args, kwargs)
  File "/usr/lib/python2.6/site-packages/suds/client.py", line 602, in invoke
    result = self.send(soapenv)
  File "/usr/lib/python2.6/site-packages/suds/client.py", line 657, in send
    result = self.failed(binding, e)
  File "/usr/lib/python2.6/site-packages/suds/client.py", line 718, in failed
    raise Exception((status, reason))
Exception: (404, u'Not Found')

Version-Release number of selected component (if applicable):

Steps to Reproduce:
1. configure Cinder with NetAppNFSDriver and the NetApp software accordingly
2. create a new volume
3. create a volume snapshot

Actual results:
snapshot creation fails and the snapshot is found in "error" state
Comment 1 Eric Harney 2013-10-29 09:12:52 EDT
It looks like this may happen if the volume's provider_location doesn't match the IP address of the NetApp machine.  Have you reconfigured networking parameters etc., lately for the NetApp machine?

What does
# mysql cinder -e "select id,provider_location from volumes"
show, versus "netapp_server_hostname" in cinder.conf (or the known address).
Comment 2 Giulio Fidente 2013-10-29 11:26:36 EDT
hi Eric,

that shouldn't be the case as the netapp vfiler didn't change nor I changed the hostname/dns/ip of it but I noticed the table has some NULL values:

| 07374fd2-af24-4b17-930a-1cd0cc33477c | vfiler02:/vol/vfilers/vfiler02 |
| 07bcd6e8-d681-4456-b8fb-08eab138b4f6 | vfiler02:/vol/vfilers/vfiler02 |
| 07d557b5-fd0b-4875-ab59-321ba131c65f | NULL                           |
| 09d2fe05-c727-42be-8255-7e3b9bf5374a | NULL                           |
| 09e3eaf0-e68e-4c83-9698-625d7e4554e5 | vfiler02:/vol/vfilers/vfiler02 |
| 0a0fcbe4-6e74-4a94-ab43-c94a5e760c41 | NULL                           |
| 0c26216b-e6ea-414f-8d6f-d925778dcc5c | vfiler02:/vol/vfilers/vfiler02 |

does that help?
Comment 3 Eric Harney 2013-10-30 17:56:45 EDT
I assume that one of those showing NULL is the volume that snapshot creation is failing on?

_get_host_id() calls _get_host_ip() which does:
        return self._get_provider_location(volume_id).split(':')[0]

So, host_id will be NULL in that case which is probably why the server returns a 404 for locating that object.
Comment 4 Giulio Fidente 2013-11-18 12:32:08 EST
hi Eric,

actually no, snapshotted volumes (volumes for which the snapshot creation failed) do report correctly the remote share path

they could probably be volumes where creation with type and extra spec failed, but this is probably not relevant to this bug
Comment 5 Eric Harney 2013-12-16 11:59:31 EST
Giulio: Have you tested for or seen this issue on RHOS 4 at all?
Comment 6 Giulio Fidente 2013-12-16 12:06:28 EST
We have not tested this with RHOS4 as we are waiting on hardware to use the newer unified driver from NetApp shipped in Havana.
Comment 7 Eric Harney 2014-01-31 17:26:56 EST
I tried this out briefly w/ 3.0.z and this NetApp server and wasn't able to produce any problem -- need to dig more.
Comment 8 Eric Harney 2014-02-05 12:52:40 EST
I haven't been able to reproduce this issue... can you please send it back my way if you can still produce it somehow?
Comment 11 Scott Lewis 2014-08-01 08:52:21 EDT
In accordance with the Red Hat Enterprise Linux OpenStack Platform Support
Policy, the one-year life cycle of Production Support for version 3 will
end on July 31, 2014. On August 1, 2014, Red Hat Enterprise Linux OpenStack
Platform version 3 will enter an inactive state and will no longer receive
updated packages, including Critical-impact security patches or
urgent-priority bug fixes. In addition, technical support through Red Hat's
Global Support Services will no longer be provided after this date.

We encourage customers to plan their migration from Red Hat Enterprise
Linux OpenStack Platform 3.0 to a supported version of Red Hat Enterprise
Linux OpenStack Platform. To upgrade to Red Hat Enterprise Linux OpenStack
Platform version 4, see Chapter "Upgrading" in the Release Notes document
linked to in the References section.

Full details of the Red Hat Enterprise Linux OpenStack Platform Life Cycle
can be found at


Note You need to log in before you can comment on or make changes to this bug.