Description of problem: While trying to create an incremental backup the following error returned from Cinder CLI : ERROR: Invalid backup: The parent backup must be available for incremental backup. (HTTP 400) (Request-ID: req-207069a5-8699-4b65-b8b5-1d72364a0f83) Version-Release number of selected component (if applicable): python-cinder-13.0.1-0.20181013185427.31ff628.el7ost.noarch python2-cinderclient-4.0.1-0.20180809133302.460229c.el7ost.noarch puppet-cinder-13.3.1-0.20181013114719.25b1ba3.el7ost.noarch openstack-cinder-13.0.1-0.20181013185427.31ff628.el7ost.noarch How reproducible: Steps to Reproduce: 1.Install setup with backup and backend in different availability zone 2.Create volume from image while using micro version 3.51 in the CLI 3.Create volume backup 3.Mount volume and add file to volume 4.Unmount the volume 5.create incremental backup by running the following command: cinder backup-create --name vol-cirros-backup-inc --availability-zone nova --incremental vol-cirros Actual results: The following error is returned from the CLI ERROR: Invalid backup: The parent backup must be available for incremental backup. (HTTP 400) (Request-ID: req-207069a5-8699-4b65-b8b5-1d72364a0f83) Expected results: Incremental backup will created successfully Additional info: (overcloud) [stack@undercloud-0 ~]$ cinder --debug backup-create --name vol-cirros-backup-inc --availability-zone nova --incremental vol-cirros DEBUG:keystoneauth.session:REQ: curl -g -i -X GET http://10.0.0.120:5000/v3 -H "Accept: application/json" -H "User-Agent: cinder keystoneauth1/3.10.0 python-requests/2.19.1 CPython/2.7.5" DEBUG:keystoneauth.session:RESP: [200] Content-Encoding: gzip Content-Length: 192 Content-Type: application/json Date: Sun, 18 Nov 2018 13:29:01 GMT Server: Apache Vary: X-Auth-Token,Accept-Encoding x-openstack-request-id: req-07901c9f-39af-4869-a90f-dcdeea5f676a DEBUG:keystoneauth.session:RESP BODY: {"version": {"status": "stable", "updated": "2018-02-28T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.10", "links": [{"href": "http://10.0.0.120:5000/v3/", "rel": "self"}]}} DEBUG:keystoneauth.session:GET call to http://10.0.0.120:5000/v3 used request id req-07901c9f-39af-4869-a90f-dcdeea5f676a DEBUG:keystoneauth.identity.v3.base:Making authentication request to http://10.0.0.120:5000/v3/auth/tokens DEBUG:keystoneauth.identity.v3.base:{"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "9914a7ee6d68421ca2a6753fac7055c2", "name": "admin"}, {"id": "ace017c68cc040c8bca658760249dccd", "name": "member"}, {"id": "6a13911276e844c9ba2a4e7e89112385", "name": "reader"}], "expires_at": "2018-11-18T14:29:01.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "11db1f9f92cd4d008768cbd8be6b401a", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://10.0.0.120:9696", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "1747ce3a6aa44fca966f74c9c8cffcbc"}, {"url": "http://172.17.1.29:9696", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "42b3a24e147046548800c9a4c13a2469"}, {"url": "http://172.17.1.29:9696", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "f44fb36d5b8e4e91b2471533fa79ef50"}], "type": "network", "id": "0d022049362d4ea5808d4ab494760ddb", "name": "neutron"}, {"endpoints": [{"url": "http://10.0.0.120:8080/v1/AUTH_11db1f9f92cd4d008768cbd8be6b401a", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "4b323269141c4bdfad78ecbd45510011"}, {"url": "http://172.17.3.21:8080", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "957a651edafd4ba8b27f12025bfcb39f"}, {"url": "http://172.17.3.21:8080/v1/AUTH_11db1f9f92cd4d008768cbd8be6b401a", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "b62cffb6132046d3b1c9c2313b06285f"}], "type": "object-store", "id": "19b219164e11442c9ea92a1c891cb45e", "name": "swift"}, {"endpoints": [{"url": "http://172.17.1.29:8042", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "671b028aacb14d4c88e12d6775504a59"}, {"url": "http://10.0.0.120:8042", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "ac8ad7e1dfff4dc0a1c0abf209f621dd"}, {"url": "http://172.17.1.29:8042", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "e7784f6170bd43bd88c444f78670e0e4"}], "type": "alarming", "id": "2cf6eec333554fb79bc1666effae93bb", "name": "aodh"}, {"endpoints": [], "type": "metering", "id": "4023e4573636473aa381c8dfc23f9f5f", "name": "ceilometer"}, {"endpoints": [{"url": "http://10.0.0.120:8778/placement", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "11c4fbeeb46b4ff18fc20a4a539f764e"}, {"url": "http://172.17.1.29:8778/placement", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "1ab5e2d25d944102869cb4294867ec13"}, {"url": "http://172.17.1.29:8778/placement", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "3a6c401bdd5240f1a4b796de2aa3fd4b"}], "type": "placement", "id": "4a5680cfddf4418e93fadee50e2bedcf", "name": "placement"}, {"endpoints": [{"url": "http://172.17.1.29:8776/v3/11db1f9f92cd4d008768cbd8be6b401a", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "24f865013c8d4ed1a4ff3901f55f8b1f"}, {"url": "http://10.0.0.120:8776/v3/11db1f9f92cd4d008768cbd8be6b401a", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "3bdd331b7fb346649ef82fd92c6e9c52"}, {"url": "http://172.17.1.29:8776/v3/11db1f9f92cd4d008768cbd8be6b401a", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "977faf4a078b4398b83d4edce986e369"}], "type": "volumev3", "id": "4e6bab7279424e21abc2dc0baefc810a", "name": "cinderv3"}, {"endpoints": [{"url": "http://10.0.0.120:8000/v1", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "2df749a8079540e484eb5f69ed2cd1e6"}, {"url": "http://172.17.1.29:8000/v1", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "8e6f965d6b6a458c9553c7fe3631377f"}, {"url": "http://172.17.1.29:8000/v1", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "d198d581e1974be5911849545cc1468e"}], "type": "cloudformation", "id": "4f45833e3d3644518da6c63705ddcf17", "name": "heat-cfn"}, {"endpoints": [{"url": "http://172.17.1.29:9292", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "3268964d090c4eedbb0322c57456f405"}, {"url": "http://10.0.0.120:9292", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "b8181f989e18446e9f923277600e66d4"}, {"url": "http://172.17.1.29:9292", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "fc708e8623c14f0988f10db857b9af0f"}], "type": "image", "id": "5f7bb8ab4be14676abe359af8ea4a411", "name": "glance"}, {"endpoints": [{"url": "http://10.0.0.120:8776/v2/11db1f9f92cd4d008768cbd8be6b401a", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "8f48db8170424a56b0cfd8650e8ad45e"}, {"url": "http://172.17.1.29:8776/v2/11db1f9f92cd4d008768cbd8be6b401a", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "b71e1c0f291a43e89446ede6ae579ec0"}, {"url": "http://172.17.1.29:8776/v2/11db1f9f92cd4d008768cbd8be6b401a", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "e8d875caad5b47d89ca4afbca01357d1"}], "type": "volumev2", "id": "7330e889b4a54e61ac2895f4bd10495a", "name": "cinderv2"}, {"endpoints": [{"url": "http://10.0.0.120:8774/v2.1", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "453ad9ae6d9248058406cd82fcd0301d"}, {"url": "http://172.17.1.29:8774/v2.1", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "c97a8dc0763d49cc9c675b542d84e377"}, {"url": "http://172.17.1.29:8774/v2.1", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "ef7bf5fcef6645a2951609e7464d2d38"}], "type": "compute", "id": "7e0c8864962d4bb8a4003c491ccf306c", "name": "nova"}, {"endpoints": [{"url": "http://172.17.1.29:8977", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "34ecd544d23a4f0bbb209237f50a71d3"}, {"url": "http://172.17.1.29:8977", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "727d776fadf543e68daaf8de70ac1577"}, {"url": "http://10.0.0.120:8977", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "86acccf4ff284cc7baf038b55019b3c1"}], "type": "event", "id": "83435330b5b84beea1dbed589589d263", "name": "panko"}, {"endpoints": [{"url": "http://10.0.0.120:8776/v1/11db1f9f92cd4d008768cbd8be6b401a", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "294ee9160ac84a69868d64323be97a78"}, {"url": "http://172.17.1.29:8776/v1/11db1f9f92cd4d008768cbd8be6b401a", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "6fd12eeca446400f8308f957f12aaf10"}, {"url": "http://172.17.1.29:8776/v1/11db1f9f92cd4d008768cbd8be6b401a", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "74763432dc464c40934fd372719a5a9b"}], "type": "volume", "id": "8c7838f3bb6f4e2badbe97cb611231b1", "name": "cinder"}, {"endpoints": [{"url": "http://10.0.0.120:8041", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "260ba3e0d3134683ac9abb9b91fabb43"}, {"url": "http://172.17.1.29:8041", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "e79cf4eb7d724ee6adb0946b4e04fe6f"}, {"url": "http://172.17.1.29:8041", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "fd0e800b9cad4dbc88f49c1a9281866c"}], "type": "metric", "id": "90992b93cd2f41fb9de2088b10e0ccea", "name": "gnocchi"}, {"endpoints": [{"url": "http://172.17.1.29:9311", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "11e4eaa97f834048ba94de2fcf1faee6"}, {"url": "http://172.17.1.29:9311", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "1934c7ce66054e5fb77902678e207e5d"}, {"url": "http://10.0.0.120:9311", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "d68fd75985a84eaeaa8ddf4ada82158c"}], "type": "key-manager", "id": "a95d9f79e1dc4e4bad5474df8a1120d0", "name": "barbican"}, {"endpoints": [{"url": "http://10.0.0.120:8004/v1/11db1f9f92cd4d008768cbd8be6b401a", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "37cf41b3ba97435a97fed9d2afd2868a"}, {"url": "http://172.17.1.29:8004/v1/11db1f9f92cd4d008768cbd8be6b401a", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "72fe062ed8b04884af5ab27b1e49c949"}, {"url": "http://172.17.1.29:8004/v1/11db1f9f92cd4d008768cbd8be6b401a", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "d53ce550f49e46b98799ee97de7f4059"}], "type": "orchestration", "id": "bd1e629930524ee5a2932070127003dd", "name": "heat"}, {"endpoints": [{"url": "http://172.17.1.29:5000", "interface": "internal", "region": "regionOne", "region_id": "regionOne", "id": "710aa453e5ac445d95bb308200237674"}, {"url": "http://192.168.24.8:35357", "interface": "admin", "region": "regionOne", "region_id": "regionOne", "id": "8121696e9f1b4696bfffe30b0e114f04"}, {"url": "http://10.0.0.120:5000", "interface": "public", "region": "regionOne", "region_id": "regionOne", "id": "edf3295b2cc94177a07b2151bc2c0320"}], "type": "identity", "id": "f1ed5125fad14361aa0857bf335f4033", "name": "keystone"}], "user": {"domain": {"id": "default", "name": "Default"}, "password_expires_at": null, "name": "admin", "id": "327285d828884f3f84a886f9e6a70631"}, "audit_ids": ["LFYlRpbWTsKJlHRQ6fzikA"], "issued_at": "2018-11-18T13:29:01.000000Z"}} DEBUG:keystoneauth:REQ: curl -g -i -X GET http://10.0.0.120:8776/v3/11db1f9f92cd4d008768cbd8be6b401a/volumes/detail?all_tenants=1&name=vol-cirros -H "Accept: application/json" -H "OpenStack-API-Version: volume 3.51" -H "User-Agent: python-cinderclient" -H "X-Auth-Token: {SHA1}a27443c0fb1d999fc1b626d0098115cd042c347c" DEBUG:keystoneauth:RESP: [200] Content-Encoding: gzip Content-Length: 780 Content-Type: application/json Date: Sun, 18 Nov 2018 13:29:02 GMT OpenStack-API-Version: volume 3.51 Server: Apache Vary: OpenStack-API-Version,Accept-Encoding x-compute-request-id: req-0beb128b-184f-4c01-bbca-2f98ed425348 x-openstack-request-id: req-0beb128b-184f-4c01-bbca-2f98ed425348 DEBUG:keystoneauth:RESP BODY: {"volumes": [{"migration_status": null, "provider_id": null, "attachments": [], "links": [{"href": "http://10.0.0.120:8776/v3/11db1f9f92cd4d008768cbd8be6b401a/volumes/c3ede874-a327-4205-8a4f-cf340f1930e1", "rel": "self"}, {"href": "http://10.0.0.120:8776/11db1f9f92cd4d008768cbd8be6b401a/volumes/c3ede874-a327-4205-8a4f-cf340f1930e1", "rel": "bookmark"}], "availability_zone": "alfa", "os-vol-host-attr:host": "hostgroup@tripleo_netapp#rhos_cinder", "encrypted": false, "updated_at": "2018-11-18T12:20:13.000000", "replication_status": null, "snapshot_id": null, "id": "c3ede874-a327-4205-8a4f-cf340f1930e1", "size": 1, "user_id": "327285d828884f3f84a886f9e6a70631", "os-vol-tenant-attr:tenant_id": "11db1f9f92cd4d008768cbd8be6b401a", "os-vol-mig-status-attr:migstat": null, "metadata": {}, "status": "available", "volume_image_metadata": {"checksum": "7316af7358dd32ca1956d72ac2c9e147", "min_ram": "0", "disk_format": "qcow2", "image_name": "cirros", "owner_specified.shade.sha256": "53ef7941846012d031db80d2936b23de65c912ae45a1a68b21adb661f080a21b", "image_id": "0ed6295e-aa77-4431-be0a-24b8849758ae", "signature_verified": "False", "owner_specified.shade.object": "images/cirros", "container_format": "bare", "min_disk": "0", "owner_specified.shade.md5": "7316af7358dd32ca1956d72ac2c9e147", "size": "12528640"}, "description": null, "multiattach": false, "service_uuid": "9353317d-52b1-492b-87b0-5ef5817f5b7d", "source_volid": null, "consistencygroup_id": null, "os-vol-mig-status-attr:name_id": null, "name": "vol-cirros", "bootable": "true", "created_at": "2018-11-14T06:53:45.000000", "volume_type": "tripleo_netapp", "group_id": null, "shared_targets": true}]} DEBUG:keystoneauth:GET call to volumev3 for http://10.0.0.120:8776/v3/11db1f9f92cd4d008768cbd8be6b401a/volumes/detail?all_tenants=1&name=vol-cirros used request id req-0beb128b-184f-4c01-bbca-2f98ed425348 DEBUG:keystoneauth:REQ: curl -g -i -X POST http://10.0.0.120:8776/v3/11db1f9f92cd4d008768cbd8be6b401a/backups -H "Accept: application/json" -H "Content-Type: application/json" -H "OpenStack-API-Version: volume 3.51" -H "User-Agent: python-cinderclient" -H "X-Auth-Token: {SHA1}a27443c0fb1d999fc1b626d0098115cd042c347c" -H "X-OpenStack-Request-ID: req-0beb128b-184f-4c01-bbca-2f98ed425348" -d '{"backup": {"force": false, "description": null, "availability_zone": "nova", "snapshot_id": null, "incremental": true, "volume_id": "c3ede874-a327-4205-8a4f-cf340f1930e1", "container": null, "name": "vol-cirros-backup-inc"}}' DEBUG:keystoneauth:RESP: [400] Content-Length: 119 Content-Type: application/json Date: Sun, 18 Nov 2018 13:29:03 GMT OpenStack-API-Version: volume 3.51 Server: Apache Vary: OpenStack-API-Version x-compute-request-id: req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074 x-openstack-request-id: req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074 DEBUG:keystoneauth:RESP BODY: {"badRequest": {"message": "Invalid backup: The parent backup must be available for incremental backup.", "code": 400}} DEBUG:keystoneauth:POST call to volumev3 for http://10.0.0.120:8776/v3/11db1f9f92cd4d008768cbd8be6b401a/backups used request id req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074 DEBUG (shell:914) Invalid backup: The parent backup must be available for incremental backup. (HTTP 400) (Request-ID: req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cinderclient/shell.py", line 909, in main for item in sys.argv[1:]]) File "/usr/lib/python2.7/site-packages/cinderclient/shell.py", line 729, in main args.func(self.cs, args) File "/usr/lib/python2.7/site-packages/cinderclient/v3/shell.py", line 2396, in do_backup_create **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/api_versions.py", line 407, in substitution return method.func(obj, *args, **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/v3/volume_backups.py", line 93, in create availability_zone) File "/usr/lib/python2.7/site-packages/cinderclient/v3/volume_backups.py", line 126, in _create_backup return self._create('/backups', body, 'backup') File "/usr/lib/python2.7/site-packages/cinderclient/base.py", line 330, in _create resp, body = self.api.client.post(url, body=body) File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 203, in post return self._cs_request(url, 'POST', **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 191, in _cs_request return self.request(url, method, **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 177, in request raise exceptions.from_response(resp, body) BadRequest: Invalid backup: The parent backup must be available for incremental backup. (HTTP 400) (Request-ID: req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074) DEBUG:cinderclient.shell:Invalid backup: The parent backup must be available for incremental backup. (HTTP 400) (Request-ID: req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cinderclient/shell.py", line 909, in main for item in sys.argv[1:]]) File "/usr/lib/python2.7/site-packages/cinderclient/shell.py", line 729, in main args.func(self.cs, args) File "/usr/lib/python2.7/site-packages/cinderclient/v3/shell.py", line 2396, in do_backup_create **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/api_versions.py", line 407, in substitution return method.func(obj, *args, **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/v3/volume_backups.py", line 93, in create availability_zone) File "/usr/lib/python2.7/site-packages/cinderclient/v3/volume_backups.py", line 126, in _create_backup return self._create('/backups', body, 'backup') File "/usr/lib/python2.7/site-packages/cinderclient/base.py", line 330, in _create resp, body = self.api.client.post(url, body=body) File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 203, in post return self._cs_request(url, 'POST', **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 191, in _cs_request return self.request(url, method, **kwargs) File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 177, in request raise exceptions.from_response(resp, body) BadRequest: Invalid backup: The parent backup must be available for incremental backup. (HTTP 400) (Request-ID: req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074) ERROR: Invalid backup: The parent backup must be available for incremental backup. (HTTP 400) (Request-ID: req-92bff459-6289-4fb0-a4f7-7ea5f6ac3074)
Questions: - Where are the logs? - What is the status of the initial backup?
I will add the log files to bugzilla The parent backup is in available state
The problem was caused by a known bug in the NFS backend. The problem was not observed when testing with a different backend.