Bug 2131386

+++ This bug was initially created as a clone of Bug #1802263 +++

Description of problem:

After successfully using cinder backup-import to recover a backup that exists in Ceph but not in the DB, if I re-run the backup-import command a second time, it deletes the backup from the DB.  In the below example, I am trying to import a backup called 'attached-full':

# Show that attached-full does not exist

(overcloud) [stack@director16 ~]$ openstack volume backup list
+--------------------------------------+---------------+-------------+-----------+------+
| ID                                   | Name          | Description | Status    | Size |
+--------------------------------------+---------------+-------------+-----------+------+
| 1b1b2d06-6dc1-4a34-9f3b-7dd852ad3fa6 | None          | None        | available |    1 |
| 6ada70b0-ed1a-47de-9661-37517844cea1 | detached-full | None        | available |    1 |
| 55904715-fe70-4e4b-ac0d-458ceb4c9633 | None          | snap-backup | available |    1 |
| 5ca9ed1b-e719-47fa-bbfd-a57d5f1b877b | None          | None        | available |    1 |
+--------------------------------------+---------------+-------------+-----------+------+

# Import the backup data

(overcloud) [stack@director16 ~]$ cinder backup-import cinder.backup.drivers.ceph.CephBackupDriver eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiODU1YWFhYTUtZTU5MS00M2I2LWJlMjAtYTcxNTBiOWRjOTM0IiwgInVzZXJfaWQiOiAiMWUwM2U0NDVlNWUzNDcyODhiNWViNTMxMjljZWU5ZTkiLCAicHJvamVjdF9pZCI6ICJhY2NkYzk2ZmFjMjU0YjJkOTVhYTNhODY2Yzg5NjliMCIsICJ2b2x1bWVfaWQiOiAiY2VlOWIyZDgtNGJhMS00MmIwLTlmMzYtNjIxZjFmYTk3YTgwIiwgImhvc3QiOiAib3ZlcmNsb3VkLWNvbnRyb2xsZXItMCIsICJhdmFpbGFiaWxpdHlfem9uZSI6ICJub3ZhIiwgImNvbnRhaW5lciI6ICJiYWNrdXBzIiwgInBhcmVudF9pZCI6IG51bGwsICJwYXJlbnQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJhdHRhY2hlZC1mdWxsIiwgImRpc3BsYXlfZGVzY3JpcHRpb24iOiBudWxsLCAic2VydmljZV9tZXRhZGF0YSI6ICJ7XCJiYXNlXCI6IFwidm9sdW1lLWNlZTliMmQ4LTRiYTEtNDJiMC05ZjM2LTYyMWYxZmE5N2E4MC5iYWNrdXAuODU1YWFhYTUtZTU5MS00M2I2LWJlMjAtYTcxNTBiOWRjOTM0XCJ9IiwgInNlcnZpY2UiOiAiY2luZGVyLmJhY2t1cC5kcml2ZXJzLmNlcGguQ2VwaEJhY2t1cERyaXZlciIsICJvYmplY3RfY291bnQiOiAwLCAidGVtcF92b2x1bWVfaWQiOiBudWxsLCAidGVtcF9zbmFwc2hvdF9pZCI6IG51bGwsICJudW1fZGVwZW5kZW50X2JhY2t1cHMiOiAwLCAic25hcHNob3RfaWQiOiBudWxsLCAiZGF0YV90aW1lc3RhbXAiOiAiMjAyMC0wMi0xMlQxNzowMjozOVoiLCAicmVzdG9yZV92b2x1bWVfaWQiOiBudWxsLCAibWV0YWRhdGEiOiB7fSwgImVuY3J5cHRpb25fa2V5X2lkIjogbnVsbCwgImNyZWF0ZWRfYXQiOiAiMjAyMC0wMi0xMlQxNzowMjozOVoiLCAidXBkYXRlZF9hdCI6ICIyMDIwLTAyLTEyVDE3OjAyOjU1WiIsICJkZWxldGVkX2F0IjogbnVsbCwgImRlbGV0ZWQiOiBmYWxzZX0=
+----------+--------------------------------------+
| Property | Value                                |
+----------+--------------------------------------+
| id       | 855aaaa5-e591-43b6-be20-a7150b9dc934 |
| name     | attached-full                        |
+----------+--------------------------------------+

# Show backup attached-full exists

(overcloud) [stack@director16 ~]$ openstack volume backup list
+--------------------------------------+---------------+-------------+-----------+------+
| ID                                   | Name          | Description | Status    | Size |
+--------------------------------------+---------------+-------------+-----------+------+
| 1b1b2d06-6dc1-4a34-9f3b-7dd852ad3fa6 | None          | None        | available |    1 |
| 855aaaa5-e591-43b6-be20-a7150b9dc934 | attached-full | None        | available |    1 |
| 6ada70b0-ed1a-47de-9661-37517844cea1 | detached-full | None        | available |    1 |
| 55904715-fe70-4e4b-ac0d-458ceb4c9633 | None          | snap-backup | available |    1 |
| 5ca9ed1b-e719-47fa-bbfd-a57d5f1b877b | None          | None        | available |    1 |
+--------------------------------------+---------------+-------------+-----------+------+

# Re-run the import which fails, correctly stating that the backup already exists

(overcloud) [stack@director16 ~]$ cinder backup-import cinder.backup.drivers.ceph.CephBackupDriver eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiODU1YWFhYTUtZTU5MS00M2I2LWJlMjAtYTcxNTBiOWRjOTM0IiwgInVzZXJfaWQiOiAiMWUwM2U0NDVlNWUzNDcyODhiNWViNTMxMjljZWU5ZTkiLCAicHJvamVjdF9pZCI6ICJhY2NkYzk2ZmFjMjU0YjJkOTVhYTNhODY2Yzg5NjliMCIsICJ2b2x1bWVfaWQiOiAiY2VlOWIyZDgtNGJhMS00MmIwLTlmMzYtNjIxZjFmYTk3YTgwIiwgImhvc3QiOiAib3ZlcmNsb3VkLWNvbnRyb2xsZXItMCIsICJhdmFpbGFiaWxpdHlfem9uZSI6ICJub3ZhIiwgImNvbnRhaW5lciI6ICJiYWNrdXBzIiwgInBhcmVudF9pZCI6IG51bGwsICJwYXJlbnQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJhdHRhY2hlZC1mdWxsIiwgImRpc3BsYXlfZGVzY3JpcHRpb24iOiBudWxsLCAic2VydmljZV9tZXRhZGF0YSI6ICJ7XCJiYXNlXCI6IFwidm9sdW1lLWNlZTliMmQ4LTRiYTEtNDJiMC05ZjM2LTYyMWYxZmE5N2E4MC5iYWNrdXAuODU1YWFhYTUtZTU5MS00M2I2LWJlMjAtYTcxNTBiOWRjOTM0XCJ9IiwgInNlcnZpY2UiOiAiY2luZGVyLmJhY2t1cC5kcml2ZXJzLmNlcGguQ2VwaEJhY2t1cERyaXZlciIsICJvYmplY3RfY291bnQiOiAwLCAidGVtcF92b2x1bWVfaWQiOiBudWxsLCAidGVtcF9zbmFwc2hvdF9pZCI6IG51bGwsICJudW1fZGVwZW5kZW50X2JhY2t1cHMiOiAwLCAic25hcHNob3RfaWQiOiBudWxsLCAiZGF0YV90aW1lc3RhbXAiOiAiMjAyMC0wMi0xMlQxNzowMjozOVoiLCAicmVzdG9yZV92b2x1bWVfaWQiOiBudWxsLCAibWV0YWRhdGEiOiB7fSwgImVuY3J5cHRpb25fa2V5X2lkIjogbnVsbCwgImNyZWF0ZWRfYXQiOiAiMjAyMC0wMi0xMlQxNzowMjozOVoiLCAidXBkYXRlZF9hdCI6ICIyMDIwLTAyLTEyVDE3OjAyOjU1WiIsICJkZWxldGVkX2F0IjogbnVsbCwgImRlbGV0ZWQiOiBmYWxzZX0=
ERROR: Invalid backup: Backup already exists in database. (HTTP 400) (Request-ID: req-6fa32aec-2c21-41f9-8845-c52d303771b5)

# Backup is now gone:

(overcloud) [stack@director16 ~]$ openstack volume backup list
+--------------------------------------+---------------+-------------+-----------+------+
| ID                                   | Name          | Description | Status    | Size |
+--------------------------------------+---------------+-------------+-----------+------+
| 1b1b2d06-6dc1-4a34-9f3b-7dd852ad3fa6 | None          | None        | available |    1 |
| 6ada70b0-ed1a-47de-9661-37517844cea1 | detached-full | None        | available |    1 |
| 55904715-fe70-4e4b-ac0d-458ceb4c9633 | None          | snap-backup | available |    1 |
| 5ca9ed1b-e719-47fa-bbfd-a57d5f1b877b | None          | None        | available |    1 |
+--------------------------------------+---------------+-------------+-----------+------+
(overcloud) [stack@director16 ~]$ 

 

Version-Release number of selected component (if applicable):

16

How reproducible:

To create the scenario in order to test the backup export/import:

Create backup of a volume, note backup ID
Export the DB data using cinder backup-export
SSH to overcloud controller
Use podman to enter the galera container
Use mysql to access the cinder database
Delete from cinder.backup where id='<backup ID>'
Exit back out to Director
Confirm the backup is gone by listing them and perform the import as detailed above.


Actual results:
Expected results:

Backup restored in database.

Additional info:

--- Additional comment from Luigi Toscano on 2020-02-13 14:02:59 UTC ---

(In reply to Darin Sorrentino from comment #0)

> 
> To create the scenario in order to test the backup export/import:

Isn't the scenario already described by the steps reported above, with all the openstack commands? If it is correct, what is the purpose of the following steps?

> 
> Create backup of a volume, note backup ID
> Export the DB data using cinder backup-export
> SSH to overcloud controller
> Use podman to enter the galera container
> Use mysql to access the cinder database
> Delete from cinder.backup where id='<backup ID>'
> Exit back out to Director
> Confirm the backup is gone by listing them and perform the import as
> detailed above.

Also, does it work if you use the "cinder" instead of openstack volume?

--- Additional comment from Darin Sorrentino on 2020-02-17 20:38:08 UTC ---

Luigi,
  No, it's not.  As far as I read, the purpose of the cinder backup is to restore the DB entries in the event of a catastrophic loss of data in the DB.  In order to "imitate a catastrophic loss of data", one must actually go into the DB and delete the record, which is the steps I provided:

> Create backup of a volume, note backup ID
> Export the DB data using cinder backup-export
> SSH to overcloud controller
> Use podman to enter the galera container
> Use mysql to access the cinder database
> Delete from cinder.backup where id='<backup ID>'
> Exit back out to Director
> Confirm the backup is gone by listing them and perform the import as
> detailed above.

Once you do that, THEN you can use the cinder backup restore procedure to restore the volume to the DB as detailed in the BZ description.

The import/export commands are "cinder" commands and not "openstack" commands.

Are you asking if "cinder list" has a different output from "openstack volume list"?  If that is what you're asking, no, it does not. Both show the same information.

--- Additional comment from Luigi Toscano on 2020-02-28 11:28:51 UTC ---

Thanks. Reproduced.

--- Additional comment from Luigi Toscano on 2020-08-24 14:28:54 UTC ---

No work and solution yet, moving to z3.