| Summary: | cinder: don't count snapshots with error_deleting as part of quota | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Dafna Ron <dron> | ||||
| Component: | openstack-cinder | Assignee: | Eric Harney <eharney> | ||||
| Status: | CLOSED WONTFIX | QA Contact: | Dafna Ron <dron> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 4.0 | CC: | eharney, fpercoco, sgotliv, yeylon | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 5.0 (RHEL 7) | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | storage | ||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2014-05-01 16:42:17 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Attachments: |
|
||||||
Snapshots in error_deleting must be counted as part of the quota. In many cases, error_deleting = the snapshot still exists and is consuming storage resources. Cinder can't currently differentiate between failures to delete before and after the snapshot itself or its backing storage was removed, so we have to assume it should still be counted. This may change eventually if/when we start reworking Cinder's error handling mechanisms. I disagree even if a snapshot does exist and consumes storage resources, as long as we cannot delete that snapshot and we cannot add any more because of quota, I think we are disrupting work for something that should be secondary like deleting snapshots. FWIW, I agree with Eric here. Not counting `error_deleting` snapshots could end up in exceeding the storage space itself too. It's true that the system failed to delete the snapshot but that's something the system operator will have to take care of. IMHO. |
Created attachment 827688 [details] logs Description of problem: if we cannot delete volume snapshots they get error_deleting status but still counted as part of quota. so we cannot delete the old snapshots to make room for the new ones and we cannot create more snapshots because of quota. Version-Release number of selected component (if applicable): openstack-cinder-2013.2-2.el6ost.noarch How reproducible: 100% Steps to Reproduce: 1. create 10 snapshots (default quota) 2. in db, change their status to error_deleting: update snapshots set status='error_deleting' where status='available'; 3. try to add new snapshots Actual results: we fail to add any snapshots and fail to delete the old snapshots Expected results: we need to allow normal system function when there is a problem deleting. Additional info: [root@cougar06 ~(keystone_admin)]# cinder snapshot-list +--------------------------------------+--------------------------------------+----------------+-------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+----------------+-------------------+------+ | 2b1a1dc5-3095-467d-8fda-0527d520ea2c | a142d020-1fec-4b39-967c-c171696920a5 | error_deleting | vol7-3 | 10 | | 48c5bc0b-5a80-40a8-b20e-bbd11d09ec73 | a142d020-1fec-4b39-967c-c171696920a5 | error_deleting | vol7-4 | 10 | | 50cab642-20b3-4b85-bcdf-0c3384c7b0a3 | a142d020-1fec-4b39-967c-c171696920a5 | error_deleting | vol7-2 | 10 | | 90a9988b-1e51-4a15-a744-8f2117bec97f | a142d020-1fec-4b39-967c-c171696920a5 | error_deleting | vol7-3 | 10 | | 9bbb15d6-4a79-4258-9221-29ad3e5ff201 | 292e6a93-922f-4370-806a-33fbf1ef48c7 | error_deleting | test1 | 9 | | a8662b0a-c26b-44a8-b129-15280146e047 | a142d020-1fec-4b39-967c-c171696920a5 | error_deleting | vol7 | 10 | | ad99b7d3-6811-49e6-8777-0aa4b05c39e4 | a142d020-1fec-4b39-967c-c171696920a5 | error_deleting | vol7 | 10 | | b374bb34-5887-4b7a-bfba-4ad0ce30faf7 | 128681f1-8af2-44fc-bcd6-e3075687f67a | error_deleting | snapshot for snap | 12 | | b975b1df-95f7-4012-9cd8-4e329f3ee15b | a142d020-1fec-4b39-967c-c171696920a5 | error_deleting | None | 10 | | ff96e8bd-2288-4f37-8f27-89826c3bfb05 | 292e6a93-922f-4370-806a-33fbf1ef48c7 | error_deleting | None | 9 | [root@cougar06 ~(keystone_admin)]# cinder snapshot-create e5025313-2b91-4c43-b471-aca33b8a78a5 --display-name gaga ERROR: SnapshotLimitExceeded: Maximum number of snapshots allowed (10) exceeded (HTTP 413) (Request-ID: req-9b95ac65-847c-442d-86af-a77e98cfea94)