Bug 2242504 - [OSP16] After FFU can't create volumes on ceph backend. Capacity is now negative.
Summary: [OSP16] After FFU can't create volumes on ceph backend. Capacity is now negat...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 16.2 (Train)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: async
: 16.2 (Train on RHEL 8.4)
Assignee: Brian Rosmaita
QA Contact: Luigi Toscano
RHOS Documentation Team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-10-06 15:13 UTC by ggrimaux
Modified: 2024-03-26 12:25 UTC (History)
10 users (show)

Fixed In Version: openstack-cinder-15.6.1-2.20230906144857.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2024-03-26 12:25:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1960206 0 None None None 2023-11-08 13:28:29 UTC
OpenStack gerrit 876474 0 None MERGED RBD: Fix total_capacity 2023-10-09 13:25:03 UTC
Red Hat Bugzilla 2203199 0 high CLOSED Backport RBD capacity calculation fix 2024-03-12 16:29:47 UTC
Red Hat Issue Tracker OSP-29306 0 None None None 2023-10-06 15:15:45 UTC
Red Hat Product Errata RHBA-2024:1519 0 None None None 2024-03-26 12:25:51 UTC

Description ggrimaux 2023-10-06 15:13:06 UTC
Description of problem:
Client completed FFU from 13 to 16.2.3 with ceph backend.
Now tried to create a 5GB volume in one of the ceph pool and it fails.

In cinder-scheduler logs we see:
 Insufficient free virtual space (-16916.4GB) to accommodate thin provisioned 5GB volume on host

After reaching out to engineering it is thought it could be this that would need backported:
https://bugs.launchpad.net/cinder/+bug/1960206

I asked the client for 'ceph df -f json' and 'ceph --version' to confirm if its the case.

If not then we need to understand if the calculation changed somewhere else during the FFU.

We have sosreport with logs in debug.

I will put below in a private comment the data we found.

Thank you.

Version-Release number of selected component (if applicable):
OSP16.2.3

How reproducible:
100% 

Steps to Reproduce:
1. FFU to 16.2.3 with ceph
2.
3.

Actual results:
Can't create a volume on that pool 

Expected results:
Create a volume on this pool

Additional info:
sosreport with cinder in debug.

Comment 12 Luigi Toscano 2024-03-19 09:00:29 UTC
The fix is present in the candidate build, namely:

python3-cinder-15.6.1-2.20230906144857.el8ost.noarch
openstack-cinder-15.6.1-2.20230906144857.el8ost.noarch

The calculation of the available space, as shown by the cinder-scheduler logs, considers the STORED field ("stored") and not the USED field ("bytes_used"), as expected.

Comment 17 errata-xmlrpc 2024-03-26 12:25:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.2.6 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:1519


Note You need to log in before you can comment on or make changes to this bug.