Created attachment 1819596 [details] usage screenshot Description of problem (please be detailed as possible and provide log snippests): Used storage on an empty cluster shows 9 GB without creating any PVCs. Looks like it may be reproducible on other platforms as well. Would like to know what is the reason for this much usage on an empty cluster. Version of all relevant components (if applicable): 4.8, can be on other versions as well Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? No Is there any workaround available to the best of your knowledge? Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 1 Can this issue reproducible? yes Can this issue reproduce from the UI? yes If this is a regression, please provide more details to justify this: Steps to Reproduce: 1. Deploy ocs and check the used space from web console 2. 3. Actual results: Expected results: Additional info:
I tested this on AWS setup: OCP 4.9.0-0.nightly-2021-09-06-055314 ODF 4.9.0-120.ci provided by Red Hat It showing 250 to 300 MB of raw capacity used
Created attachment 1820930 [details] raw_capacity_used
may i know what additional info i can provide? maybe i am missing permission to view comments?
Marked the comments public
@akandath As I commented above, I have tried multiple times in AWS platform, But i can't reproduce this issue. It showing 250 to 300 MB of raw capacity used. Are you facing this issue when OCS deployed on top of IBM flash system?
Hi @gshanmug, The disks for LSO are from the ds8k server, which is not an IBM Flash system.
Here the disks are DASD's
The best way to investigate would be to see `ceph df detail`, to be sure if the metrics are reporting the correct info or not. If the metrics are reporting the correct info, it should be an issue in the ceph configuration. @Abdul. Can you provide that information please?
Installed LSO on vsphere. Raw capacity shows 3.8G occupied althogh fresh install. must gather http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/bz2000006/
Created attachment 1822988 [details] ceph df output
We can able to reproduce this issue on the BM setup (vsphere also).
Raw storage is showing 3.8G RAW STORAGE: CLASS SIZE AVAIL USED RAW USED %RAW USED hdd 300 GiB 296 GiB 840 MiB 3.8 GiB 1.27 TOTAL 300 GiB 296 GiB 840 MiB 3.8 GiB 1.27 POOLS: POOL ID STORED OBJECTS USED %USED MAX AVAIL ocs-storagecluster-cephblockpool 1 256 MiB 305 809 MiB 0.31 84 GiB ocs-storagecluster-cephobjectstore.rgw.control 2 0 B 8 0 B 0 84 GiB ocs-storagecluster-cephfilesystem-metadata 3 53 KiB 24 1.5 MiB 0 84 GiB ocs-storagecluster-cephfilesystem-data0 4 158 B 1 192 KiB 0 84 GiB ocs-storagecluster-cephobjectstore.rgw.meta 5 3.3 KiB 14 2.2 MiB 0 84 GiB ocs-storagecluster-cephobjectstore.rgw.log 6 3.6 KiB 181 6.6 MiB 0 84 GiB ocs-storagecluster-cephobjectstore.rgw.buckets.index 7 0 B 22 0 B 0 84 GiB ocs-storagecluster-cephobjectstore.rgw.buckets.non-ec 8 0 B 0 0 B 0 84 GiB .rgw.root 9 4.8 KiB 16 2.8 MiB 0 84 GiB ocs-storagecluster-cephobjectstore.rgw.buckets.data 10 1.0 KiB 1 256 KiB 0 84 GiB
Hi @asachan, I also have similar output with one DASD on each worker node (3). [root@m1301015 ~]# oc rsh rook-ceph-tools-7f97f8fdb8-h45l8 ceph df RAW STORAGE: CLASS SIZE AVAIL USED RAW USED %RAW USED ssd 291 GiB 288 GiB 191 MiB 3.2 GiB 1.09 TOTAL 291 GiB 288 GiB 191 MiB 3.2 GiB 1.09 POOLS: POOL ID STORED OBJECTS USED %USED MAX AVAIL ocs-storagecluster-cephblockpool 1 63 MiB 69 188 MiB 0.08 81 GiB ocs-storagecluster-cephobjectstore.rgw.control 2 0 B 8 0 B 0 81 GiB ocs-storagecluster-cephfilesystem-metadata 3 2.2 KiB 22 96 KiB 0 81 GiB ocs-storagecluster-cephfilesystem-data0 4 0 B 0 0 B 0 81 GiB ocs-storagecluster-cephobjectstore.rgw.meta 5 3.2 KiB 14 144 KiB 0 81 GiB ocs-storagecluster-cephobjectstore.rgw.log 6 3.5 KiB 180 408 KiB 0 81 GiB ocs-storagecluster-cephobjectstore.rgw.buckets.index 7 0 B 22 0 B 0 81 GiB ocs-storagecluster-cephobjectstore.rgw.buckets.non-ec 8 0 B 0 0 B 0 81 GiB .rgw.root 9 4.8 KiB 16 180 KiB 0 81 GiB ocs-storagecluster-cephobjectstore.rgw.buckets.data 10 1 KiB 1 12 KiB 0 81 GiB [root@m1301015 ~]#
Not sure where the unexpected usage is coming from, someone from core ceph would need to take a look since these are just standard `ceph df` and related commands.
So it sounds like you are seeing this unexpected usage on bare metal (LSO) but not on AWS?
We are seeing it on baremetal where OCS is deployed with LSO.
Neha, PTAL. This is blocking IBM team