Description of problem: `ceph-bluestore-tool bluefs-bdev-expand --path /var/lib/ceph/osd/ceph-0` fails to read the bluestore type Error: `inferring bluefs devices from bluestore path expected bluestore, but type is` Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1.Deploy rook using downstream ceph (commit: b53d9dfff6b1021fbab8e28f2b873d7d49cf5665) 2.Observe the osd creation flow Actual results:The init container for `expand-bluefs` fails sh-5.1# ceph-bluestore-tool bluefs-bdev-expand --path /var/lib/ceph/osd/ceph-0 --log-level 30 inferring bluefs devices from bluestore path expected bluestore, but type is sh-5.1# Expected results: The command should not fail. Additional info: BZ
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat Ceph Storage 6.1 Bug Fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:4473
`ceph osd metadata osd.0` command still shows `bluestore` even after OSD is migrated to use `bluestore-rdr` Before upgrade: sh-4.4$ ceph osd metadata osd.0 { "id": 0, "arch": "x86_64", "back_addr": "[v2:10.244.1.9:6802/2446454422,v1:10.244.1.9:6803/2446454422]", "back_iface": "", "bluefs": "1", "bluefs_dedicated_db": "0", "bluefs_dedicated_wal": "0", "bluefs_single_shared_device": "1", "bluestore_bdev_access_mode": "blk", "bluestore_bdev_block_size": "4096", "bluestore_bdev_dev_node": "/dev/dm-0", "bluestore_bdev_devices": "vdc", "bluestore_bdev_driver": "KernelDevice", "bluestore_bdev_optimal_io_size": "0", "bluestore_bdev_partition_path": "/dev/dm-0", "bluestore_bdev_rotational": "1", "bluestore_bdev_size": "10720641024", "bluestore_bdev_support_discard": "1", "bluestore_bdev_type": "hdd", "bluestore_min_alloc_size": "4096", "ceph_release": "quincy", "ceph_version": "ceph version 17.2.6-373-g5406f4b1 (5406f4b1729bf4f476338ad344cc86b126e0c5b3) quincy (stable)", "ceph_version_short": "17.2.6-373-g5406f4b1", "ceph_version_when_created": "ceph version 17.2.6-373-g5406f4b1 (5406f4b1729bf4f476338ad344cc86b126e0c5b3) quincy (stable)", "container_hostname": "rook-ceph-osd-0-6466b5bcb5-d8lmm", "container_image": "quay.io/guits/ceph-volume:bs-rdr", "cpu": "Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz", "created_at": "2023-08-25T15:08:41.150964Z", "default_device_class": "hdd", "device_ids": "", "device_paths": "", "devices": "vdc", "distro": "centos", "distro_description": "CentOS Stream 8", "distro_version": "8", "front_addr": "[v2:10.244.1.9:6800/2446454422,v1:10.244.1.9:6801/2446454422]", "front_iface": "", "hb_back_addr": "[v2:10.244.1.9:6806/2446454422,v1:10.244.1.9:6807/2446454422]", "hb_front_addr": "[v2:10.244.1.9:6804/2446454422,v1:10.244.1.9:6805/2446454422]", "hostname": "minikube-m02", "journal_rotational": "1", "kernel_description": "#1 SMP Fri Jan 27 18:05:35 UTC 2023", "kernel_version": "5.10.57", "mem_swap_kb": "0", "mem_total_kb": "2165900", "network_numa_unknown_ifaces": "back_iface,front_iface", "objectstore_numa_unknown_devices": "vdc", "os": "Linux", "osd_data": "/var/lib/ceph/osd/ceph-0", "osd_objectstore": "bluestore", "osdspec_affinity": "", "pod_name": "rook-ceph-osd-0-6466b5bcb5-d8lmm", "pod_namespace": "rook-ceph", "rotational": "1" } sh-4.4$ sh-4.4# ceph-bluestore-tool show-label --path /var/lib/ceph/osd/ceph-0 inferring bluefs devices from bluestore path { "/var/lib/ceph/osd/ceph-0/block": { "osd_uuid": "7add3b6c-b8e9-42b0-8dc4-feb1af6e5394", "size": 10720641024, "btime": "2023-08-25T15:08:38.966496+0000", "description": "main", "bfm_blocks": "2617344", "bfm_blocks_per_key": "128", "bfm_bytes_per_block": "4096", "bfm_size": "10720641024", "bluefs": "1", "ceph_fsid": "e0c985eb-b626-4807-9681-b7a836209793", "ceph_version_when_created": "ceph version 17.2.6-373-g5406f4b1 (5406f4b1729bf4f476338ad344cc86b126e0c5b3) quincy (stable)", "created_at": "2023-08-25T15:08:41.150964Z", "kv_backend": "rocksdb", "magic": "ceph osd volume v026", "mkfs_done": "yes", "osd_key": "AQDqw+hkvhDHKhAA5r+WYvA++VOyplGBHynBOg==", "ready": "ready", "require_osd_release": "17", "type": "bluestore", "whoami": "0" } } sh-4.4# cat /var/lib/ceph/osd/ceph-0/type bluestore ----------------------------------------------------------------------------------- After Upgrade: sh-4.4$ ceph osd metadata osd.0 { "id": 0, "arch": "x86_64", "back_addr": "[v2:10.244.1.14:6802/2651410559,v1:10.244.1.14:6803/2651410559]", "back_iface": "", "bluefs": "1", "bluefs_dedicated_db": "0", "bluefs_dedicated_wal": "0", "bluefs_single_shared_device": "1", "bluestore_bdev_access_mode": "blk", "bluestore_bdev_block_size": "4096", "bluestore_bdev_dev_node": "/dev/dm-0", "bluestore_bdev_devices": "vdc", "bluestore_bdev_driver": "KernelDevice", "bluestore_bdev_optimal_io_size": "0", "bluestore_bdev_partition_path": "/dev/dm-0", "bluestore_bdev_rotational": "1", "bluestore_bdev_size": "10720641024", "bluestore_bdev_support_discard": "1", "bluestore_bdev_type": "hdd", "bluestore_min_alloc_size": "4096", "ceph_release": "quincy", "ceph_version": "ceph version 17.2.6-373-g5406f4b1 (5406f4b1729bf4f476338ad344cc86b126e0c5b3) quincy (stable)", "ceph_version_short": "17.2.6-373-g5406f4b1", "ceph_version_when_created": "ceph version 17.2.6-373-g5406f4b1 (5406f4b1729bf4f476338ad344cc86b126e0c5b3) quincy (stable)", "container_hostname": "rook-ceph-osd-0-85db7bfdd8-vxq65", "container_image": "quay.io/guits/ceph-volume:bs-rdr", "cpu": "Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz", "created_at": "2023-08-25T15:15:51.148428Z", "default_device_class": "hdd", "device_ids": "", "device_paths": "", "devices": "vdc", "distro": "centos", "distro_description": "CentOS Stream 8", "distro_version": "8", "front_addr": "[v2:10.244.1.14:6800/2651410559,v1:10.244.1.14:6801/2651410559]", "front_iface": "", "hb_back_addr": "[v2:10.244.1.14:6806/2651410559,v1:10.244.1.14:6807/2651410559]", "hb_front_addr": "[v2:10.244.1.14:6804/2651410559,v1:10.244.1.14:6805/2651410559]", "hostname": "minikube-m02", "journal_rotational": "1", "kernel_description": "#1 SMP Fri Jan 27 18:05:35 UTC 2023", "kernel_version": "5.10.57", "mem_swap_kb": "0", "mem_total_kb": "2165900", "network_numa_unknown_ifaces": "back_iface,front_iface", "objectstore_numa_unknown_devices": "vdc", "os": "Linux", "osd_data": "/var/lib/ceph/osd/ceph-0", "osd_objectstore": "bluestore", "osdspec_affinity": "", "pod_name": "rook-ceph-osd-0-85db7bfdd8-vxq65", "pod_namespace": "rook-ceph", "rotational": "1" } sh-4.4$ sh-4.4# cat /var/lib/ceph/osd/ceph-0/type bluestore-rdr sh-4.4# ceph-bluestore-tool show-label --path /var/lib/ceph/osd/ceph-0 inferring bluefs devices from bluestore path { "/var/lib/ceph/osd/ceph-0/block": { "osd_uuid": "16f9170b-8ef0-4287-96ed-60f745781128", "size": 10720641024, "btime": "2023-08-25T15:15:49.513276+0000", "description": "main", "bfm_blocks": "2617344", "bfm_blocks_per_key": "128", "bfm_bytes_per_block": "4096", "bfm_size": "10720641024", "bluefs": "1", "ceph_fsid": "e0c985eb-b626-4807-9681-b7a836209793", "ceph_version_when_created": "ceph version 17.2.6-373-g5406f4b1 (5406f4b1729bf4f476338ad344cc86b126e0c5b3) quincy (stable)", "created_at": "2023-08-25T15:15:51.148428Z", "kv_backend": "rocksdb", "magic": "ceph osd volume v026", "mkfs_done": "yes", "osd_key": "AQCVxehkJ+3qNhAAHf5zZWPKGZ6XMlm8SI6saw==", "ready": "ready", "require_osd_release": "17", "type": "bluestore-rdr", "whoami": "0" } } sh-4.4#
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days