Bug 1822134

Summary: [ceph-osd] osd failed to come up(ceph_assert(ondisk_format > 0)) on rhel 7.8 rhcs4 deployment
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: subhash <vpoliset>
Component: Ceph-VolumeAssignee: Andrew Schoen <aschoen>
Status: CLOSED WONTFIX QA Contact: Ameena Suhani S H <amsyedha>
Severity: high Docs Contact:
Priority: high    
Version: 4.0CC: agk, agunn, akupczyk, amsyedha, anharris, aschoen, bhubbard, ceph-eng-bugs, ceph-qe-bugs, dzafman, gmeno, hyelloji, jdurgin, kchai, knortema, msekleta, nojha, prajnoha, rzarzyns, sseshasa, tchandra, vashastr, vereddy, vumrao
Target Milestone: ---Keywords: AutomationBlocker
Target Release: 4.2z1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
.Ceph OSD fails to start because `udev` resets the permissions for BlueStore DB and WAL devices When specifying the BlueStore DB and WAL partitions for an OSD using the `ceph-volume lvm create` command or specifying the partitions, using the `lvm_volume` option with Ceph Ansible can cause those devices to fail on startup. The `udev` subsystem resets the partition permissions back to `root:disk`. To work around this issue, manually start the systemd `ceph-volume` service. For example, to start the OSD with an ID of 8, run the following: `systemctl start 'ceph-volume@lvm-8-*'`. You can also use the `service` command, for example: `service ceph-volume@lvm-8-4c6ddc44-9037-477d-903c-63b5a789ade5 start`. Manually starting the OSD results in the partition having the correct permission, `ceph:ceph`.
Story Points: ---
Clone Of:
: 1831105 (view as bug list) Environment:
Last Closed: 2021-01-27 09:05:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1816167, 1831105    

Comment 17 Josh Durgin 2020-04-22 15:55:10 UTC
Moving back to ceph-volume since that's where the permissions are set

Comment 33 Andrew Schoen 2020-05-04 16:23:22 UTC
Moving this to 4.1z1. A workaround for 4.1 will be documented with https://bugzilla.redhat.com/show_bug.cgi?id=1831105

Comment 63 Veera Raghava Reddy 2021-01-27 09:05:55 UTC
This BZ is intermittently occurring only in ceph-ci for baremetal deployments. Not reproducible in setups outside ceph-ci. Multipe attempts explored by Dev and QE to find a consistent reproduce without much success. Will close the current BZ. If occurring again will open new BZ.

Comment 64 Red Hat Bugzilla 2023-09-15 00:30:54 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days