Bug 1944933
| Summary: | [RBD] ceph-immutable-object-cache daemon fails to start | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Harish Munjulur <hmunjulu> |
| Component: | RBD | Assignee: | Ilya Dryomov <idryomov> |
| Status: | CLOSED ERRATA | QA Contact: | Harish Munjulur <hmunjulu> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 5.0 | CC: | bniver, ceph-eng-bugs, kdreyer, rmandyam, sostapov |
| Target Milestone: | --- | ||
| Target Release: | 5.0 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | ceph-16.2.0-8.el8cp | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-08-30 08:29:31 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1929673 | ||
This looks like it is cephx related Harish what document did you follow? Can we edit it to correct the steps, or make this easier? Hi Ilya, as you suggested I ran the below command to change it to ceph user. But I see the below error. Wanted to check if I need to create a ceph user? [root@magna007 ~]# sudo chown ceph:ceph /etc/ceph/ceph.client.ceph-immutable-object-cache.readonlyc.keyring chown: invalid user: ‘ceph:ceph’ The "ceph-common" package creates the ceph user and group with a static UID 167. I guess ceph-immutable-object-cache does not require that package. Should it? Verified
Steps:
1. # yum install ceph-immutable-object-cache
2. # ceph auth get-or-create client.ceph-immutable-object-cache.readonlycache mon 'profile rbd' osd 'profile rbd-read-only'
Key will be generated and this key needs to be added to /etc/ceph/ceph.client.admin.keyring or create a file /etc/ceph/ceph.client.ceph-immutable-object-cache.readonlycache.keyring
3. # systemctl enable ceph-immutable-object-cache
4. # systemctl start ceph-immutable-object-cache
5. # systemctl status ceph-immutable-object-cache
[root@ceph-hmunjulu-1618804778222-node1-mon-mgr-installer-node-export ceph]# systemctl status ceph-immutable-object-cache
● ceph-immutable-object-cache.service - Ceph immutable object cache daemon
Loaded: loaded (/usr/lib/systemd/system/ceph-immutable-object-cache@.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2021-04-19 02:06:03 EDT; 3s ago
Main PID: 39330 (ceph-immutable-)
Tasks: 15 (limit: 23505)
Memory: 7.0M
CGroup: /system.slice/system-ceph\x2dimmutable\x2dobject\x2dcache.slice/ceph-immutable-object-cache.service
└─39330 /usr/bin/ceph-immutable-object-cache -f --cluster ceph --id ceph-immutable-object-cache.readonlyc --setuser ceph --setgroup ceph
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat Ceph Storage 5.0 bug fix and enhancement), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:3294 |
Description of problem: ceph-immutable-object-cache daemon fails to start Version-Release number of selected component (if applicable): ceph version 16.1.0-1323.el8cp (46ac37397f0332c20aceceb8022a1ac1ddf8fa73) pacific (rc) Steps to Reproduce: 1. # yum install ceph-immutable-object-cache 2. # ceph auth get-or-create client.ceph-immutable-object-cache.readonlycache mon 'allow r' osd 'profile rbd-read-only' 3. # systemctl enable ceph-immutable-object-cache 4. # systemctl status ceph-immutable-object-cache 5. #ceph auth caps client.ceph-immutable-object-cache.readonlyc mon 'profile rbd' osd 'profile rbd-read-only' - (Jason suggested to make this change) Actual results: [root@ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph]# systemctl status ceph-immutable-object-cache ● ceph-immutable-object-cache.service - Ceph immutable object cache daemon Loaded: loaded (/usr/lib/systemd/system/ceph-immutable-object-cache@.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2021-03-30 21:14:43 EDT; 23s ago Main PID: 313810 (code=exited, status=1/FAILURE) Expected results: Active: active (running) Additional info: journalctl -u ceph-immutable-object-cache.service Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export systemd[1]: Started Ceph immutable object cache daemon. Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph-immutable-object-cache[312325]: 2021-03-30T21:01:47.331-0400 7f359d443540 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.ceph-immutable-object-cache.readonlyc.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/k> Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph-immutable-object-cache[312325]: 2021-03-30T21:01:47.331-0400 7f359d443540 -1 AuthRegistry(0x561c55424620) no keyring found at /etc/ceph/ceph.client.ceph-immutable-object-cache.readonlyc.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyr> Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph-immutable-object-cache[312325]: 2021-03-30T21:01:47.332-0400 7f359d443540 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.ceph-immutable-object-cache.readonlyc.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/k> Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph-immutable-object-cache[312325]: 2021-03-30T21:01:47.332-0400 7f359d443540 -1 AuthRegistry(0x7ffeda3c09f0) no keyring found at /etc/ceph/ceph.client.ceph-immutable-object-cache.readonlyc.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyr> Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph-immutable-object-cache[312325]: 2021-03-30T21:01:47.334-0400 7f358ce56700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1] Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph-immutable-object-cache[312325]: 2021-03-30T21:01:47.334-0400 7f359d443540 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph-immutable-object-cache[312325]: failed to fetch mon config (--no-mon-config to skip) Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export systemd[1]: ceph-immutable-object-cache.service: Main process exited, code=exited, status=1/FAILURE Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export systemd[1]: ceph-immutable-object-cache.service: Failed with result 'exit-code'. Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export systemd[1]: ceph-immutable-object-cache.service: Service RestartSec=100ms expired, scheduling restart. Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export systemd[1]: ceph-immutable-object-cache.service: Scheduled restart job, restart counter is at 2. Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export systemd[1]: Stopped Ceph immutable object cache daemon. Mar 30 21:01:47 ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export systemd[1]: Started Ceph immutable object cache daemon. [root@ceph-hmunjulu-1617041019750-node1-mon-mgr-installer-node-export ceph]# ls ceph.client.admin.keyring ceph.client.ceph-immutable-object-cache.readonlyc.keyring ceph.conf ceph.keyring ceph.pub podman-auth.json rbdmap vi ceph.keyring [client.admin] key = AQBEGGJgkoVCABAAL27uBeOWCex39ngjkNTGMg== [client.ceph-immutable-object-cache.readonlyc] key = AQA9yWNgwKNFGRAA+u++ya2flMvfbqvT17LC/Q==