Description of problem: RGW failes to restart , in the scenario where we are trying to setup multiple replicated realms. Version-Release number of selected component (if applicable): ceph version 14.2.11-49.el7cp ansible-2.9.14-1.el7ae.noarch ceph-ansible-4.0.34-1.el7cp.noarch How reproducible: RUNNING HANDLER [ceph-handler : restart ceph rgw daemon(s)] ************************************************************************************************** Thursday 15 October 2020 02:58:27 -0400 (0:00:00.997) 0:05:04.726 ****** skipping: [ceph-tejas-1602577535750-node2-monrgwosd] => (item=ceph-tejas-1602577535750-node4-rgw) failed: [ceph-tejas-1602577535750-node2-monrgwosd -> ceph-tejas-1602577535750-node3-monrgwosd] (item=ceph-tejas-1602577535750-node3-monrgwosd) => changed=true ansible_loop_var: item cmd: - /usr/bin/env - bash - /tmp/restart_rgw_daemon.sh delta: '0:01:40.472566' end: '2020-10-15 03:00:15.202965' item: ceph-tejas-1602577535750-node3-monrgwosd msg: non-zero return code rc: 1 start: '2020-10-15 02:58:34.730399' stderr: '' stderr_lines: <omitted> stdout: |- Socket file could not be found, which means Rados Gateway is not running. Showing ceph-rgw unit logs now: Oct 15 02:58:34 ceph-tejas-1602577535750-node3-monrgwosd systemd[1]: Stopping Ceph rados gateway... Oct 15 02:58:34 ceph-tejas-1602577535750-node3-monrgwosd radosgw[14486]: 2020-10-15 02:58:34.746 7feb29b07700 -1 received signal: Terminated from /usr/lib/systemd/systemd --system --deserialize 17 (PID: 1) UID: 0 Oct 15 02:58:34 ceph-tejas-1602577535750-node3-monrgwosd radosgw[14486]: 2020-10-15 02:58:34.747 7feb3efd6900 -1 shutting down Oct 15 02:58:35 ceph-tejas-1602577535750-node3-monrgwosd systemd[1]: Stopped Ceph rados gateway. Oct 15 02:58:35 ceph-tejas-1602577535750-node3-monrgwosd systemd[1]: Started Ceph rados gateway. Oct 15 02:58:35 ceph-tejas-1602577535750-node3-monrgwosd radosgw[34532]: 2020-10-15 02:58:35.364 7f08ebc70900 -1 Couldn't init storage provider (RADOS) Oct 15 02:58:35 ceph-tejas-1602577535750-node3-monrgwosd systemd[1]: ceph-radosgw.rgw0.service: main process exited, code=exited, status=5/NOTINSTALLED Oct 15 02:58:35 ceph-tejas-1602577535750-node3-monrgwosd systemd[1]: Unit ceph-radosgw.rgw0.service entered failed state. Oct 15 02:58:35 ceph-tejas-1602577535750-node3-monrgwosd systemd[1]: ceph-radosgw.rgw0.service failed. Oct 15 02:58:35 ceph-tejas-1602577535750-node3-monrgwosd systemd[1]: ceph-radosgw.rgw0.service holdoff time over, scheduling restart. Checking the ceph.conf : [client.rgw.ceph-tejas-1602577535750-node2-monrgwosd.rgw0] host = ceph-tejas-1602577535750-node2-monrgwosd keyring = /var/lib/ceph/radosgw/ceph-rgw.ceph-tejas-1602577535750-node2-monrgwosd.rgw0/keyring log file = /var/log/ceph/ceph-rgw-ceph-tejas-1602577535750-node2-monrgwosd.rgw0.log rgw frontends = beast endpoint=10.0.102.58:8080 rgw thread pool size = 512 rgw_realm = france rgw_zone = paris rgw_zonegroup = idf But no realm created yet [root@ceph-tejas-1602577535750-node2-monrgwosd cephuser]# radosgw-admin realm list { "default_info": "", "realms": [] } After commenting out the lines rgw_realm = france rgw_zone = paris rgw_zonegroup = idf ]# systemctl status ceph-radosgw.rgw0 ● ceph-radosgw.rgw0.service - Ceph rados gateway Loaded: loaded (/usr/lib/systemd/system/ceph-radosgw@.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2020-10-15 04:01:05 EDT; 4s ago Main PID: 33128 (radosgw) CGroup: /system.slice/system-ceph\x2dradosgw.slice/ceph-radosgw.rgw0.service └─33128 /usr/bin/radosgw -f --cluster ceph --name client.rgw.ceph-tejas-1602577535750-node2-monrgwosd.rgw0 --setuser ceph --setgroup ceph
*** Bug 1917144 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat Ceph Storage security, bug fix, and enhancement Update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:1452