Bug 1850036 - Ceph RGW remains unavailable after load AVG goes below threshold
Summary: Ceph RGW remains unavailable after load AVG goes below threshold
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-tripleo
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z2
: 16.1 (Train on RHEL 8.2)
Assignee: Giulio Fidente
QA Contact: Itzik Brown
URL:
Whiteboard:
: 1886852 (view as bug list)
Depends On:
Blocks: 1815662
TreeView+ depends on / blocked
 
Reported: 2020-06-23 12:39 UTC by Itzik Brown
Modified: 2024-03-25 16:04 UTC (History)
24 users (show)

Fixed In Version: puppet-tripleo-11.5.0-0.20200707193424.fe9ae10.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-28 15:38:11 UTC
Target Upstream Version:
Embargoed:
mkogan: needinfo-
mkogan: needinfo-
mkogan: needinfo-


Attachments (Terms of Use)
Ceph rgw container log (1.20 MB, text/plain)
2020-06-23 13:35 UTC, Itzik Brown
no flags Details
ceph.conf from rgw container (1.79 KB, text/plain)
2020-06-23 13:37 UTC, Itzik Brown
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github ceph ceph pull 35904 0 None closed rgw: fix boost::asio::async_write() does not return error... 2021-01-04 07:18:57 UTC
OpenStack gerrit 738600 0 None MERGED Switch RGW HAProxy healthcheck to use special RGW healthcheck url 2021-01-04 07:18:57 UTC
OpenStack gerrit 739677 0 None MERGED Switch RGW HAProxy healthcheck to use special RGW healthcheck url 2021-01-04 07:19:34 UTC
Red Hat Issue Tracker OSP-30830 0 None None None 2023-12-15 18:16:08 UTC
Red Hat Product Errata RHEA-2020:4284 0 None None None 2020-10-28 15:38:31 UTC

Description Itzik Brown 2020-06-23 12:39:47 UTC
Description of problem:
After install and uninstall Openshift on Openstack when using Ceph+RGW I get the following:

$ swift stat
 Account HEAD failed: https://10.46.43.200:13808/swift/v1/AUTH_912ae5c5c6904c4996e3eef306b076ce 503 Service Unavailable

After a while running 
$ sudo systemctl restart ceph-radosgw.rgw0.service
And the service is available


Version-Release number of selected component (if applicable):
RHOS-16.1-RHEL-8-20200616.n.0

How reproducible:
First time I see it

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Itzik Brown 2020-06-23 12:45:00 UTC
ceph-radosgw-14.2.8-59.el8cp.x86_64
ceph radows gw image 4-27

Comment 2 Giulio Fidente 2020-06-23 12:56:49 UTC
"podman logs" in debug 20 mode seem to point to err_no=2218 but it's unclear why

2020-06-23 09:11:59.661 7fe7087d7700 10 req 118010 0.000s s3:list_buckets scheduling with dmclock client=3 cost=1
2020-06-23 09:11:59.661 7fe7087d7700  0 req 118010 0.000s s3:list_buckets Scheduling request failed with -2218
2020-06-23 09:11:59.661 7fe7087d7700 20 op->ERRORHANDLER: err_no=-2218 new_err_no=-2218
2020-06-23 09:11:59.662 7fe7087d7700  2 req 118010 0.001s s3:list_buckets op status=0
2020-06-23 09:11:59.662 7fe7087d7700  2 req 118010 0.001s s3:list_buckets http status=503
2020-06-23 09:11:59.662 7fe7087d7700  1 ====== req done req=0x7fe6d0529890 op status=0 http_status=503 latency=0.001s

Comment 3 Itzik Brown 2020-06-23 13:35:55 UTC
Created attachment 1698458 [details]
Ceph rgw container log

Comment 4 Itzik Brown 2020-06-23 13:37:45 UTC
Created attachment 1698459 [details]
ceph.conf from rgw container

Comment 36 Itzik Brown 2020-10-11 00:44:37 UTC
Done the same scenario as in the bug description, bug was not reproduced.
Checked with RHOS-16.1-RHEL-8-20201007.n.0-

Comment 37 John Fulton 2020-10-12 15:22:21 UTC
*** Bug 1886852 has been marked as a duplicate of this bug. ***

Comment 41 errata-xmlrpc 2020-10-28 15:38:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4284


Note You need to log in before you can comment on or make changes to this bug.