Bug 1388647
| Summary: | [1.3.3/0.94.9-5.el7cp radosgw-admin fails during purgedata | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Vasu Kulkarni <vakulkar> |
| Component: | RGW | Assignee: | Orit Wasserman <owasserm> |
| Status: | CLOSED ERRATA | QA Contact: | ceph-qe-bugs <ceph-qe-bugs> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 1.3.3 | CC: | cbodley, ceph-eng-bugs, kbader, kdreyer, mbenjamin, owasserm, sweil, tserlin |
| Target Milestone: | rc | ||
| Target Release: | 1.3.4 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | RHEL: ceph-0.94.9-8.el7cp Ubuntu: ceph_0.94.9-9redhat1trusty | Doc Type: | Bug Fix |
| Doc Text: |
After backporting certain upstream changes, an attempt to delete objects with underscore characters failed. The change causing this bug has been removed, and objects can now be deleted as expected.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-12-01 21:39:09 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Vasu Kulkarni
2016-10-25 20:00:10 UTC
It seems like the test called "user rm" command twice and the second call returned ENOENT because the user was already removed. Orit, the code hasn't changed and it worked before, but I do see the issue in the code since the config should never had second client https://github.com/ceph/ceph-qa-suite/blob/hammer/tasks/s3tests.py#L216-L230 Do you think ENOENT is valid now and based on hardware speed? This is the first time I see this problem I just see the double logging. I am not sure what happened as the configuration seems fine. We don't have any radosgw logs (the log level is too low) so we are not sure there is no other level. Can you try to run again with higher log level for radosgw? I looked at it more closely, if you look at the second command the user is 'foo' and in the first one its 'bar', so its failing for 'foo', I will try to give the system in same state. This is what i see in logs, one of the bucket delete returns -2
2016-10-27 18:29:47.929434 7ffa7a7fc700 1 ====== req done req=0x7ffa6400d490 http_status=200 ======
2016-10-27 18:29:47.929467 7ffa7a7fc700 1 civetweb: 0x7ffa640008c0: 127.0.0.1 - - [27/Oct/2016:18:29:47 +0000] "GET /test-client.0-imqaibnu3v3kufj-/ HTTP/1.1" 200 0 - Boto/2.43.0 Python/2.7.5 Linux/3.10.0-327.36.3.el7.x86_64
2016-10-27 18:29:47.932001 7ff9d7fff700 1 ====== starting new request req=0x7ff9c8031bb0 =====
2016-10-27 18:29:48.006921 7ff9d7fff700 1 ====== req done req=0x7ff9c8031bb0 http_status=204 ======
2016-10-27 18:29:48.006963 7ff9d7fff700 1 civetweb: 0x7ff9c80008c0: 127.0.0.1 - - [27/Oct/2016:18:29:47 +0000] "DELETE /test-client.0-imqaibnu3v3kufj-/foo HTTP/1.1" 204 0 - Boto/2.43.0 Python/2.7.5 Linux/3.10.0-327.36.3.el7.x86_64
2016-10-27 18:29:48.008890 7ff9f4ff9700 1 ====== starting new request req=0x7ff9d000d980 =====
2016-10-27 18:29:48.491281 7ff9f4ff9700 1 ====== req done req=0x7ff9d000d980 http_status=204 ======
2016-10-27 18:29:48.491317 7ff9f4ff9700 1 civetweb: 0x7ff9d00008c0: 127.0.0.1 - - [27/Oct/2016:18:29:48 +0000] "DELETE /test-client.0-imqaibnu3v3kufj-/ HTTP/1.1" 204 0 - Boto/2.43.0 Python/2.7.5 Linux/3.10.0-327.36.3.el7.x86_64
2016-10-27 18:32:35.567047 7ffafabf9700 0 ERROR: rgw_bucket_sync_user_stats() for user=foo.client.0, bucket=test-client.0-imqaibnu3v3kufj-(@{i=.rgw.buckets.index,e=.rgw.buckets.extra}.rgw.buckets[default.4128.196]) returned -2
2016-10-27 18:32:35.567063 7ffafabf9700 0 WARNING: sync_bucket() returned r=-2
rerun with higher log level http://magna002.ceph.redhat.com/vasu-2016-10-28_14:54:15-smoke-hammer---basic-pluto/258439/teuthology.log This is the error:
2016-10-28 19:02:32.534216 7f1750ff9700 20 cls_bucket_header() returned -2
2016-10-28 19:02:32.534219 7f1750ff9700 0 ERROR: rgw_bucket_sync_user_stats() for user=foo.client.0, bucket=test-client.0-84e5wlt0fkx0mg4-172(@{i=.rgw.buckets.index,e=.rgw.buckets.extra}.rgw.buckets[default.4121.175]) returned -2
2016-10-28 19:02:32.534225 7f1750ff9700 0 WARNING: sync_bucket() returned r=-2
we don't have cls logs.
I will try to reproduce it locally.
Verified in v0.94.9-8 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2847.html |