Bug 2248862 - [rgw][s3select]: rgw going down executing query "select trim( leading 132140533849470 from _3 ) from s3object;"
Summary: [rgw][s3select]: rgw going down executing query "select trim( leading 1321405...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 7.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 7.0
Assignee: gal salomon
QA Contact: Hemanth Sai
Rivka Pollack
URL:
Whiteboard:
Depends On:
Blocks: 2237662
TreeView+ depends on / blocked
 
Reported: 2023-11-09 12:05 UTC by Hemanth Sai
Modified: 2024-03-28 05:32 UTC (History)
5 users (show)

Fixed In Version: ceph-18.2.0-127.el9cp
Doc Type: Bug Fix
Doc Text:
.Ceph Object Gateway daemon no longer crashes with the trim command Previously, due to the trim skip type checking within the query for `select trim(LEADING '1' from '111abcdef111') from s3object;`, the Ceph Object Gateway daemon would crash. With this fix, the type is checked and is identified if wrong and reported, no longer causing the daemon to crash.
Clone Of:
Environment:
Last Closed: 2023-12-13 15:24:55 UTC
Embargoed:
gsalomon: needinfo-
gsalomon: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-7881 0 None None None 2023-11-09 12:05:53 UTC
Red Hat Product Errata RHBA-2023:7780 0 None None None 2023-12-13 15:24:59 UTC

Description Hemanth Sai 2023-11-09 12:05:33 UTC
Description of problem:
rgw going down executing trim command with integer as "remove chars"

[cephuser@extensa022 ~]$ venv/bin/aws s3api --endpoint-url http://extensa027.ceph.redhat.com:80 select-object-content  --bucket bkt1 --key polarion_data.csv --expression-type 'SQL' --input-serialization '{"CSV": {}, "CompressionType": "NONE"}' --output-serialization '{"CSV": {}}' --expression "select trim( leading 132140533849470 from _3 ) from s3object;" /dev/stdout

Could not connect to the endpoint URL: "http://extensa027.ceph.redhat.com:80/bkt1/polarion_data.csv?select&select-type=2"
[cephuser@extensa022 ~]$


crash info:

     0> 2023-11-09T11:44:01.109+0000 7f941d32b640 -1 *** Caught signal (Segmentation fault) **
 in thread 7f941d32b640 thread_name:radosgw

 ceph version 18.2.0-116.el9cp (6c6dfc1f2b2ce1896bf2696daea444dff96645af) reef (stable)
 1: /lib64/libc.so.6(+0x54df0) [0x7f94b1050df0]
 2: /lib64/libc.so.6(+0xc56fd) [0x7f94b10c16fd]
 3: /usr/bin/radosgw(+0x6326ec) [0x558c60cf56ec]
 4: /usr/bin/radosgw(+0x637272) [0x558c60cfa272]
 5: /usr/bin/radosgw(+0xbba3b9) [0x558c6127d3b9]
 6: /usr/bin/radosgw(+0x644d9c) [0x558c60d07d9c]
 7: (RGWSelectObj_ObjStore_S3::run_s3select_on_csv(char const*, char const*, unsigned long)+0x8cb) [0x558c60d1483b]
 8: (RGWSelectObj_ObjStore_S3::csv_processing(ceph::buffer::v15_2_0::list&, long, long)+0x507) [0x558c60d18377]
 9: (RGWGetObj_BlockDecrypt::process(ceph::buffer::v15_2_0::list&, unsigned long, unsigned long)+0x9a) [0x558c60d4bf7a]
 10: (RGWGetObj_BlockDecrypt::handle_data(ceph::buffer::v15_2_0::list&, long, long)+0x1ae) [0x558c60d53dde]
 11: (RGWRados::get_obj_iterate_cb(DoutPrefixProvider const*, rgw_raw_obj const&, long, long, long, bool, RGWObjState*, void*)+0x11a) [0x558c60e0e7fa]
 12: /usr/bin/radosgw(+0x747846) [0x558c60e0a846]
 13: (RGWRados::iterate_obj(DoutPrefixProvider const*, RGWObjectCtx&, RGWBucketInfo&, rgw_obj const&, long, long, unsigned long, int (*)(DoutPrefixProvider const*, rgw_raw_obj const&, long, long, long, bool, RGWObjState*, void*), void*, optional_yield)+0x428) [0x558c60e0f0f8]
 14: (RGWRados::Object::Read::iterate(DoutPrefixProvider const*, long, long, RGWGetDataCB*, optional_yield)+0x134) [0x558c60e0f8b4]
 15: (RGWGetObj::execute(optional_yield)+0x122c) [0x558c60c304bc]
 16: (RGWSelectObj_ObjStore_S3::execute(optional_yield)+0xc1) [0x558c60d1ac41]
 17: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, rgw::sal::Driver*, bool)+0xa72) [0x558c60ae4c52]
 18: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0x1039) [0x558c60ae64d9]
 19: /usr/bin/radosgw(+0xb6ddf6) [0x558c61230df6]
 20: /usr/bin/radosgw(+0x37bc51) [0x558c60a3ec51]
 21: make_fcontext()
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.



similar commands which caused rgw down:

executing query-2335: select cast( trim( trailing 205979192405432 from 'kQXLXjV283jqvsSVtUQj' ) as string) from s3object;

executing query-3754: select sum( trim( both 112839306443463 from '2g rL44I0tRJi1tLS1WE' ) ) from s3object;

executing query-4620: select trim( both min( 153401582910812 ) from min( 'Dwj5M28JLOnd2dxm6V0n' ) ) from s3object;

executing query-5605: select extract( year from trim( trailing 172215640942740 from 'NfEZRuJaDY87tLjrrmVL' ) ) from s3object;

executing query-5933: select extract( day from trim( leading 170191654430978 from 'MCPSK3qVHk9fnZx9x6ld' ) ) from s3object;

executing query-7243: select to_string( trim( both 118424875481762 from 'Hb dGpdmYZ486VGWKekF' ) ,  yy) from s3object;


Version-Release number of selected component (if applicable):
ceph version 18.2.0-116.el9cp

How reproducible:
always

Steps to Reproduce:
1.deploy rhcs7.0 ceph cluster
2.upload an csv object using aws-cli
3.execute the query "select trim( leading 132140533849470 from _3 ) from s3object;"

Actual results:
rgw daemon going down

Expected results:
no rgw crashes should be seen

Additional info:
automation script: https://docs.google.com/document/d/1eg2IbKpGWlGD8dG9PX8oDVKC77xNN-URU-A_r68KUmg/edit?usp=sharing

Comment 2 gal salomon 2023-11-13 12:22:01 UTC
this issue (and others) are resolved https://github.com/ceph/ceph/pull/54298

(the trim is crashing because of a missing type-check.  thus, any combination with trim will cause a crash)

Comment 4 gal salomon 2023-11-15 16:17:37 UTC
the issue is resolved in the following PR
https://github.com/ceph/ceph/pull/54298

Comment 5 gal salomon 2023-11-15 16:17:49 UTC
the issue is resolved in the following PR
https://github.com/ceph/ceph/pull/54298

Comment 11 errata-xmlrpc 2023-12-13 15:24:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 7.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:7780


Note You need to log in before you can comment on or make changes to this bug.