Bug 2248866 - [rgw][s3select]: rgw going down executing query "select cast( trim( leading 132140533849470.72 from _3 ) as float) from s3object;"
Summary: [rgw][s3select]: rgw going down executing query "select cast( trim( leading 1...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 7.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 7.0
Assignee: gal salomon
QA Contact: Hemanth Sai
Rivka Pollack
URL:
Whiteboard:
Depends On:
Blocks: 2237662
TreeView+ depends on / blocked
 
Reported: 2023-11-09 12:23 UTC by Hemanth Sai
Modified: 2024-03-28 05:33 UTC (History)
5 users (show)

Fixed In Version: ceph-18.2.0-127.el9cp
Doc Type: Bug Fix
Doc Text:
.Ceph Object Gateway daemon no longer crashes with `cast( trim)` queries Previously, due to the trim skip type checking within the query for `select cast( trim( leading 132140533849470.72 from _3 ) as float) from s3object;`, the Ceph Object Gateway daemon would crash. With this fix the type is checked and is identified if wrong and reported, no longer causing the daemon to crash.
Clone Of:
Environment:
Last Closed: 2023-12-13 15:24:56 UTC
Embargoed:
gsalomon: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-7882 0 None None None 2023-11-09 12:25:51 UTC
Red Hat Product Errata RHBA-2023:7780 0 None None None 2023-12-13 15:25:00 UTC

Description Hemanth Sai 2023-11-09 12:23:26 UTC
Description of problem:
rgw going down executing trim command with float as "remove chars"

[cephuser@extensa022 ~]$ venv/bin/aws s3api --endpoint-url http://extensa027.ceph.redhat.com:80 select-object-content  --bucket bkt1 --key polarion_data.csv --expression-type 'SQL' --input-serialization '{"CSV": {}, "CompressionType": "NONE"}' --output-serialization '{"CSV": {}}' --expression "select cast( trim( leading 132140533849470.72 from _3 ) as float) from s3object;" /dev/stdout

Could not connect to the endpoint URL: "http://extensa027.ceph.redhat.com:80/bkt1/polarion_data.csv?select&select-type=2"
[cephuser@extensa022 ~]$
[cephuser@extensa022 ~]$ venv/bin/aws s3api --endpoint-url http://extensa027.ceph.redhat.com:80 select-object-content  --bucket bkt1 --key polarion_data.csv --expression-type 'SQL' --input-serialization '{"CSV": {}, "CompressionType": "NONE"}' --output-serialization '{"CSV": {}}' --expression "select trim( leading 132140533849470.72 from _3 ) from s3object;" /dev/stdout

Could not connect to the endpoint URL: "http://extensa027.ceph.redhat.com:80/bkt1/polarion_data.csv?select&select-type=2"
[cephuser@extensa022 ~]$


crash info:

     0> 2023-11-09T10:40:04.546+0000 7f615e4a3640 -1 *** Caught signal (Segmentation fault) **
 in thread 7f615e4a3640 thread_name:radosgw

 ceph version 18.2.0-116.el9cp (6c6dfc1f2b2ce1896bf2696daea444dff96645af) reef (stable)
 1: /lib64/libc.so.6(+0x54df0) [0x7f62762d0df0]
 2: /lib64/libc.so.6(+0xc56fd) [0x7f62763416fd]
 3: /usr/bin/radosgw(+0x6326ec) [0x561dbe33a6ec]
 4: /usr/bin/radosgw(+0x637272) [0x561dbe33f272]
 5: /usr/bin/radosgw(+0xbba3b9) [0x561dbe8c23b9]
 6: /usr/bin/radosgw(+0x644d9c) [0x561dbe34cd9c]
 7: (RGWSelectObj_ObjStore_S3::run_s3select_on_csv(char const*, char const*, unsigned long)+0x8cb) [0x561dbe35983b]
 8: (RGWSelectObj_ObjStore_S3::csv_processing(ceph::buffer::v15_2_0::list&, long, long)+0x507) [0x561dbe35d377]
 9: (RGWGetObj_BlockDecrypt::process(ceph::buffer::v15_2_0::list&, unsigned long, unsigned long)+0x9a) [0x561dbe390f7a]
 10: (RGWGetObj_BlockDecrypt::handle_data(ceph::buffer::v15_2_0::list&, long, long)+0x1ae) [0x561dbe398dde]
 11: (RGWRados::get_obj_iterate_cb(DoutPrefixProvider const*, rgw_raw_obj const&, long, long, long, bool, RGWObjState*, void*)+0x11a) [0x561dbe4537fa]
 12: /usr/bin/radosgw(+0x747846) [0x561dbe44f846]
 13: (RGWRados::iterate_obj(DoutPrefixProvider const*, RGWObjectCtx&, RGWBucketInfo&, rgw_obj const&, long, long, unsigned long, int (*)(DoutPrefixProvider const*, rgw_raw_obj const&, long, long, long, bool, RGWObjState*, void*), void*, optional_yield)+0x428) [0x561dbe4540f8]
 14: (RGWRados::Object::Read::iterate(DoutPrefixProvider const*, long, long, RGWGetDataCB*, optional_yield)+0x134) [0x561dbe4548b4]
 15: (RGWGetObj::execute(optional_yield)+0x122c) [0x561dbe2754bc]
 16: (RGWSelectObj_ObjStore_S3::execute(optional_yield)+0xc1) [0x561dbe35fc41]
 17: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, rgw::sal::Driver*, bool)+0xa72) [0x561dbe129c52]
 18: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0x1039) [0x561dbe12b4d9]
 19: /usr/bin/radosgw(+0xb6ddf6) [0x561dbe875df6]
 20: /usr/bin/radosgw(+0x37bc51) [0x561dbe083c51]
 21: make_fcontext()
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.



similar commands which caused rgw down:

executing query-3099: select to_timestamp( trim( leading 196705646675451.66 from 'SVLKnNkQNFy1zHQwz9kB' ) ) from s3object;

executing query-4298: select trim( trim( leading 127156843458650.06 from 'QKByMV0xRIcuHiwx6CjR' ) ) from s3object;

executing query-7460: select to_string( trim( leading 200572764796854.0 from 'xnYhYsYc6DNTf817sOtH' ) ,  yyyy) from s3object;



Version-Release number of selected component (if applicable):
ceph version 18.2.0-116.el9cp

How reproducible:
always

Steps to Reproduce:
1.deploy rhcs7.0 ceph cluster
2.upload an csv object using aws-cli
3.execute the query "select trim( leading 132140533849470.72 from _3 ) from s3object;"

Actual results:
rgw daemon going down

Expected results:
no rgw crashes should be seen

Additional info:
automation script: https://docs.google.com/document/d/1eg2IbKpGWlGD8dG9PX8oDVKC77xNN-URU-A_r68KUmg/edit?usp=sharing

Comment 2 gal salomon 2023-11-13 12:17:09 UTC
this issue (and others)resolve in https://github.com/ceph/ceph/pull/54298

Comment 7 errata-xmlrpc 2023-12-13 15:24:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 7.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:7780


Note You need to log in before you can comment on or make changes to this bug.