Bug 1934210
| Summary: | Bad HTTP method requests filling up /var/log/messages with stack traces | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Julio Entrena Perez <jentrena> |
| Component: | Packaging | Assignee: | satellite6-bugs <satellite6-bugs> |
| Status: | CLOSED ERRATA | QA Contact: | Lukas Pramuk <lpramuk> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.7.0 | CC: | bbuckingham, egolov, ekohlvan, mshimura, pcreech, vijsingh |
| Target Milestone: | 6.12.0 | Keywords: | Triaged |
| Target Release: | Unused | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | tfm-rubygem-rails-6.0.4 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-11-16 13:32:29 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Apologies, it's not /var/log/foreman/production.log but /var/log/httpd/error_log It looks like this is https://github.com/rails/rails/issues/38998. It has https://github.com/rails/rails/pull/40246 as a resolution and https://github.com/rails/rails/commit/5ccc67605ec462a8e164635c75be84a3af0234f5 as a cherry pick to 6.0-stable but no Rails release includes it yet. It's aligned to Rails 6.0.4 according to the current milestones. (In reply to Julio Entrena Perez from comment #1) > Apologies, it's not /var/log/foreman/production.log but > /var/log/httpd/error_log then we should edit the summary (In reply to Ewoud Kohl van Wijngaarden from comment #2) > It looks like this is https://github.com/rails/rails/issues/38998. It has > https://github.com/rails/rails/pull/40246 as a resolution and > https://github.com/rails/rails/commit/ > 5ccc67605ec462a8e164635c75be84a3af0234f5 as a cherry pick to 6.0-stable but > no Rails release includes it yet. It's aligned to Rails 6.0.4 according to > the current milestones. Rails 6.0.4 is out since June 15th: https://rubygems.org/gems/rails/versions/6.0.4. I think this should be solved by a packaging update. VERIFIED. @Satellite 6.12.0 Snap10 rubygem-rails-6.0.4.7-1.el8sat.noarch by the manual reproducer described in comment#0: 1) Issue a bad HTTP method to Satellite while monitoring logs: # curl -X ABCD http://$(hostname) # journalctl -efu foreman Sep 14 06:33:04 dhcp-2-87.vms.sat.rdu2.redhat.com foreman[2000]: 2022-09-14 06:33:04 -0400 Rack app ("ABCD /" - (10.1.2.87)): #<ActionController::UnknownHttpMethod: ABCD, accepted HTTP methods are OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, CONNECT, PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK, VERSION-CONTROL, REPORT, CHECKOUT, CHECKIN, UNCHECKOUT, MKWORKSPACE, UPDATE, LABEL, MERGE, BASELINE-CONTROL, MKACTIVITY, ORDERPATCH, ACL, SEARCH, MKCALENDAR, and PATCH> >>> the error in the log is no longer followed by stacktrace Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Satellite 6.12 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:8506 |
Description of problem: When httpd in Satellite receives a request with an invalid HTTP method, it logs the condition to /var/log/httpd/error_log with the following text: [ 2021-02-14 10:09:15.7381 28224/7f6e0e815700 Pool2/Implementation.cpp:1274 ]: [App 28283 stderr] [ 2021-02-14 10:09:15.7380 6537/0x000000000c291948(Worker 1) utils.rb:74 ]: *** Exception ActionController::UnknownHttpMethod in Rack application object (ABCD, accepted HTTP methods are OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, CONNECT, PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK, VERSION-CONTROL, REPORT, CHECKOUT, CHECKIN, UNCHECKOUT, MKWORKSPACE, UPDATE, LABEL, MERGE, BASELINE-CONTROL, MKACTIVITY, ORDERPATCH, ACL, SEARCH, MKCALENDAR, and PATCH) (process 6537, thread 0x000000000c291948(Worker 1)): This is reasonable, but after that there are 33 more lines of stack trace: [ 2021-03-02 17:21:13.0880 40626/7f438426d700 Pool2/Implementation.cpp:1274 ]: [App 40766 stderr] from /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/http/request.rb:431:in `check_method' [ 2021-03-02 17:21:13.0880 40626/7f438426d700 Pool2/Implementation.cpp:1274 ]: [App 40766 stderr] from /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/http/request.rb:143:in `request_method' [...] A Qualys security scanner is regularly hitting the Satellite with bad HTTP methods and this is filling up the filesystem where /var/log/httpd/ resides. Version-Release number of selected component (if applicable): satellite-6.7.1-1.el7sat.noarch How reproducible: Always Steps to Reproduce: 1. Issue a bad HTTP method to httpd in Satellite: $ curl -X ABCD http://<fqdn_of_satellite> Actual results: The above line is logged to /var/log/httpd/error.log followed by 30+ lines of stack trace. Expected results: Only the above line is logged to /var/log/httpd/error.log . Additional info: This condition is being triggered very frequently by a Qualys security scanner sending all sorts of bad methods to the Satellite: $ grep "accepted HTTP methods are" error_log | awk '{print $24}' | sed -e 's/^(//' -e 's/,$//' | sort | uniq -c 57 ABCD 18 BADMETHOD 60 BADMTHD 18 BDMTHD 18 CFYZ 18 DEBUG 143 get 18 INDEX 38 QUALYS 37 rndmmtd 18 RNDMMTD 72 TRACK The resulting stack traces are filling up the filesystem where /var/log resides making the Satellite unavailable. The same behaviour is observable on Satellite 6.8.