Bug 2092160
| Summary: | jq runs out of memory and causes mod_auth_openidc to seqfault | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Kevin Fenzi (rh) <kfenzi> |
| Component: | jq | Assignee: | Tomas Halman <thalman> |
| Status: | CLOSED ERRATA | QA Contact: | Scott Poore <spoore> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 8.6 | CC: | aboscatt, kevin, spoore |
| Target Milestone: | rc | Keywords: | Triaged |
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | sync-to-jira | ||
| Fixed In Version: | jq-1.6-7.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-11-14 15:26:18 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hi Kevin, could you provide your (sanitized?) httpd configuration so I can try to reproduce the issue? Do you need everything ? Or just the mod_auth_openidc ? auth_openidc.conf: ``` OIDCOAuthIntrospectionEndpoint https://id.fedoraproject.org/openidc/TokenInfo OIDCOAuthClientID src-verifier OIDCOAuthClientSecret redacted OIDCOAuthIntrospectionEndpointAuth client_secret_basic OIDCOAuthIntrospectionEndpointMethod POST OIDCOAuthTokenIntrospectionInterval 60 OIDCOAuthSSLValidateServer On OIDCOAuthAcceptTokenAs basic ``` httpush.conf: ``` <Location /> AuthType oauth20 Require all granted </Location> <LocationMatch ".*/git-receive-pack"> AuthType oauth20 Require claims_expr '(.scope | index("https://src.fedoraproject.org/push") != null)' </LocationMatch> <LocationMatch ".*/info/refs"> <If "%{QUERY_STRING} =~ /service=git-upload-pack/"> Require all granted </If> <Else> AuthType oauth20 Require claims_expr '(.scope | index("https://src.fedoraproject.org/push") != null)' </Else> </LocationMatch> SuexecUserGroup pagure package ``` I can get the rest if you want the entire thing... Hi Kevin, thank you, this is enough for apache httpd config. So far I was not able to reproduce the issue. Can you provide the coredump from the crash? It would be good to see how the claims/scopes looks in your case. T. Proved to be difficult to get httpd to give a coredump... but I think I got one. ;) https://infrastructure.fedoraproject.org/infra/tmp/core.httpd.48.a5f728fa134843f98c38f66e860c6613.4056581.16548800650 Hi Kevin, Unfortunately the link doesn't work. Could you provide exact rpm version of involved components? (something like `rpm -qa | grep -e jq -e httpd -e mod_auth`) How does the claim/scope looks like in your setup? Tomas Sigh. It should be: https://infrastructure.fedoraproject.org/infra/tmp/core.httpd.48.a5f728fa134843f98c38f66e860c6613.4056581.1654880065000000.lz4 (Unless bugzilla is cutting it off?) httpd-2.4.37-47.module+el8.6.0+14529+083145da.1.x86_64 httpd-filesystem-2.4.37-47.module+el8.6.0+14529+083145da.1.noarch httpd-tools-2.4.37-47.module+el8.6.0+14529+083145da.1.x86_64 jq-1.6-3.el8.x86_64 jq-debuginfo-1.6-3.el8.x86_64 jq-debugsource-1.6-3.el8.x86_64 mod_auth_gssapi-1.6.1-7.1.el8.x86_64 mod_auth_openidc-2.3.7-11.module+el8.6.0+14082+b6f23e95.x86_64 pagure-web-apache-httpd-5.13.3-2.el8.noarch redhat-logos-httpd-84.5-1.el8.noarch Require claims_expr '(.scope | index("https://src.fedoraproject.org/push") != null)' Thank you Kevin(In reply to Kevin Fenzi from comment #6) > Sigh. It should be: > > https://infrastructure.fedoraproject.org/infra/tmp/core.httpd.48. > a5f728fa134843f98c38f66e860c6613.4056581.1654880065000000.lz4 > > (Unless bugzilla is cutting it off?) > > httpd-2.4.37-47.module+el8.6.0+14529+083145da.1.x86_64 > httpd-filesystem-2.4.37-47.module+el8.6.0+14529+083145da.1.noarch > httpd-tools-2.4.37-47.module+el8.6.0+14529+083145da.1.x86_64 > jq-1.6-3.el8.x86_64 > jq-debuginfo-1.6-3.el8.x86_64 > jq-debugsource-1.6-3.el8.x86_64 > mod_auth_gssapi-1.6.1-7.1.el8.x86_64 > mod_auth_openidc-2.3.7-11.module+el8.6.0+14082+b6f23e95.x86_64 > pagure-web-apache-httpd-5.13.3-2.el8.noarch > redhat-logos-httpd-84.5-1.el8.noarch > > Require claims_expr '(.scope | index("https://src.fedoraproject.org/push") > != null)' Thank you Kevin Happy to help, hopefully you can duplicate it. Let me know if I can provide any more info. Hi Kevin, thank you for your patience. Finally I have a reproducer so I can start working on fix. Jq has an issue when it is used in multi-threading environment. The workaround for you might be to set Apache Httpd MPM configuration to use forking instead of threads. HTH Tom Upstream patch https://github.com/stedolan/jq/pull/2546 Tom Verified. Version :: jq-1.6-7.el8.x86_64 Results :: gating passed no segfault: # gcc -o segfault_test segfault_in_multi_threaded_env.c -lpthread -ljq # ./segfault_test jq_init jq: 0x7f2dd4000b60 prg: .data jq_init jq: 0x7f2dcc000b60 prg: .data compiled dumped: 1 compiled dumped: 1 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (jq bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:6923 |
With jq-1.6-3.el8 and using mod_auth_openidc with apache httpd we are getting segfaults. Here's the trace: #0 0x0000000000000000 in ?? () #1 0x00007f1c98bc1019 in jv_nomem_handler (handler=0x0, data=0x0) at src/jv_alloc.c:82 #2 0x00007f1c98bba10d in jq_compile_args (jq=<optimized out>, str=<optimized out>, args=...) at src/execute.c:1162 #3 0x00007f1c994cab6d in oidc_authz_match_claims_expr () from target:/etc/httpd/modules/mod_auth_openidc.so #4 0x00007f1c994cafbf in oidc_authz_worker24 () from target:/etc/httpd/modules/mod_auth_openidc.so #5 0x00007f1c994f2eaa in oidc_authz_checker () from target:/etc/httpd/modules/mod_auth_openidc.so #6 0x00007f1ca5755d61 in apply_authz_sections () from target:/etc/httpd/modules/mod_authz_core.so #7 0x00007f1ca5755e45 in apply_authz_sections () from target:/etc/httpd/modules/mod_authz_core.so #8 0x00007f1ca575611a in authorize_user_core () from target:/etc/httpd/modules/mod_authz_core.so #9 0x00005637546a82e8 in ap_run_auth_checker () #10 0x00005637546aa8bc in ap_process_request_internal () #11 0x00005637546c9840 in ap_process_async_request () #12 0x00005637546c5ce0 in ap_process_http_connection () #13 0x00005637546bc0c8 in ap_run_process_connection () #14 0x00007f1c9dd5fa47 in process_socket () from target:/etc/httpd/modules/mod_mpm_event.so #15 0x00007f1c9dd603ea in worker_thread () from target:/etc/httpd/modules/mod_mpm_event.so #16 0x00007f1ca903f1cf in start_thread () from target:/lib64/libpthread.so.0 #17 0x00007f1ca8aa7d83 in clone () from target:/lib64/libc.so.6 Our config has: Require claims_expr '(.scope | index("https://src.fedoraproject.org/push") != null)' and that seems to cause jq to run out of memory now? It may also have something to do with our httpd setup, but we aren't limiting things that I can think of. Downgrading back to jq-1.5-12.el8.x86_64 and restarting httpd (note: just reloading doesn't do it) seems to work around the issue. Happy to gather more info, etc.