Bug 1276032
| Summary: | Crash on QMP input exceeding limits | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Markus Armbruster <armbru> | |
| Component: | qemu-kvm-rhev | Assignee: | Markus Armbruster <armbru> | |
| Status: | CLOSED ERRATA | QA Contact: | CongLi <coli> | |
| Severity: | unspecified | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 7.0 | CC: | chayang, coli, huding, juzhang, ngu, virt-maint, xfu | |
| Target Milestone: | rc | |||
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | qemu-kvm-rhev-2.5.0-1.el7 | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: JSON parser's implementation is ridiculously inefficient, and error handling when nesting exceeds limit is flawed.
Consequence: excessive memory use when processing QMP input (~500MiB for a test case with ~100k tokens), crash on input exceeding nesting limit (1024 curly braces or 1024 square brackets, counted separately).
Fix: use simpler parsing techniques and data structurs, correct input size limiting.
Result: reasonable memory use (test case down to ~20MiB), input exceeding nesting limit (1024 braces + brackets, *not* counted separately) is cleanly rejected.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1276036 (view as bug list) | Environment: | ||
| Last Closed: | 2016-11-07 20:51:00 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1276036, 1288337 | |||
Proposed upstream fix: http://lists.gnu.org/archive/html/qemu-devel/2015-11/msg04518.html Related bug: the parser is a ridiculous memory hog. If we get that fixed upstream together with this bug, I'll widen this bug's scope to cover it. If not, I'll file a new one. Both the crash and the excessive memory use have been fixed upstream: df64983 qjson: Limit number of tokens in addition to total size 9bada89 qjson: surprise, allocating 6 QObjects per token is expensive 95385fe qjson: store tokens in a GQueue d538b25 qjson: Convert to parser to recursive descent d2ca7c0 qjson: replace QString in JSONLexer with GString 6b9606f qjson: Inline token_is_escape() and simplify 50e2a46 qjson: Inline token_is_keyword() and simplify c546166 qjson: Give each of the six structural chars its own token type b8d3b1d qjson: Spell out some silent assumptions f0ae030 check-qjson: Add test for JSON nesting depth limit 0753113 qjson: Don't crash when input exceeds nesting limit 4f2d31f qjson: Apply nesting limit more sanely Reproduce this bug on:
qemu-kvm-rhev-2.3.0-31.el7.x86_64
Steps:
1. Run qemu with a qmp monitor.
# /usr/libexec/qemu-kvm -nodefaults -S -display none -qmp stdio
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 3, "major": 2}, "package": " (qemu-kvm-rhev-2.3.0-31.el7)"}, "capabilities": []}}
2. Input 1025 left braces.
{{{{{{{{{{{{{...
Results:
Qemu aborted with following error.
**
ERROR:qobject/json-parser.c:295:parser_context_peek_token: assertion failed: (ctxt->tokens.pos < ctxt->tokens.count)
Aborted (core dumped)
Verify this bug on:
qemu-kvm-rhev-2.6.0-19.el7.x86_64
Steps:
1. Run qemu with a qmp monitor.
# /usr/libexec/qemu-kvm -nodefaults -S -display none -qmp stdio{"QMP": {"version": {"qemu": {"micro": 0, "minor": 6, "major": 2}, "package": " (qemu-kvm-rhev-2.6.0-19.el7)"}, "capabilities": []}}
2. Input 1025 left braces.
{{{{{{{{{{{{{...
Results:
1. qemu not aborted and there is error in qmp.
{"error": {"class": "GenericError", "desc": "Invalid JSON syntax"}}
2. qemu not crash with 1025 left braces and 1024 right braces,
got same error as above.
Please be free to correct if there is any problem.
Thanks.
Change the bug status to verified according to comment #5. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2673.html |
Description of problem: The QMP parser attempts to limit nesting depth and input size to defend against input triggering excessive heap or stack memory use. However, it crashes when the nesting depth limit is exceeded. I suspect it also crashes when the input size is exceeded, but haven't actually tested. This is a regression since RHEL-6. How reproducible: Always Steps to Reproduce: 1. Run with a QMP monitor, e.g. $ qemu-kvm -nodefaults -S -display none -qmp stdio 2. Input at least 1025 left braces Actual results: ERROR:/home/armbru/work/qemu-kvm-rhev7/qobject/json-parser.c:295:parser_context_peek_token: assertion failed: (ctxt->tokens.pos < ctxt->tokens.count) Aborted (core dumped) Expected results: Same behavior as in RHEL-6, i.e. {"error": {"class": "JSONParsing", "desc": "Invalid JSON syntax", "data": {}}} Additional info: It's really the 1025-th left brace. 1024 left braces followed by 1024 right braces doesn't crash. Broken upstream in commit 65c0f1e "json-parser: don't replicate tokens at each level of recursion", v1.2.0.