Bug 1829782 - openscap scanner runs out of memory when scanning systems with big number of files
Summary: openscap scanner runs out of memory when scanning systems with big number of ...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: openscap
Version: 7.8
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Jan Černý
QA Contact: BaseOS QE Security Team
Mirek Jahoda
URL:
Whiteboard:
Depends On: 1824152
Blocks: 1816199
TreeView+ depends on / blocked
 
Reported: 2020-04-30 10:55 UTC by Matus Marhefka
Modified: 2020-07-01 15:16 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
.Scanning large numbers of files with OpenSCAP causes systems to run out of memory The OpenSCAP scanner stores all collected results in the memory until the scan finishes. As a consequence, the system might run out of memory on systems with low RAM when scanning large numbers of files, for example, from the large package groups _Server with GUI_ and _Workstation_. To work around this problem, use smaller package groups, for example, _Server_ and _Minimal Install_ on systems with limited RAM. If your scenario requires large package groups, you can test whether your system has sufficient memory in a virtual or staging environment. Alternatively, you can tailor the scanning profile to deselect rules that involve recursion over the entire `/` filesystem: * `rpm_verify_hashes` * `rpm_verify_permissions` * `rpm_verify_ownership` * `file_permissions_unauthorized_world_writable` * `no_files_unowned_by_user` * `dir_perms_world_writable_system_owned` * `file_permissions_unauthorized_suid` * `file_permissions_unauthorized_sgid` * `file_permissions_ungroupowned` * `dir_perms_world_writable_sticky_bits` This prevents the OpenSCAP scanner from causing the system to run out of memory.
Clone Of: 1824152
Environment:
Last Closed: 2020-05-12 09:25:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Matus Marhefka 2020-04-30 10:55:43 UTC
+++ This bug was initially created as a clone of Bug #1824152 +++

Description of problem:
The openscap scanner runs out of memory when scanning systems with big number of packages (for example 'Server with GUI' group). This is always reproducible on systems with minimal HW requirements for RHEL-7 [1] and with 'Server with GUI' group of packages installed. The issue is dependent on a content which is processed by the scanner. Mainly rules which utilize rpm, textfilecontent54 or file probes are causing this issue as some of these rules are processing all rpms/files on a filesystem (for example rules "rpm_verify_*", "file_permissions_*", "dir_perms_*", "no_files_unowned_by_user", etc.).

Note: This is probably due to memory management in openscap scanner. All the collected results are stored in memory until scan finishes.


Version-Release number of selected component (if applicable):
openscap-1.2.17-9.el7
openscap-scanner-1.2.17-9.el7


How reproducible:
always


Steps to Reproduce:
1. Setup a VM with 1 CPU core and 1.5 GB RAM (network install [1]).
2. Install package group 'Server with GUI'.
3. Tailor a profile which uses many rules like rpm_verify_*", "file_permissions_*", "dir_perms_*", "no_files_unowned_by_user", etc. and use it to scan the system. Alternatively use e8 profile from scap-security-guide as it selects many such rules.
4. Scanner is killed during the scan as it consumes all free memory on the system (OOM kill).


Actual results:
Scanner is killed because system runs out of memory.


Expected results:
Scanner finishes the scan successfully without allocating all the system memory resources.


Additional info:
[1] https://access.redhat.com/articles/rhel-limits#minimum-required-memory-3

Comment 3 Matěj Týč 2020-05-12 09:25:48 UTC
Resolving this issue requires a large-scale rework of the OpenSCAP codebase, which is out of scope for the RHEL7 as it its now in Maintenanace Phase 2. It is not an urgent issue, ase the problem isn't likely to occur on typical scanned systems, which typically are servers without the GUI and a minimal package set.


Note You need to log in before you can comment on or make changes to this bug.