RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2147604 - oscap compliance reporting fails due to lack of memory (via Insights and directly)
Summary: oscap compliance reporting fails due to lack of memory (via Insights and dire...
Keywords:
Status: CLOSED DUPLICATE of bug 2161499
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: openscap
Version: 8.7
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Jan Černý
QA Contact: BaseOS QE Security Team
Petr Hybl
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-11-24 11:38 UTC by Benjamin Blasco
Modified: 2023-03-27 09:52 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
This known issue and possible workarounds are described in KBS https://access.redhat.com/articles/6999111.
Clone Of:
Environment:
Last Closed: 2023-02-28 09:10:19 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-140383 0 None None None 2022-11-24 11:57:52 UTC

Description Benjamin Blasco 2022-11-24 11:38:41 UTC
Description of problem:
oscap scan against ACSC Essential 8 profile fails both locally and via Insights Compliance service when there is insufficient memory on the system


Version-Release number of selected component (if applicable):
[root@node2 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.7 (Ootpa)
[root@node2 ~]# rpm -qa | grep insights
insights-client-3.1.7-8.el8.noarch
[root@node2 ~]# rpm -qa | grep scap
scap-security-guide-0.1.63-4.el8.noarch
openscap-1.3.6-4.el8.x86_64
openscap-scanner-1.3.6-4.el8.x86_64
[root@node2 ~]# 

How reproducible:
100% of the time when system has ~1GB memory

Steps to Reproduce:
METHOD A:
1. Install RHEL 8.7, insights-client, sacp-security-guide (latest versions)
2. Register the system with Insights
3. Associate the system with the Essential 8 compliance profile
4. Run `insights-client --compliance`
5. Compliance scan will fail with errors reported here: 
http://pastebin.test.redhat.com/1082608

METHOD B: 
1. Install RHEL 8.7, insights-client, sacp-security-guide (latest versions)
2. Run ` oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_e8 --results scan.xml /usr/share/xml/scap/ssg/content/ssg-rhel8-ds.xml`
3. Compliance scan will fail with errors reported here: 
http://pastebin.test.redhat.com/1082609

Actual results:
Scans fail at test CCE-82282-5 but don't report a meaningful problem

Expected results:
Scan completes successfully (and is uploaded to Insights if configured)

Additional info:
Work-around to fix problem:
1. Add more swap to the system using commands similar to that below:
```
dd if=/dev/zero of=/swapfile bs=1M count=2048
chmod 0600 /swapfile 
mkswap /swapfile 
swapon /swapfile
```

2. Re-run the scan

Expectation for resolution of this issue:
1.  Should oscap print a meaningful message to say there is a problem rather than just failing without an error?
2.  Can we publish minimum memory requirements or recommendations for compliance scans?

Comment 1 Jan Černý 2022-11-24 13:31:04 UTC
Hi,

I am getting the same output on a fresh RHEL 8.7 virtual machine. But I think that the output is expected and isn't a bug. The errors are caused by insufficient permissions of the user, oscap can't read some system configuration files. In general, we recommend to run oscap with sudo or with root privileges. 

The details can be found in the HTML report (add "--report report.html" to the command to generate a HTML report) or in the ARF file (the --results-arf option).

Can you elaborate why you think that you have a memory related issue? Is your experience different on systems with more memory?

Also, can you try to run the oscap with sudo or as root user? Please share the results here.

Comment 2 Marek Haicman 2022-11-24 21:31:00 UTC
Hello Benjamin, could you check the page on our official RHEL documentation if it answers your question? I understand the configuration you use is below minimal required system specs.

https://access.redhat.com/articles/rhel-limits

Comment 3 Benjamin Blasco 2022-11-24 22:02:34 UTC
1. Everything tested has been run as the root user, so there is no permissions issue.
2. Please see my previous instructions on adding 2GB of swap in the work-around section.  This resolved the issue immediately with no other changes.  The issue was consistently reproducible until I increased the memory in the system.
3. The memory in the system is below the minimum as per the link you shared, BUT in AWS users are able to deploy instances with 1GiB of memory or less, which is below that requirement.  This is contradictory information in a broader RHEL sense.

Comment 4 Benjamin Blasco 2022-11-24 22:15:31 UTC
I just re-tested using the following command and got the attached results

```
[root@node2 ~]# oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_e8 --report report.html --results-arf results.arf --results scan.xml /usr/share/xml/scap/ssg/content/ssg-rhel8-ds.xml
WARNING: Datastream component 'scap_org.open-scap_cref_security-data-oval-com.redhat.rhsa-RHEL8.xml.bz2' points out to the remote 'https://access.redhat.com/security/data/oval/com.redhat.rhsa-RHEL8.xml.bz2'. Use '--fetch-remote-resources' option to download it.
WARNING: Skipping 'https://access.redhat.com/security/data/oval/com.redhat.rhsa-RHEL8.xml.bz2' file which is referenced from datastream
WARNING: Skipping ./security-data-oval-com.redhat.rhsa-RHEL8.xml.bz2 file which is referenced from XCCDF content
--- Starting Evaluation ---

etc...

Rule    xccdf_org.ssgproject.content_rule_sshd_set_loglevel_info
Ident   CCE-82282-5
Result  fail




Killed

```

So in the above it just said "killed" which is different to the output from before now that i have added --result-arf and --report report.html, which is interesting.  I then went to install tmux so I could monitor memory usage whilst running the command again, and got this:

```
[root@node2 ~]# yum install tmux
Updating Subscription Management repositories.
Last metadata expiration check: 2:34:54 ago on Thu 24 Nov 2022 19:37:55 UTC.
Dependencies resolved.
=========================================================================================================================
 Package             Architecture          Version                    Repository                                    Size
=========================================================================================================================
Installing:
 tmux                x86_64                2.7-1.el8                  rhel-8-for-x86_64-baseos-rpms                317 k

Transaction Summary
=========================================================================================================================
Install  1 Package

Total download size: 317 k
Installed size: 770 k
Is this ok [y/N]: y
Downloading Packages:
tmux-2.7-1.el8.x86_64.rpm                                                                458 kB/s | 317 kB     00:00    
-------------------------------------------------------------------------------------------------------------------------
Total                                                                                    411 kB/s | 317 kB     00:00     
[Errno 12] Cannot allocate memory
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'yum clean packages'.
[root@node2 ~]# 
```

Then I check the memory:
```
[root@node2 ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:            776         203         283           5         289         451
Swap:             0           0           0
[root@node2 ~]# 

```

I am quite confident that the system is running out of memory during the scan, and in this instance didn't clean up when it was "killed".

Comment 5 Jan Černý 2022-11-25 09:28:40 UTC
Thanks for clarification.

I'm sorry I was confused by the output you pasted in the bug description (http://pastebin.test.redhat.com/1082609) where oscap is run under non-root user without sudo and where the output doesn't contain anything unexpected.

From your latest comment, I see it's definitely a memory issue, when you run as root you get the correct "fail" rule result, but then the process is killed during generation of XML/HTML results. If the system is below minimum required memory, it doesn't have enough memory to be able to transform the results to XML.

We have seen this problem in past, we have workarounds for some situations. For example in https://bugzilla.redhat.com/show_bug.cgi?id=1829782 it was triggered by too many files on the file system and in the release note we suggested to remove some rules that are data intensive. But in our team we don't see any quick solution, we have a conclusion that the overall design of the openscap tool isn't memory-friendly.

Comment 6 Benjamin Blasco 2022-11-25 23:02:26 UTC
No problem Jan.  I was using output captured by Marek for your reference, but my results were the same when I had tested via `sudo -i`.  If oscap is memory intensive, then do you think this should become a documentation issue?  Having said all of that, should the initial basic version of the command output something more meaningful when it fails mid-run rather than just stopping with no useful info?

Comment 8 Evgeny Kolesnikov 2023-02-28 09:10:19 UTC

*** This bug has been marked as a duplicate of bug 2161499 ***

Comment 9 Jan Černý 2023-03-27 08:56:30 UTC
This bug boils down to the outstanding memory problems of OpenSCAP which are covered by the KBS https://access.redhat.com/articles/6999111 and that are unlikely to be fixed.


Note You need to log in before you can comment on or make changes to this bug.