RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2161499 - Insights compliance command fails with scan failed error
Summary: Insights compliance command fails with scan failed error
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: openscap
Version: 8.6
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Evgeny Kolesnikov
QA Contact: BaseOS QE Security Team
Petr Hybl
URL:
Whiteboard:
: 2147604 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-17 06:14 UTC by Nikhil Gupta
Modified: 2024-01-08 13:14 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
.OpenSCAP memory-consumption problems On systems with limited memory, the OpenSCAP scanner might stop prematurely or it might not generate the results files. To work around this problem, you can customize the scanning profile to deselect rules that involve recursion over the entire `/` file system: * `rpm_verify_hashes` * `rpm_verify_permissions` * `rpm_verify_ownership` * `file_permissions_unauthorized_world_writable` * `no_files_unowned_by_user` * `dir_perms_world_writable_system_owned` * `file_permissions_unauthorized_suid` * `file_permissions_unauthorized_sgid` * `file_permissions_ungroupowned` * `dir_perms_world_writable_sticky_bits` For more details and more workarounds, see the related link:https://access.redhat.com/articles/6999111[Knowledgebase article].
Clone Of:
Environment:
Last Closed: 2023-03-27 08:56:34 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker OPENSCAP-3067 0 None None None 2023-01-17 06:20:00 UTC
Red Hat Issue Tracker RHELPLAN-145263 0 None None None 2023-01-17 06:16:52 UTC

Description Nikhil Gupta 2023-01-17 06:14:16 UTC 
Description of problem:
While running the 'insights-client --compliance' command, getting the following error:
~~~
2022-11-30 13:38:27,611 INFO insights.client.apps.compliance Running scan for xccdf_org.ssgproject.content_profile_cis_workstation_l1... this may take a while
2022-11-30 13:38:27,612 DEBUG insights.util.subproc Executing: [['oscap', 'xccdf', 'eval', '--profile', 'xccdf_org.ssgproject.content_profile_cis_workstation_l1', '--results', '/var/tmp/insights-client/insights-archive-hi4pf70l/insights-dmzgrg06.gbc.local-20221130133822/oscap_results-xccdf_org.ssgproject.content_profile_cis_workstation_l1.xml', '/usr/share/xml/scap/ssg/content/ssg-rhel8-ds-1.2.xml']]
2022-11-30 13:42:41,702 ERROR insights.client.apps.compliance Scan failed
2022-11-30 13:42:41,882 ERROR insights.client.apps.compliance WARNING: This content points out to the remote resources. Use `--fetch-remote-resources' option to download them.
~~~

The oom-killer has been invoked while running compliance command:
~~~
Dec  1 11:28:48 dmzgrg06 kernel: nsrexecd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Dec  1 11:28:48 dmzgrg06 kernel: CPU: 1 PID: 1549 Comm: nsrexecd Kdump: loaded Not tainted 4.18.0-372.32.1.el8_6.x86_64 #1
Dec  1 11:28:48 dmzgrg06 kernel: Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.0 12/17/2019
Dec  1 11:28:48 dmzgrg06 kernel: Call Trace:
Dec  1 11:28:48 dmzgrg06 kernel: dump_stack+0x41/0x60
Dec  1 11:28:48 dmzgrg06 kernel: dump_header+0x4a/0x1df
Dec  1 11:28:48 dmzgrg06 kernel: oom_kill_process.cold.32+0xb/0x10
Dec  1 11:28:48 dmzgrg06 kernel: out_of_memory+0x1bd/0x4e0
Dec  1 11:28:48 dmzgrg06 kernel: __alloc_pages_slowpath+0xbe0/0xcc0
Dec  1 11:28:48 dmzgrg06 kernel: __alloc_pages_nodemask+0x2e2/0x320
Dec  1 11:28:48 dmzgrg06 kernel: pagecache_get_page+0xce/0x310
Dec  1 11:28:48 dmzgrg06 kernel: filemap_fault+0x78b/0xa10
Dec  1 11:28:48 dmzgrg06 kernel: ? __mod_lruvec_page_state+0x5e/0x80
...
Dec  1 11:28:49 dmzgrg06 kernel: [3353301]     0 3353301  3426205  1329185 22974464  1405758             0 oscap
Dec  1 11:28:49 dmzgrg06 kernel: [3362023]     0 3362023     1830       18    61440        0             0 sleep
Dec  1 11:28:49 dmzgrg06 kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1012.slice/session-459.scope,task=oscap,pid=3353301,uid=0
Dec  1 11:28:49 dmzgrg06 kernel: Out of memory: Killed process 3353301 (oscap) total-vm:13704820kB, anon-rss:5316740kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:22436kB oom_score_adj:0
Dec  1 11:28:49 dmzgrg06 kernel: oom_reaper: reaped process 3353301 (oscap), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
~~~

Version-Release number of selected component (if applicable):
openscap-1.3.6-3.el8.x86_64
openscap-scanner-1.3.6-3.el8.x86_64
scap-security-guide-0.1.63-1.el8_6.noarch

How reproducible:
Always (On one of the hosts in the customer environment)

Steps to Reproduce:
1. Register host with insights
2. Assign 'cis_workstation_l1' compliance policy to the host
3. Run the 'insights-client --compliance' command on the host.

Actual results:
The compliance scan failed and the result is not uploaded to the cloud.

Expected results:
The scan should be successful and the result should be uploaded to the cloud.

Additional info:
The server has a mount /u01 for 750G, with 550G full.
Comment 1 Nikhil Gupta 2023-01-17 06:17:46 UTC 
oscap --verbose DEVEL xccdf eval --profile cis_server_l1 --report cis_server_l1.html /usr/share/xml/scap/ssg/content/ssg-rhel8-ds.xml

and after an hour or more it got me this:

==========================================================
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
Killed
[root@dmzgrg06 ~]#
===========================================================

I did not kill it, something inside did it.


Scrolling up, I get the error here:

=================================================
D: oscap:     Signaling `notfull' [oscap(1916230):icache_worker(7f79f6ffd700):icache.c:241:probe_icache_worker]
D: oscap:     Handling NOP [oscap(1916230):icache_worker(7f79f6ffd700):icache.c:265:probe_icache_worker]
D: oscap:     Sync [oscap(1916230):probe_worker(7f78e47f0700):icache.c:477:probe_icache_nop]
D: oscap:     old flag: 0, new flag: 2. [oscap(1916230):probe_worker(7f78e47f0700):probe-api.c:688:probe_cobj_set_flag]
D: oscap:     handler result = 0x7f79e8030ba0, return code = 0 [oscap(1916230):probe_worker(7f78e47f0700):worker.c:99:probe_worker_runfn]
D: oscap:     probe thread deleted [oscap(1916230):probe_worker(7f78e47f0700):worker.c:119:probe_worker_runfn]
D: oscap:     Sorting blocks & building iterator array [oscap(1916230):probe_worker(7f78e47f0700):sexp-manip.c:1408:SEXP_list_sort]
D: oscap:     Iterator count = 1 [oscap(1916230):probe_worker(7f78e47f0700):sexp-manip.c:1442:SEXP_list_sort]
D: oscap:     cnt = 0 [oscap(1916230):probe_worker(7f78e47f0700):seap-message.c:138:SEAP_msgattr_exists]
D: oscap:     no-reply not set: sending full reply [oscap(1916230):probe_worker(7f78e47f0700):seap.c:481:SEAP_reply]
D: oscap:     MSG -> SEXP [oscap(1916230):probe_worker(7f78e47f0700):seap-packet.c:260:SEAP_packet_msg2sexp]
D: oscap: ("seap.msg" ":id" 29 ":reply-id" 29 (2 () ((("linux:rpminfo_item" ":id" "11916230751678" ) ("name" "xorg-x11-server-common" ) ("arch" "x86_64" ) ("epoch" "(none)" ) ("release" "9.el8" ) ("version" "1.20.11" ) [evr_string]("evr" "0:1.20.11-9.el8" ) ("signature_keyid" "199e2f91fd431d51" ) ("extended_name" "xorg-x11-server-common-0:1.20.11-9.el8.x86_64" ) ) ) () ) ) [oscap(1916230):probe_worker(7f78e47f0700):seap-packet.c:261:SEAP_packet_msg2sexp]
D: oscap:     packet size: 2076 [oscap(1916230):probe_worker(7f78e47f0700):seap-packet.c:262:SEAP_packet_msg2sexp]
D: oscap:     name=reply-id, value=0x7f79e8033550 [oscap(1916230):probe_worker(7f78e47f0700):seap-message.c:73:SEAP_msg_free]
D: oscap:     probe_worker_runfn has finished [oscap(1916230):probe_worker(7f78e47f0700):worker.c:179:probe_worker_runfn]
D: oscap:     Received packet [oscap(1916230):oscap(7f7a64d5d0c0):seap-packet.c:794:SEAP_packet_recv]
D: oscap: ("seap.msg" ":id" 29 ":reply-id" 29 (2 () ((("linux:rpminfo_item" ":id" "11916230751678" ) ("name" "xorg-x11-server-common" ) ("arch" "x86_64" ) ("epoch" "(none)" ) ("release" "9.el8" ) ("version" "1.20.11" ) [evr_string]("evr" "0:1.20.11-9.el8" ) ("signature_keyid" "199e2f91fd431d51" ) ("extended_name" "xorg-x11-server-common-0:1.20.11-9.el8.x86_64" ) ) ) () ) ) [oscap(1916230):oscap(7f7a64d5d0c0):seap-packet.c:795:SEAP_packet_recv]
D: oscap:     packet size: 2076 [oscap(1916230):oscap(7f7a64d5d0c0):seap-packet.c:796:SEAP_packet_recv]
D: oscap:     Message received. [oscap(1916230):oscap(7f7a64d5d0c0):oval_probe_ext.c:579:oval_probe_comm]
D: oscap:     name=(null), value=0x7f79e80363c0 [oscap(1916230):oscap(7f7a64d5d0c0):seap-message.c:73:SEAP_msg_free]
D: oscap:     Syschar entry type: 9003 'linux' => decoded OK [oscap(1916230):oscap(7f7a64d5d0c0):oval_sexp.c:982:oval_sexp_to_sysitem]
I: oscap:     Test 'oval:ssg-test_package_xorg-x11-server-common_removed:tst:1' requires that none of the objects defined by 'oval:ssg-obj_test_package_xorg-x11-server-common_removed:obj:1' exist on the system. [oscap(1916230):oscap(7f7a64d5d0c0):oval_resultTest.c:909:_oval_result_test_evaluate_items]
I: oscap:     1 objects defined by 'oval:ssg-obj_test_package_xorg-x11-server-common_removed:obj:1' exist on the system. [oscap(1916230):oscap(7f7a64d5d0c0):oval_resultTest.c:918:_oval_result_test_evaluate_items]
I: oscap:     Test 'oval:ssg-test_package_xorg-x11-server-common_removed:tst:1' does not contain any state to compare object with. [oscap(1916230):oscap(7f7a64d5d0c0):oval_resultTest.c:920:_oval_result_test_evaluate_items]
I: oscap:     All items matching object 'oval:ssg-obj_test_package_xorg-x11-server-common_removed:obj:1' were collected. (flag=complete) [oscap(1916230):oscap(7f7a64d5d0c0):oval_resultTest.c:963:_oval_result_test_evaluate_items]
I: oscap:   Test 'oval:ssg-test_package_xorg-x11-server-common_removed:tst:1' evaluated as false. [oscap(1916230):oscap(7f7a64d5d0c0):oval_resultTest.c:1164:oval_result_test_eval]
I: oscap: Definition 'oval:ssg-package_xorg-x11-server-common_removed:def:1' evaluated as false. [oscap(1916230):oscap(7f7a64d5d0c0):oval_resultDefinition.c:170:oval_result_definition_eval]
Result  fail

D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
D: oscap: rbt_str_add: non-zero return code [oscap(1916230):oscap(7f7a64d5d0c0):oval_string_map.c:211:oval_string_map_put]
Comment 2 Evgeny Kolesnikov 2023-01-20 05:39:40 UTC 
Our preliminary investigation result: the OOM situation could be caused by one of the rules that traverse filesystems looking for invalid rights/ownership.

The suspects are:

- file_permissions_unauthorized_world_writable
- no_files_unowned_by_user
- file_permissions_ungroupowned

In order to confirm if this is the case one can either:

- execute oscap --verbose DEVEL xccdf eval --profile cis_server_l1 --skip-rule file_permissions_unauthorized_world_writable --skip-rule no_files_unowned_by_user --skip-rule file_permissions_ungroupowned --report cis_server_l1.html /usr/share/xml/scap/ssg/content/ssg-rhel8-ds.xml, effectively excluding these rules from the scan
- or temporarily unmount /u01 during the scan

The workaround for the problem is to exclude all offending rules from the scan using a tailoring file.
Comment 6 Evgeny Kolesnikov 2023-02-28 09:10:19 UTC 
*** Bug 2147604 has been marked as a duplicate of this bug. ***
Comment 7 Jan Černý 2023-03-27 08:56:34 UTC 
This bug boils down to the outstanding memory problems of OpenSCAP which are covered by the KBS https://access.redhat.com/articles/6999111 and that are unlikely to be fixed.
Note You need to log in before you can comment on or make changes to this bug.