Bug 1655943
Summary: | Network-mounted filesystems are recursed into despite the content specifies local-only recursion | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | amitkuma | ||||||||
Component: | openscap | Assignee: | Jan Černý <jcerny> | ||||||||
Status: | CLOSED ERRATA | QA Contact: | Matus Marhefka <mmarhefk> | ||||||||
Severity: | medium | Docs Contact: | |||||||||
Priority: | medium | ||||||||||
Version: | 7.5 | CC: | amitkuma, andrewrpalmer, matyc, mhaicman, mmarhefk, mthacker, openscap-maint, santony, yoliynyk | ||||||||
Target Milestone: | rc | ||||||||||
Target Release: | 7.7 | ||||||||||
Hardware: | Unspecified | ||||||||||
OS: | Unspecified | ||||||||||
Whiteboard: | |||||||||||
Fixed In Version: | openscap-1.2.17-3.el7 | Doc Type: | Bug Fix | ||||||||
Doc Text: |
Cause: OpenSCAP scanner wasn't able to identify some configurations of autofs network mounts.
Consequence: Scans based on some OVAL checks were traversing the whole filesystem (including those network mounts), although the check specifies local-only recursion.
Fix: The autofs mtab entries that are not mounted at the time of the scan are no longer considered local by default.
Result: If there is an autofs filesystem mounted at the time the scan is started, it is identified properly. If the directory is not mounted, it is considered as a remote filesystem.
|
Story Points: | --- | ||||||||
Clone Of: | Environment: | ||||||||||
Last Closed: | 2019-08-06 13:18:50 UTC | Type: | Bug | ||||||||
Regression: | --- | Mount Type: | --- | ||||||||
Documentation: | --- | CRM: | |||||||||
Verified Versions: | Category: | --- | |||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||
Embargoed: | |||||||||||
Attachments: |
|
Description
amitkuma
2018-12-04 09:59:37 UTC
The SOSReport from the customer case reveals that these packages are used: openscap-1.2.16-8 scap-security-guide-0.1.36-9 (SSG) The problem which has been logged indicates that there occur: - a regexp matching that involves strings that are not valid UTF8, or - a bug in the scanner causes memory corruption, so although no SSG or local content are invalid UTF8, those strings are created in the scanner. The rule which manifests the problem is xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands. The customer case mentions that some elements of the datastream have been changed (... I've set all recurse_file_system items in ssg-rhel7-ds-dll.xml to "local" ...), so is it correct to assume that you have used tailoring? If so, could you please share with us what were the changes? Anyway, the datastream in scap-security-guide-0.1.36-9 contains unanchored filepaths, which may trigger this issue, so it may be that scap-security-guide in 7.6 doesn't suffer from this. You can extract the datastream from the package: http://download.eng.bos.redhat.com/brewroot/packages/scap-security-guide/0.1.40/12.el7/noarch/scap-security-guide-0.1.40-12.el7.noarch.rpm The nature of this issue is a little bit unclear - we can safely say that the scanner could provide better feedback, so it is easier to pinpoint issues like this one. However, the trigger for this bug may be a bug in the SSG content. Dear Matus, Very thanks for response. ||ssg-rhel7-ds-dll.xml Where we can find this file. How its generated, and what's its purpose? since i cannot find it in /usr/share/xml/scap/ssg/content. Is it same as 'ssg-rhel7-ds.xml'? ||You can extract the datastream from the package: scap-security-guide-0.1.40-12.el7.noarch.rpm I believe this is the option, when customer does not move to "scap-security-guide-0.1.40-12.el7.noarch.rpm", since this rpm will provide upgraded datastream. Am I correct? I have relayed the instruction to customer. Will get back with update :) Hello, the mentioning of ssg-rhel7-ds-dll.xml is a quote of customer from the customer case that indicates that the customer used tailoring, or otherwise edited the datastream. Concerning 0.1.40-12.el7.noarch.rpm, you are right - it is a datastream shipped in RHEL 7.6, that contains, among other things, bugfixes for the 7.5 content. Dear Matus/Matej, Great Thanks for response. This is update from Customer: 1. Changed recurse_file_system=local (everywhere) but did not helped. 2. No tailoring file used. 3. Installed scap-security-guide-0.1.40-12.el7.noarch.rpm(Provided by RHEL-7.6) - changed "recurse_file_system" everywhere in /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml into "local" again - restarted the scan with: /usr/bin/oscap xccdf eval \ --profile xccdf_org.ssgproject.content_profile_stig-rhel7-disa \ --results-arf arf.xml \ --verbose WARNING \ --report report.$$.html \ /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml >msr.$$.out 2>&1 - still the same errors (pcre_exec function) and I also see oscap still accessing the remote filesystems (all processes below except the automounter dll root@zeus ~# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/rootvg-rootlv 12G 2.4G 9.7G 20% / devtmpfs 1.9G 0 1.9G 0% /dev /dev/sda1 509M 115M 394M 23% /boot /dev/mapper/rootvg-varlv 4.0G 2.4G 1.7G 58% /var /dev/mapper/datavg02-dbalv 175G 33M 175G 1% /app/dll_dba /dev/mapper/datavg01-linuxlv 80G 34G 47G 42% /dll_linux /dev/mapper/rootvg-tmplv 4.0G 52M 4.0G 2% /tmp /dev/mapper/datavg01-dokuwikilv 20G 308M 20G 2% /app/dokuwiki /dev/mapper/rootvg-homelv 1014M 381M 634M 38% /home bess5053:/DLL_unix 180G 155G 26G 86% /nfs/DLL_unix ehvs6215:/stage 1.8T 1.7T 103G 95% /nfs/dbastage dll root@zeus ~# lsof | egrep '/nfs/DLL_unix|/nfs/dbastage' automount 9891 root 21r DIR 0,43 0 47205 /nfs/DLL_unix automount 9891 root 22r DIR 0,44 0 47206 /nfs/dbastage automount 9891 9901 root 21r DIR 0,43 0 47205 /nfs/DLL_unix automount 9891 9901 root 22r DIR 0,44 0 47206 /nfs/dbastage automount 9891 9902 root 21r DIR 0,43 0 47205 /nfs/DLL_unix automount 9891 9902 root 22r DIR 0,44 0 47206 /nfs/dbastage automount 9891 9983 root 21r DIR 0,43 0 47205 /nfs/DLL_unix automount 9891 9983 root 22r DIR 0,44 0 47206 /nfs/dbastage automount 9891 10003 root 21r DIR 0,43 0 47205 /nfs/DLL_unix automount 9891 10003 root 22r DIR 0,44 0 47206 /nfs/dbastage automount 9891 10008 root 21r DIR 0,43 0 47205 /nfs/DLL_unix automount 9891 10008 root 22r DIR 0,44 0 47206 /nfs/dbastage probe_fil 56405 root 3r DIR 0,45 40960 2702375 /nfs/dbastage/11.2.0/ctx/data/inxight/lang (ehvs6215:/stage) icache_wo 56405 56406 root 3r DIR 0,45 4096 2702373 /nfs/dbastage/11.2.0/ctx (ehvs6215:/stage) signal_ha 56405 56407 root 3r DIR 0,45 4096 2702656 /nfs/dbastage/11.2.0/ctx/data (ehvs6215:/stage) probe_wor 56405 56409 root 3r DIR 0,45 40960 2702375 /nfs/dbastage/11.2.0/ctx/data/inxight/lang (ehvs6215:/stage) Dear Matus/Matej, Supplementary information from Customer. The only thing I changed was to NOT descend remote filesystems. Customer is chasing for update, Can you kindly assist. It looks like that the customer edited the OVAL in the datasteam, setting recurse_file_system="local" in file_object/behaviors. Despite that, the scanner accesses remote filesystems - this has been determined by lsof. Remote systems access is not directly related to this issue itself, but this setting could be used as a workaround. We have to investigate this and double-check how it could not work. We can't resolve it now, as majority of the team is on PTO - we will be able look into it after the Christmas holiday. Dear Matěj Týč , My question to customer - how "ssg-rhel7-ds-dll.xml" came into the picture, since oscap provides /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml. Customer's response: - See my message of Dec 11 2018 at 10:43 AM +01:00 for lsof output. See the last 4 processes. They are part of oscap. dll root@zeus ~# lsof | egrep '/nfs/DLL_unix|/nfs/dbastage' ....... probe_fil 56405 root 3r DIR 0,45 40960 2702375 /nfs/dbastage/11.2.0/ctx/data/inxight/lang (ehvs6215:/stage) icache_wo 56405 56406 root 3r DIR 0,45 4096 2702373 /nfs/dbastage/11.2.0/ctx (ehvs6215:/stage) signal_ha 56405 56407 root 3r DIR 0,45 4096 2702656 /nfs/dbastage/11.2.0/ctx/data (ehvs6215:/stage) probe_wor 56405 56409 root 3r DIR 0,45 40960 2702375 /nfs/dbastage/11.2.0/ctx/data/inxight/lang (ehvs6215:/stage) The customer is discussing processes: probe_fil icache_wo signal_ha probe_wor But as I go into the source code of openscap scanner: openscap-1.2.17 I cannot find particular processes forked, neither in /usr/libexec/openscap/ Ticket is now, waiting on Engineering. Thanks Have you added <behaviors recurse_file_system="local"> as a child element to every file_object element in the ssg-rhel7-ds.xml? Or have you only changed already existing occurrences of behaviors element? In other words, is there any file_object that doesn't contain behaviors child element? Dear Jan Černý, We have relayed the query to customer, will revert as he responds. Thanks Information from Customer: I only changed already existing occurrences of behaviors elements. Can we achieve this? - Is there any other way to only recurse local filesystems? - If not: how to change the xml file to prevent recursion of all remote filesystems? > Is there any other way to only recurse local filesystems? No, there isn't, because unfortunately the OVAL specification doesn't have anything that would allow to set this option globally. > If not: how to change the xml file to prevent recursion of all remote filesystems? Basically you need to make sure no OVAL object describes any entity from the remote filesystems. Add <behaviors recurse_file_system="local"> as a child to every file_object, fileextendedattribute_object, selinuxsecuritycontext_object, filehash_object, filehash58_object, textfilecontent54_object element. Created attachment 1522150 [details]
Patched openscap library
Created attachment 1522151 [details]
Corresponding openscap scanner
Please try to install the attached packages - they should give you an updated openscap library that prints a more helpful error message that we hope could show us what string exactly is causing the problem. Those packages can be seen as successors of RHEL7.6 packages - there are no earth-shaking changes. The openscap RPM package should be enough, but I have also attached the openscap-scanner in case. Dear Matěj Týč, Customer installed the rpms ran the test and provided the attached output(mr.38348.out). Kindly have a look. Appreciate your help! Amit Here are a few notes from our investigation investigation (no solution yet): The problem manifests itself on the following rules: * xccdf_org.ssgproject.content_rule_no_user_host_based_files * xccdf_org.ssgproject.content_rule_no_host_based_files * xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands * xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned Those rules work all the similar way. They use an OVAL that searches the whole filesystem for files whose name matches a regular expression. In order to find the files matching the given regular expression, we need to examine the name of every single file. The problem is that currently we support only UTF-8 strings in OpenSCAP. However, the file names can have arbitrary encoding. In order to treat the file names correctly, Changing aforementioned rules in a way they check only certain paths doesn't seem to be a proper solution, because at any path there can exist a file. However, we can improve the rules so that it is less likely. Due to this, the solution is most likely changing OpenSCAP library code. We need to investigate if there is a way how to check if a given file name is a valid UTF-8 string and if not, we will have to implement a conversion function that will attempt to covert the string to UTF-8 to be sure we run a regex match only on valid UTF-8 string. Dear Jan Černý, Many thanks for update on investigation. We are curious for forthcoming solution. Thanks We gave the problem a thought, and I ended up with this conclusion: - A filename is a sequence of bytes that may be encoded using an encoding. When doing string operations on the filename (s.a. name matching), it needs to be transformed from the byte sequence to a string, i.e. decoded. When one attempts to decode bytes using incorrect decoding algorithm, one may either encounter a failure that indicates that the decoding algorithm is not the right one, but one may also end up with incorrectly decoded string without an indication that something went wrong. In a nutshell, it is impossible to reliably detect the encoding if it is not unknown. - OpenSCAP assumes that filenames are encoded in a UTF-8 compatible encoding. The log output has revealed several filenames that are encoded differently (our guess is that the ISO-8859-2 encoding has been used), which has caused the error to manifest. The sosreport indicates that the system locale is set to UTF8, so there is no way for the scanner how to determine the right encoding. - One possible fix for that would involve the ability of the openscap library to accept "hints", i.e. an ordered list of encodings to try if the preferred encoding fails. However, this approach has it's inherent flaws, s.a. if there are multiple encodings in the filesystem, an incorrect one may be picked anyway. So instead, I propose you to go through the offending files and re-encode their filenames, as they may be causing you problems elsewhere. There is a related article about this in the Red Hat knowledge Base: https://access.redhat.com/solutions/23252 In a nutshell, you can use the convmv utility to fix the encoding to the expected value, and you will get rid of OpenSCAP problems you experience right now, and you may avoid future problems coming from naming files in a way that is not used and expected nowadays. Dear Matěj Týč, Appreicate your investigation and response. I have asked customer to go through the offending files and re-encode filenames to UTF-8. Will revert with Customer's response. Dear Matěj Týč, =======Customer have this query===== 1. Is it possible to use filters in ssg-rhel7-ds.xml to prevent descending into remote filesystems? I know filtering is an option but I don't know how to deploy it. 2. Another option for me is to exclude the 4 rules. ==================================== I have asked him to try option-2, But insights of option-1 would be great! Customer tried option 2 - Another option for me is to exclude the 4 rules. This works but the result is an incomplete report. We have investigated the problem of remote file systems. We think that setting recurse_file_system="local" should work. There might be 2 possibilities why this attribute is ignored. We need to narrow down the problem to one of them. The possibilities are: 1. There might be a bug in OpenSCAP during processing recurse_file_system which we weren't able to reproduce so far. 2. OpenSCAP doesn't recognize your remote file system as a remote filesystem. A filesystem is considered remote filesystem by OpenSCAP if it starts either with // or it contains /: in mnt_fsname returned by getmntent(3). We have checked /etc/fstab in your sosreport and no filesystem with name is there. Could you tell us what way the remote system is mounted? Should we detect that the filesystem is remote? Dear Jan Černý, Thanks for response. ||Should we detect that the filesystem is remote? Do you mean remote session with Customer? We use bomgar tool to do it, Will you join the remote, If so I will arrange a remote session. What time/day suits you? How will you join the call? I customer is located in Europe/Paris? Dear Jan Černý, Response from Customer: Mounting takes place via the automounter. So if nothing has been mounted and oscap starts all defined remote mounts will be activated. In order to skip files or directories, my means of content modification, you have to edit the datastream and find all file_object and textfilecontent54 elements related to affected rules which could possibly match the path to your external filesystems. Once you will have all affected file_object elements, you will need to add a “filter” child element to each of them. For example: <ns4:filter action="exclude">oval:ssg-state_external_filesystems:ste:1</ns4:filter> This element is a reference to OVAL state. A single OVAL state should be created for all of them as a child element of ‘states’ element, for example like <ns4:file_state id="oval:ssg-state_external_filesystems:ste:1" version="1"> <ns4:path operation="pattern match">^/mnt/.*</ns4:path> </ns4:file_state> Please pay attention to namespaces - using something else than ns4 may be necessary depending on the context. Dear Matěj Týč, response from Customer: //Created By: Koert Gielen (3/13/2019 4:14 PM)// Could you please provide me with 1 full example on how to apply filters (for example to exclude directory /dll_linux)? I am not able (yet) to find the proper documentation on how to apply them. The only thing I found was https://bugzilla.redhat.com/show_bug.cgi?id=1598166 but the contents of this site won't help me any further. And what do you mean by "please pay attention to namespaces"? ///////////////////////////////////////////////// Created attachment 1543959 [details]
Full example of usage of filter element in an OVAL definition
Hi, > Could you please provide me with 1 full example on how to apply filters (for example to exclude directory /dll_linux)? I have added a full example as attachment 1543959 [details]. In the example, the filter element is used to exclude the /dll_directory from an OVAL object which matches the root filesystem. > I am not able (yet) to find the proper documentation on how to apply them. For documentation about usage of OVAL filters, see the filter section in OVAL language specification: https://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#filter > And what do you mean by "please pay attention to namespaces"? That means that by the OVAL specification the <filter> element should be in "http://oval.mitre.org/XMLSchema/oval-definitions-5" namespace. Also, the .*_state elements have to be in the same namespace as the corresponding .*_object namespace, for example the <file_state> element has to be in namespace "http://oval.mitre.org/XMLSchema/oval-definitions-5#unix" because <file_object> is in "http://oval.mitre.org/XMLSchema/oval-definitions-5#unix" namespace. To check if the OVAL is valid, you can run "oscap oval eval <oval_file>". **Customer's Update Created By: Koert Gielen (3/20/2019 4:19 PM)** See the attachment. And a few example files with wrong filenames: dll root@ferns openscap# ionice find /dll_linux /DLL_unix -name "Licences_Ismertet*" /DLL_unix/software/Advicesoft/2015/AS_289_DLL_01_FULL_v2/Advisesoft/Docs/AS_285_ADV/Licences_Ismertet?_285.xls /DLL_unix/software/Advicesoft/2015/AS_289_DLL_01_FULL_v2/Advisesoft/Docs/AS_286_ADV/Licences_Ismertet?_286.xls /DLL_unix/software/Advicesoft/2015/AS_289_DLL_01_FULL_v2/Advisesoft/Docs/AS_287_ADV/Licences_Ismertet?_287.xls /DLL_unix/software/Advicesoft/2015/AS_289_DLL_01_FULL_v2/Advisesoft/Docs/AS_288_ADV/Licences_Ismertet?_288.xls /DLL_unix/software/Advicesoft/2015/AS_289_DLL_01_FULL_v2/Advisesoft/Docs/AS_289_ADV/Licences_Ismertet?_289.xls *************************** We have been able to reproduce the issue thanks to the data that we got from the customer. It seems to manifest on certain type of automount configurations. We are now analysing possible ways of how to fix this issue, and how to test this kind of issues. Dear Matěj Týč, Once fix is identified and done, Please do send a oscap test package for customer to test in his environment. (This is requested by Customer) Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2341 |