Bug 2133151

I noticed that the files in https://access.redhat.com/security/data/metrics/ds/ are having a bit different XML stuff , compared with the files in https://public.cyber.mil/stigs/scap/ or the files provided by scap-security-guide ( /usr/share/xml/scap/ssg/content/*.xml )


e.g. blocks like these don't exist there in the files from https://access.redhat.com/security/data/metrics/ds/ 


  <ds:data-stream id="scap_org.open-scap_datastream_from_xccdf_ssg-rhel7-xccdf-1.2.xml" scap-version="1.3" use-case="OTHER">
    <ds:dictionaries>
      <ds:component-ref id="scap_org.open-scap_cref_ssg-rhel7-cpe-dictionary.xml" xlink:href="#scap_org.open-scap_comp_ssg-rhel7-cpe-dictionary.xml">
        <cat:catalog>
          <cat:uri name="ssg-rhel7-cpe-oval.xml" uri="#scap_org.open-scap_cref_ssg-rhel7-cpe-oval.xml"/>
        </cat:catalog>
      </ds:component-ref>
    </ds:dictionaries>


But maybe i am on a wrong track and the issue is somewhere else i.e. perhaps some XML formatting is missing. I have checked with xmllint and i don't see any such issues and usually UI would have complained about invalid XML file type if that would be the case.

Error from foreman-proxy/proxy.log 


2022-10-08T12:43:42 95e06759 [I] Started POST /compliance/scap_content/policies 
2022-10-08T12:43:42 95e06759 [E] Error occurred: Failed to parse profiles
2022-10-08T12:43:42 95e06759 [W] Error details for Error occurred: Failed to parse profiles: <Exception>: Error occurred: Failed to parse profiles
2022-10-08T12:43:42 95e06759 [W] Error occurred: Failed to parse profiles: <Exception>: Error occurred: Failed to parse profiles
2022-10-08T12:43:42 95e06759 [I] Finished POST /compliance/scap_content/policies with 500 (547.66 ms)


From any working scap datastream files:

# grep "Profile id" /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_C2S">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_anssi_nt28_enhanced">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_anssi_nt28_high">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_anssi_nt28_intermediary">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_anssi_nt28_minimal">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_cis">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_cis_server_l1">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_cis_workstation_l1">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_cis_workstation_l2">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_cjis">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_cui">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_e8">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_hipaa">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_ncp">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_ospp">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_pci-dss">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_rhelh-stig">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_rhelh-vpp">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_rht-ccp">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_standard">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_stig">
      <xccdf-1.2:Profile id="xccdf_org.ssgproject.content_profile_stig_gui">



$ grep "Profile id" U_RHEL_8_V1R6_STIG_SCAP_1-2_Benchmark.xml
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-1_Classified">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-1_Public">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-1_Sensitive">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-2_Classified">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-2_Public">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-2_Sensitive">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-3_Classified">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-3_Public">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_MAC-3_Sensitive">
      <xccdf:Profile id="xccdf_mil.disa.stig_profile_CAT_I_Only">



For non-working ones i.e. the ones from https://access.redhat.com/security/data/metrics/ds/

$ grep "Profile id" com.redhat.rhsa-RHEL7.ds.xml -i
$ echo $?
1

$ grep "Profile id" com.redhat.rhsa-RHEL9.ds.xml -i
$ echo $?
1

So perhaps this is the issue i.e. missing profiles and Satellite can't handle it properly ?

fetch_profiles ( https://github.com/theforeman/foreman_openscap/blob/5.1-stable/app/models/foreman_openscap/scap_content.rb#L48-L52  ) :

uses 

profiles = api.fetch_policies_for_scap_content(scap_file)


fetch_policies_for_scap_content is defined in https://github.com/theforeman/foreman_openscap/blob/5.1-stable/app/lib/proxy_api/openscap.rb#L10-L12 and the "parse" function is defined in https://github.com/theforeman/foreman_openscap/blob/5.1-stable/app/controllers/policies_controller.rb#L28-L30 that uses "to_html"

to_html is defined in https://github.com/theforeman/foreman_openscap/blob/5.1-stable/app/models/foreman_openscap/policy.rb#L53-L65 and https://github.com/theforeman/foreman_openscap/blob/5.1-stable/app/models/foreman_openscap/policy.rb#L64 seems to be trying:

api.policy_html_guide(scap_content.scap_file, scap_content_profile.try(:profile_id))


And then as per "https://github.com/theforeman/smart_proxy_openscap/blob/master/lib/smart_proxy_openscap/profiles_parser.rb" we get to see "Error occurred: Failed to parse profiles"

Sayan, the input you're using is not valid. I believe that's OVAL definition. The SCAP integration with Satellite currently only supports compliance data streams. Both are in form of XML. We used to have some sort of validation, checking for the content, but it turned out 3rd party content wouldn't be considered valid anymore. Therefore the validation was removed and we only check the XML is valid. We generally accept any valid XML at this point.

I'd suggest to close this bug we NOTABUG and would recommend avoiding even opening the RFE for better validation, to avoid BZs like https://bugzilla.redhat.com/show_bug.cgi?id=2053478. If you think it's still valid, please open that for tracking purposes as a separate BZ.

Hi Marek,

I agree. 

I had to open the BZ because of what the customer had mentioned in the case i.e. 
~~
also my prior profiles I have downloaded form redhat only and it all worked.
~~

And the source as per him was https://www.redhat.com/security/data/oval/v2/ 

So later I requested any of such files which worked at past and he had shared the file ssg-rhel8-ds.xml . If I am not wrong, This file was never there in the link but it comes directly from the scap-security-guide package ( and should be accessible from /usr/share/xml/scap/ssg/content/*.xml of satellite ).

And then should be able to get all of them imported using the hammer or foreman-rake command as explained in https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/administering_red_hat_satellite/index#Loading_the_Default_OpenSCAP_Content_admin .

This is what I recommended on the case just a couple of minutes back. Let's leave this BZ open for some time and once the confusion have been cleared with the customer, I will close the BZ by myself. 


( lowering the severity to low )



-- Sayan

Hello Marek,

Any files from https://access.redhat.com/security/data/metrics/ds/ can be uploaded as Scap Content in Satellite 6.9 but that does not work on Sat 6.10 or 6.11. 

From that point of view, Should this bug not be a valid one to investigate further, or Is it an intentional change or perhaps an effect of the fix from https://bugzilla.redhat.com/show_bug.cgi?id=2053478 ?




-- Sayan

Yes, this is the impact of the BZ 2053478, we found out we can't do better validation than just making sure it's a valid XML document. I understand it may have caused a confusion when we announced the OVAL scanning functionality on 6.10 as a tech preview. However OVAL support is a very different type of functionality. Starting with 6.10, it can be enabled through settings as an experimental feature. But that has it's own "OVAL content" and "OVAL policy" objects. SCAP policy only deals with compliance data stream files.

Thanks for the quick reply.

This should be considered a big change in the behavior of Satellite 6.10.5 and above.


We should put some sort of notes somewhere in either the release notes or else in 

"7.3.3. Extra SCAP Content"  -> for 6.11 https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/administering_red_hat_satellite/index#Extra_SCAP_Content_admin

"7.2.3. Extra SCAP Content"  -> for 6.10 https://access.redhat.com/documentation/en-us/red_hat_satellite/6.10/html-single/administering_red_hat_satellite/index#extra-scap-content_admin


If you can suggest a suitable statement that we can provide there in the description of that section, I think we can convert this BZ into a doc BZ and have the same details mentioned there. 

It will be helpful for the customers who still expect to be able to use datastream files from https://access.redhat.com/security/data/metrics/ds/ or https://www.redhat.com/security/data/oval/v2/ or some other source which satellite will not be able to work with.

How about this:

Satellite 6.11 removed the additional validation of XML uploaded for SCAP content and Tailoring file, which prevented some valid XMLs from being used for this purpose. Satellite 6.11 and newer only checks for valid XML document but leaves the actual content verification on the user. User should only upload datastreams supported by oscap xccdf eval command.

*** Bug 2167937 has been marked as a duplicate of this bug. ***