Bug 1940779

Summary: Provide better visibility into 'SKIP' scan result status as well as into OpenSCAP 'not applicable'
Product: OpenShift Container Platform Reporter: Prashant Dhamdhere <pdhamdhe>
Component: Compliance OperatorAssignee: Jakub Hrozek <jhrozek>
Status: CLOSED ERRATA QA Contact: Prashant Dhamdhere <pdhamdhe>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.6CC: akaris, jhrozek, josorior, mrogers, nkinder, xiyuan
Target Milestone: ---   
Target Release: 4.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1920577
: 1940783 (view as bug list) Environment:
Last Closed: 2021-03-31 21:50:42 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1920577    
Bug Blocks: 1940783    

Comment 4 Prashant Dhamdhere 2021-03-22 04:09:04 UTC
[Bug Verification]

Looks good to me. Now for the non-applicable rules, the user-visible status also shows NOT-APPLICABLE instead of SKIP

Verified on:
4.7.0-0.nightly-2021-03-21-181832
compliance-operator.v0.1.29


# oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.nightly-2021-03-21-181832   True        False         27m     Cluster version is 4.7.0-0.nightly-2021-03-21-181832


# oc get csv
NAME                          DISPLAY               VERSION   REPLACES   PHASE
compliance-operator.v0.1.29   Compliance Operator   0.1.29               Succeeded


# oc get sub openshift-compliance-operator -o jsonpath='{.spec.channel}'
4.7


# oc get pods
NAME                                              READY   STATUS    RESTARTS   AGE
compliance-operator-7d89b4c589-hlzrh              1/1     Running   0          7m37s
ocp4-openshift-compliance-pp-7f7d49575c-m6872     1/1     Running   0          6m37s
rhcos4-openshift-compliance-pp-85b5bf6c69-fft57   1/1     Running   0          6m37s


# oc patch scansetting default -p '{"debug":true}' --type='merge'
scansetting.compliance.openshift.io/default patched


# oc get -oyaml scansetting default |grep debug
debug: true
      f:debug: {}


# oc create -f - <<EOF
> apiVersion: compliance.openshift.io/v1alpha1
> kind: ScanSettingBinding 
> metadata:
>   name: my-companys-compliance-requirements
> profiles:
>   # Node checks
>   - name: ocp4-cis-node
>     kind: Profile
>     apiGroup: compliance.openshift.io/v1alpha1
>   # Cluster checks
>   - name: ocp4-cis
>     kind: Profile
>     apiGroup: compliance.openshift.io/v1alpha1
> settingsRef:
>   name: default
>   kind: ScanSetting
>   apiGroup: compliance.openshift.io/v1alpha1
> EOF
scansettingbinding.compliance.openshift.io/my-companys-compliance-requirements created


# oc get suite
NAME                                  PHASE   RESULT
my-companys-compliance-requirements   DONE    NON-COMPLIANT


# oc get pods
NAME                                                    READY   STATUS      RESTARTS   AGE
aggregator-pod-ocp4-cis                                 0/1     Completed   0          2m8s
aggregator-pod-ocp4-cis-node-master                     0/1     Completed   0          114s
aggregator-pod-ocp4-cis-node-worker                     0/1     Completed   0          118s
compliance-operator-7d89b4c589-hlzrh                    1/1     Running     0          13m
ocp4-cis-api-checks-pod                                 0/2     Completed   0          2m58s
ocp4-openshift-compliance-pp-7f7d49575c-m6872           1/1     Running     0          12m
openscap-pod-087832379ed87df937ac87295297347504affdb4   0/2     Completed   0          2m59s
openscap-pod-2019d34accbdfa5e7515264555806cc933a11b45   0/2     Completed   0          3m
openscap-pod-5bfc279f3ab09b94e6bb4b3030795fb488cd0005   0/2     Completed   0          2m59s
openscap-pod-5e7a9d31eeb1162bee95e52952db83a2fd073baa   0/2     Completed   0          2m59s
openscap-pod-c74301f4207e2b17c49e6e1197494e51333fd0d9   0/2     Completed   0          2m59s
rhcos4-openshift-compliance-pp-85b5bf6c69-fft57         1/1     Running     0          12m


# oc logs openscap-pod-5e7a9d31eeb1162bee95e52952db83a2fd073baa -c scanner | grep -A 10 "unique_ca"
Rule    xccdf_org.ssgproject.content_rule_etcd_unique_ca
I: oscap: Evaluating XCCDF rule 'xccdf_org.ssgproject.content_rule_etcd_unique_ca'.
I: oscap: Evaluating definition 'oval:ssg-installed_app_is_ocp4:def:1': Red Hat OpenShift Container Platform.
I: oscap: Definition 'oval:ssg-installed_app_is_ocp4:def:1' evaluated as false.
I: oscap: Evaluating definition 'oval:ssg-installed_app_is_ocp4_node:def:1': Red Hat OpenShift Container Platform Node.
I: oscap: Definition 'oval:ssg-installed_app_is_ocp4_node:def:1' evaluated as true.
I: oscap: Evaluating definition 'oval:ssg-node_is_ocp4_master_node:def:1': Node is Red Hat OpenShift Container Platform 4 Master Node.
I: oscap:   Evaluating file test 'oval:ssg-test_kube_api_pod_exists:tst:1': Testing if /etc/kubernetes/static-pod-resources/kube-apiserver-certs exists.
I: oscap:     Querying file object 'oval:ssg-object_kube_api_pod_exists:obj:1', flags: 0.
I: oscap:     Creating new syschar for file_object 'oval:ssg-object_kube_api_pod_exists:obj:1'.
I: oscap:     Switching probe to PROBE_OFFLINE_OWN mode.
I: oscap:     I will run file_probe_main:
--
I: oscap: Rule 'xccdf_org.ssgproject.content_rule_etcd_unique_ca' is not applicable.
Result  notapplicable


# oc get compliancecheckresult |grep "NAME\|ocp4-cis-node-worker-etcd-unique-ca"
NAME                                                                           STATUS           SEVERITY
ocp4-cis-node-worker-etcd-unique-ca                                            NOT-APPLICABLE   medium


# oc get compliancecheckresult -l compliance.openshift.io/check-status=NOT-APPLICABLE |head -5
NAME                                                                  STATUS           SEVERITY
ocp4-cis-node-worker-etcd-unique-ca                                   NOT-APPLICABLE   medium
ocp4-cis-node-worker-file-groupowner-controller-manager-kubeconfig    NOT-APPLICABLE   medium
ocp4-cis-node-worker-file-groupowner-etcd-data-dir                    NOT-APPLICABLE   medium
ocp4-cis-node-worker-file-groupowner-etcd-data-files                  NOT-APPLICABLE   medium


# oc get compliancecheckresult -l compliance.openshift.io/check-status=SKIP
No resources found in openshift-compliance namespace.

Comment 6 errata-xmlrpc 2021-03-31 21:50:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Compliance Operator version 0.1.29 for OpenShift Container Platform 4.7), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1022