Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1937472

Summary:	ocp4-cis scan reports FAIL for audit logforward check
Product:	OpenShift Container Platform	Reporter:	dtarabor
Component:	Compliance Operator	Assignee:	Juan Antonio Osorio <josorior>
Status:	CLOSED ERRATA	QA Contact:	Prashant Dhamdhere <pdhamdhe>
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	4.6	CC:	cruhm, josorior, mrogers, xiyuan
Target Milestone:	---
Target Release:	4.8.0
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	1940780 (view as bug list)		Environment:
Last Closed:	2021-07-07 11:29:56 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1940780, 1940784

Description dtarabor 2021-03-10 17:52:23 UTC

Description of problem:

We use the compliance operator to scan openshift compliance based on ocp4-cis benchmark. We configured clusterlogforward to send audit logs to remote SIEM server. However the ocp4-cis-audit-log-forwarding-enabled scan returns FAIL.
We assume, the check is not working properly, as it expects the audit inputRef in all pipelines of the logforwarding configuration. However, we have two pipelines, one for audit forward to SIEM and one for infra and app inputRef forwarding to local ES logStore.
The scan completes with PASS, when we add audit inputRef to all pipelines. But that is not the desired configuration.

How reproducible:

Every time

Steps to Reproduce:
1. configure logforwarding without inputRef
2. run compliance operator
3. fail result

Comment 5 Prashant Dhamdhere 2021-03-29 12:14:47 UTC

[Bug Verification]

Looks good. Now, the `ocp4-cis-audit-log-forwarding-enabled` rule scan returns result as PASS even though 
the audit inputRef adds in single pipeline.


Verified on:
4.8.0-0.nightly-2021-03-29-000904
compliance-operator.v0.1.29


$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-03-29-000904   True        False         6h12m   Cluster version is 4.8.0-0.nightly-2021-03-29-000904


$ oc get csv -w -nopenshift-operators-redhat
NAME                              DISPLAY                            VERSION    REPLACES   PHASE
elasticsearch-operator.5.0.1-30   OpenShift Elasticsearch Operator   5.0.1-30              Succeeded


$ oc get pods -nopenshift-operators-redhat
NAME                                      READY   STATUS    RESTARTS   AGE
elasticsearch-operator-85cfd8ffbd-mlsp7   1/1     Running   35         6h11m


$ oc get csv -w -nopenshift-logging
NAME                              DISPLAY                            VERSION    REPLACES   PHASE
cluster-logging.5.0.1-30          Red Hat OpenShift Logging          5.0.1-30              Succeeded
elasticsearch-operator.5.0.1-30   OpenShift Elasticsearch Operator   5.0.1-30              Succeeded


$ oc get pods -nopenshift-logging
NAME                                     READY   STATUS    RESTARTS   AGE
cluster-logging-operator-5b5bfb9-dsqgz   1/1     Running   0          6h11m


$ oc project openshift-logging
Now using project "openshift-logging" on server "https://api.aiyengar-48bv.qe.devcluster.openshift.com:6443".


$ oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/logging/clusterlogging/customresource-fluentd.yaml
clusterlogging.logging.openshift.io/instance created


$ oc get pods -w
NAME                                            READY   STATUS      RESTARTS   AGE
cluster-logging-operator-5b5bfb9-dsqgz          1/1     Running     0          6h23m
curator-1617016200-sbrvf                        0/1     Completed   0          10s
elasticsearch-cdm-55u1bkto-1-84b4d9d79b-jc5n6   2/2     Running     0          111s
fluentd-59wjs                                   1/1     Running     0          110s
fluentd-7xtpz                                   1/1     Running     0          110s
fluentd-c2nr8                                   1/1     Running     0          110s
fluentd-hn4cf                                   1/1     Running     0          110s
fluentd-jwqfp                                   1/1     Running     0          110s
fluentd-ppncj                                   1/1     Running     0          110s
kibana-787698bf66-774qx                         2/2     Running     0          77s
kibana-787698bf66-wxn8q                         2/2     Running     0          77s


$ oc create sa fluentdserver
serviceaccount/fluentdserver created


$ oc adm policy add-scc-to-user  privileged system:serviceaccount:openshift-logging:fluentdserver
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:privileged added: "fluentdserver"


$ oc create -f https://raw.githubusercontent.com/openshift/verification-tests/master/testdata/logging/clusterlogforwarder/fluentd/insecure/configmap.yaml
configmap/fluentdserver created


$ oc create -f https://raw.githubusercontent.com/openshift/verification-tests/master/testdata/logging/clusterlogforwarder/fluentd/insecure/deployment.yaml 
deployment.apps/fluentdserver created


$ oc expose deployment/fluentdserver
service/fluentdserver exposed



$ oc create -f - <<EOF 
> apiVersion: logging.openshift.io/v1
> kind: ClusterLogForwarder
> metadata:
>   name: instance
>   namespace: openshift-logging
> spec:
>   outputs:
>     - name: fluentd-created-by-user
>       type: fluentdForward
>       url: 'tcp://fluentdserver.openshift-logging.svc:24224'
>   pipelines:
>    - name: audit-logs 
>      inputRefs: 
>      - audit
>      outputRefs:
>      - fluentd-created-by-user 
>    - name: infra-logs 
>      inputRefs:
>      - infrastructure
>      - application
>      outputRefs:
>      - default 
> EOF
clusterlogforwarder.logging.openshift.io/instance created



$ oc project openshift-compliance
Now using project "openshift-compliance" on server "https://api.aiyengar-48bv.qe.devcluster.openshift.com:6443".


$ oc get csv
NAME                              DISPLAY                            VERSION    REPLACES   PHASE
compliance-operator.v0.1.29       Compliance Operator                0.1.29                Succeeded
elasticsearch-operator.5.0.1-30   OpenShift Elasticsearch Operator   5.0.1-30              Succeeded


$ oc get sub
NAME                            PACKAGE               SOURCE                CHANNEL
openshift-compliance-operator   compliance-operator   compliance-operator   4.8


$ oc get sub openshift-compliance-operator -o jsonpath='{.spec.channel}'
4.8


$ oc get pods
NAME                                              READY   STATUS    RESTARTS   AGE
compliance-operator-65654655b9-dhj8h              1/1     Running   0          10m
ocp4-openshift-compliance-pp-5d7dd59f77-bg4c5     1/1     Running   0          9m46s
rhcos4-openshift-compliance-pp-6674bff585-6btbf   1/1     Running   0          9m46s


$ oc create -f - << EOF
> apiVersion: compliance.openshift.io/v1alpha1
> kind: ScanSettingBinding
> metadata:
>   name: my-ssb-r
> profiles:
>   - name: ocp4-cis
>     kind: Profile
>     apiGroup: compliance.openshift.io/v1alpha1
>   - name: ocp4-cis-node
>     kind: Profile
>     apiGroup: compliance.openshift.io/v1alpha1
> settingsRef:
>   name: default
>   kind: ScanSetting
>   apiGroup: compliance.openshift.io/v1alpha1
> EOF
scansettingbinding.compliance.openshift.io/my-ssb-r created


$ oc get pods
NAME                                                    READY   STATUS      RESTARTS   AGE
aggregator-pod-ocp4-cis                                 0/1     Completed   0          2m46s
aggregator-pod-ocp4-cis-node-master                     0/1     Completed   0          2m42s
aggregator-pod-ocp4-cis-node-worker                     0/1     Completed   0          2m26s
compliance-operator-65654655b9-dhj8h                    1/1     Running     0          14m
ocp4-cis-api-checks-pod                                 0/2     Completed   0          3m27s
ocp4-openshift-compliance-pp-5d7dd59f77-bg4c5           1/1     Running     0          13m
openscap-pod-0514ee8c21ef32c9be924d834d86f725b9e99e03   0/2     Completed   0          3m27s
openscap-pod-079ac9a63b118feed5029017ba07d1778a7021b5   0/2     Completed   0          3m27s
openscap-pod-55c4c42844ad7a0f57cd8b6ba73f37ee504f88e0   0/2     Completed   0          3m27s
openscap-pod-8a7fe214292bd19012320de2dfa0c2ebd565dff1   0/2     Completed   0          3m26s
openscap-pod-dd0283db0d346003c87cd41e084db24e0350c71a   0/2     Completed   0          3m26s
openscap-pod-e86cac3cdf2738dd2f621990ad0c04c83b19a85b   0/2     Completed   0          3m26s
rhcos4-openshift-compliance-pp-6674bff585-6btbf         1/1     Running     0          13m


$  oc get suite
NAME       PHASE   RESULT
my-ssb-r   DONE    NON-COMPLIANT


$ oc get compliancecheckresult ocp4-cis-audit-log-forwarding-enabled
NAME                                    STATUS   SEVERITY
ocp4-cis-audit-log-forwarding-enabled   PASS     medium


$ oc project openshift-logging
Now using project "openshift-logging" on server "https://api.aiyengar-48bv.qe.devcluster.openshift.com:6443".


$ oc get csv
NAME                              DISPLAY                            VERSION    REPLACES   PHASE
cluster-logging.5.0.1-30          Red Hat OpenShift Logging          5.0.1-30              Succeeded
elasticsearch-operator.5.0.1-30   OpenShift Elasticsearch Operator   5.0.1-30              Succeeded



$ oc get pods
NAME                                            READY   STATUS      RESTARTS   AGE
cluster-logging-operator-5b5bfb9-dsqgz          1/1     Running     0          7h21m
curator-1617019200-7nbrj                        0/1     Completed   0          8m37s
elasticsearch-cdm-55u1bkto-1-84b4d9d79b-jc5n6   2/2     Running     0          60m
elasticsearch-im-app-1617019200-8g2qk           0/1     Completed   0          8m37s
elasticsearch-im-audit-1617019200-2dcpl         0/1     Completed   0          8m36s
elasticsearch-im-infra-1617019200-zp8pf         0/1     Completed   0          8m36s
fluentd-746xn                                   1/1     Running     0          55m
fluentd-8n6qp                                   1/1     Running     0          54m
fluentd-lxdqt                                   1/1     Running     0          55m
fluentd-nf7tk                                   1/1     Running     0          54m
fluentd-p9qht                                   1/1     Running     0          55m
fluentd-xn5gw                                   1/1     Running     0          54m
fluentdserver-5fb6ffb5d-tqfzc                   1/1     Running     0          56m
kibana-787698bf66-774qx                         2/2     Running     0          59m
kibana-787698bf66-wxn8q                         2/2     Running     0          59m



$ oc rsh fluentdserver-5fb6ffb5d-tqfzc
/ # grep "compliance-operator.v0.1.29" /fluentd/log/audit.log |tail -2
2021-03-29T12:02:27+00:00	k8s-audit.log	{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"info","auditID":"f716bd1f-5a49-477b-8f09-80386ba7c5d4","stage":"ResponseStarted","requestURI":"/apis/compliance.openshift.io/v1alpha1/namespaces/openshift-compliance/scansettings?allowWatchBookmarks=true&resourceVersion=181370&timeoutSeconds=389&watch=true","verb":"watch","user":{"username":"system:serviceaccount:openshift-compliance:compliance-operator","uid":"190e1168-89f4-4051-82b4-33fa8c84a8dc","groups":["system:serviceaccounts","system:serviceaccounts:openshift-compliance","system:authenticated"]},"sourceIPs":["10.0.211.4"],"userAgent":"compliance-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"scansettings","namespace":"openshift-compliance","apiGroup":"compliance.openshift.io","apiVersion":"v1alpha1"},"responseStatus":{"status":"Success","message":"Connection closed early","code":200},"requestReceivedTimestamp":"2021-03-29T12:02:27.087799Z","stageTimestamp":"2021-03-29T12:08:56.088989Z","annotations":{"authentication.k8s.io/legacy-token":"system:serviceaccount:openshift-compliance:compliance-operator","authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by RoleBinding \"compliance-operator.v0.1.29-compliance-operator-df6bf598/openshift-compliance\" of Role \"compliance-operator.v0.1.29-compliance-operator-df6bf598\" to ServiceAccount \"compliance-operator/openshift-compliance\""},"k8s_audit_level":"Metadata","message":null,"hostname":"ip-10-0-168-22.us-east-2.compute.internal","pipeline_metadata":{"collector":{"ipaddr4":"10.0.168.22","inputname":"fluent-plugin-systemd","name":"fluentd","received_at":"2021-03-29T12:08:56.090401+00:00","version":"1.7.4 1.6.0"}},"@timestamp":"2021-03-29T12:02:27.087799+00:00","viaq_index_name":"audit-write","viaq_msg_id":"ZjM1M2UwOGQtNTE3Ny00Mjg2LWIwMjQtM2E4ODZiM2IyMGM3","kubernetes":{}}
2021-03-29T12:02:27+00:00	k8s-audit.log	{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"info","auditID":"f716bd1f-5a49-477b-8f09-80386ba7c5d4","stage":"ResponseComplete","requestURI":"/apis/compliance.openshift.io/v1alpha1/namespaces/openshift-compliance/scansettings?allowWatchBookmarks=true&resourceVersion=181370&timeoutSeconds=389&watch=true","verb":"watch","user":{"username":"system:serviceaccount:openshift-compliance:compliance-operator","uid":"190e1168-89f4-4051-82b4-33fa8c84a8dc","groups":["system:serviceaccounts","system:serviceaccounts:openshift-compliance","system:authenticated"]},"sourceIPs":["10.0.211.4"],"userAgent":"compliance-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"scansettings","namespace":"openshift-compliance","apiGroup":"compliance.openshift.io","apiVersion":"v1alpha1"},"responseStatus":{"status":"Success","message":"Connection closed early","code":200},"requestReceivedTimestamp":"2021-03-29T12:02:27.087799Z","stageTimestamp":"2021-03-29T12:08:56.089070Z","annotations":{"authentication.k8s.io/legacy-token":"system:serviceaccount:openshift-compliance:compliance-operator","authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by RoleBinding \"compliance-operator.v0.1.29-compliance-operator-df6bf598/openshift-compliance\" of Role \"compliance-operator.v0.1.29-compliance-operator-df6bf598\" to ServiceAccount \"compliance-operator/openshift-compliance\""},"k8s_audit_level":"Metadata","message":null,"hostname":"ip-10-0-168-22.us-east-2.compute.internal","pipeline_metadata":{"collector":{"ipaddr4":"10.0.168.22","inputname":"fluent-plugin-systemd","name":"fluentd","received_at":"2021-03-29T12:08:56.090501+00:00","version":"1.7.4 1.6.0"}},"@timestamp":"2021-03-29T12:02:27.087799+00:00","viaq_index_name":"audit-write","viaq_msg_id":"ZjE3ZmVlMGUtMmZjNS00MzA0LTllNmEtZWU0MzQxYWUyODYw","kubernetes":{}}
/ # exit


$ oc get role compliance-operator.v0.1.29-compliance-operator-df6bf598 -nopenshift-compliance
NAME                                                       CREATED AT
compliance-operator.v0.1.29-compliance-operator-df6bf598   2021-03-29T11:52:33Z

Comment 9 errata-xmlrpc 2021-07-07 11:29:56 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Compliance Operator version 0.1.35 for OpenShift Container Platform 4.6-4.8), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2652