Bug 1848268
| Summary: | The enabling debug flag of the logcollector does not work until the fileintegrities CR recreates | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | xiyuan |
| Component: | File Integrity Operator | Assignee: | Matt Rogers <mrogers> |
| Status: | CLOSED ERRATA | QA Contact: | xiyuan |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 4.6 | CC: | jhrozek, josorior, nkinder, pdhamdhe |
| Target Milestone: | --- | ||
| Target Release: | 4.6.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-10-27 16:08:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Could you re-verify this? This should have been addressed by the move to the daemon instead of the dedicated workloads. Hi Juan & Matt,
Verification pass. Also works with parameter gracePeriod.
$ oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.6.0-0.nightly-2020-08-16-072105 True False 21h Cluster version is 4.6.0-0.nightly-2020-08-16-072105
When debug is false:
$ oc create -f - << EOF
> apiVersion: fileintegrity.openshift.io/v1alpha1
> kind: FileIntegrity
> metadata:
> name: example-fileintegrity
> namespace: openshift-file-integrity
> spec:
> # Change to debug: true to enable more verbose logging from the logcollector
> # container in the aide pods
> debug: false
> config: {}
> EOF
fileintegrity.fileintegrity.openshift.io/example-fileintegrity created
$ oc logs pod/aide-ds-example-fileintegrity-4d29x
Starting the AIDE runner daemon
running aide check
aide check returned status 7
Created log configMap 'aide-ds-example-fileintegrity-juzhao-osp-2lp4l-master-0'
running aide check
aide check returned status 7
Created log configMap 'aide-ds-example-fileintegrity-juzhao-osp-2lp4l-master-0'
Update debug to true:
$ oc apply -f - <<EOF
apiVersion: fileintegrity.openshift.io/v1alpha1
kind: FileIntegrity
metadata:
name: example-fileintegrity
namespace: openshift-file-integrity
spec:
debug: true
config:
name: myconf
namespace: openshift-file-integrity
key: aide-conf
gracePeriod: 15
EOF
Warning: oc apply should be used on resource created by either oc create --save-config or oc apply
fileintegrity.fileintegrity.openshift.io/example-fileintegrity configured
$ oc get pod
NAME READY STATUS RESTARTS AGE
aide-ds-example-fileintegrity-27p8f 1/1 Running 0 2m21s
aide-ds-example-fileintegrity-8cb4w 1/1 Running 0 2m26s
aide-ds-example-fileintegrity-9cch8 1/1 Running 0 2m15s
aide-ds-example-fileintegrity-b8gck 1/1 Running 0 2m25s
aide-ds-example-fileintegrity-hnkvg 1/1 Running 0 2m19s
aide-ds-example-fileintegrity-v8h44 1/1 Running 0 2m26s
aide-ds-example-fileintegrity-w99p2 1/1 Running 0 2m17s
file-integrity-operator-65db875847-88sxm 1/1 Running 0 20h
juzhao-osp-2lp4l-master-0-rmholdoff 0/1 Completed 0 20h
juzhao-osp-2lp4l-master-1-rmholdoff 0/1 Completed 0 20h
juzhao-osp-2lp4l-master-2-rmholdoff 0/1 Completed 0 20h
juzhao-osp-2lp4l-worker-4w666-rmholdoff 0/1 Completed 0 14h
juzhao-osp-2lp4l-worker-9f9z5-rmholdoff 0/1 Completed 0 20h
juzhao-osp-2lp4l-worker-klbk5-rmholdoff 0/1 Completed 0 20h
juzhao-osp-2lp4l-worker-q8dqz-rmholdoff 0/1 Completed 0 20h
juzhao-osp-2lp4l-worker-qrmrj-rmholdoff 0/1 Completed 0 20h
$ oc logs pod/aide-ds-example-fileintegrity-27p8f
Starting the AIDE runner daemon
debug: No scan result available
debug: aide files locked by aideLoop
running aide check
aide check returned status 6
debug: aide files unlocked by aideLoop
debug: aide files locked by logCollectorMainLoop
debug: Integrity check failed, continuing to collect log file
debug: Opening /hostroot/etc/kubernetes/aide.log
debug: aide files unlocked by logCollectorMainLoop
debug: Getting FileIntegrity openshift-file-integrity/example-fileintegrity
debug: added 0 changed 2 removed 1
Created log configMap 'aide-ds-example-fileintegrity-juzhao-osp-2lp4l-worker-q8dqz'
debug: aide files locked by aideLoop
running aide check
aide check returned status 6
debug: aide files unlocked by aideLoop
debug: aide files locked by logCollectorMainLoop
debug: Integrity check failed, continuing to collect log file
debug: Opening /hostroot/etc/kubernetes/aide.log
debug: aide files unlocked by logCollectorMainLoop
debug: Getting FileIntegrity openshift-file-integrity/example-fileintegrity
debug: added 0 changed 2 removed 1
Created log configMap 'aide-ds-example-fileintegrity-juzhao-osp-2lp4l-worker-q8dqz'
debug: aide files locked by aideLoop
running aide check
aide check returned status 6
debug: aide files unlocked by aideLoop
debug: aide files locked by logCollectorMainLoop
debug: Integrity check failed, continuing to collect log file
debug: Opening /hostroot/etc/kubernetes/aide.log
debug: aide files unlocked by logCollectorMainLoop
debug: Getting FileIntegrity openshift-file-integrity/example-fileintegrity
debug: added 0 changed 2 removed 1
Created log configMap 'aide-ds-example-fileintegrity-juzhao-osp-2lp4l-worker-q8dqz'
debug: aide files locked by aideLoop
running aide check
aide check returned status 6
debug: aide files unlocked by aideLoop
debug: aide files locked by logCollectorMainLoop
debug: Integrity check failed, continuing to collect log file
debug: Opening /hostroot/etc/kubernetes/aide.log
debug: aide files unlocked by logCollectorMainLoop
debug: Getting FileIntegrity openshift-file-integrity/example-fileintegrity
debug: added 0 changed 2 removed 1
Created log configMap 'aide-ds-example-fileintegrity-juzhao-osp-2lp4l-worker-q8dqz'
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196 |
Description of problem: The enabling debug flag of the logcollector does not work until the fileintegrities CR recreates Version-Release number of selected component (if applicable): 4.5.0-0.nightly-2020-06-16-205659 How reproducible: Always Steps to Reproduce: 1. install the fileintegrity operator oc login -u kubeadmin -p <pw> oc create -f file-integrity-operator/deploy/ns.yaml oc project openshift-file-integrity for l in `ls -1 file-integrity-operator/deploy/crds/*crd.yaml`; do oc create -f $l; done oc create -f file-integrity-operator/deploy/ oc create -f file-integrity-operator/deploy/crds/fileintegrity.openshift.io_v1alpha1_fileintegrity_cr.yaml 2. check the debug value of logcollector 3. create configmap and apply the configmap with "debug: true": $ oc create configmap myconf --from-file=aide-conf=file-integrity-operator/aide.conf.rhel8 $ oc apply -f - <<EOF apiVersion: fileintegrity.openshift.io/v1alpha1 kind: FileIntegrity metadata: name: example-fileintegrity namespace: openshift-file-integrity spec: debug: true config: name: myconf namespace: openshift-file-integrity key: aide-conf gracePeriod: 15 EOF Actual results: 2. for step 2, the result is: $ oc describe pod/aide-ds-example-fileintegrity-5s679 ... logcollector: Container ID: cri-o://92eff460f2650137ae8d09287e504e1b48371c6fc987cef435ef348d1dffefc1 Image: quay.io/file-integrity-operator/file-integrity-operator:latest Image ID: quay.io/file-integrity-operator/file-integrity-operator@sha256:60a35963ec7be42ea7e33c59ca946b679444f8cfd3a9551935ecd7131abdf1f9 Port: <none> Host Port: <none> Args: logcollector --file=/hostroot/etc/kubernetes/aide.log --config-map-prefix=aide-ds-example-fileintegrity --owner=example-fileintegrity --namespace=openshift-file-integrity --interval=10 --debug=false $ oc logs pod/aide-ds-example-fileintegrity-5s679 -c logcollector Starting the file-integrity log collector Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' 3. after step 3, the aide re-init, the debug flag is false in the logcollector container in all the aide-ds pods. And there is no debug info in the logcollector container in the logs of all aide-ds pods. $ oc describe pod/aide-ds-example-fileintegrity-9br5g ... logcollector: Container ID: cri-o://4ba8c7e633b89fa10964943f68b1989a363fb26f68a36b074739f18789f05094 Image: quay.io/file-integrity-operator/file-integrity-operator:latest Image ID: quay.io/file-integrity-operator/file-integrity-operator@sha256:60a35963ec7be42ea7e33c59ca946b679444f8cfd3a9551935ecd7131abdf1f9 Port: <none> Host Port: <none> Args: logcollector --file=/hostroot/etc/kubernetes/aide.log --config-map-prefix=aide-ds-example-fileintegrity --owner=example-fileintegrity --namespace=openshift-file-integrity --interval=10 --debug=false --<<debug flag $ oc logs pod/aide-ds-example-fileintegrity-9br5g -c logcollector Starting the file-integrity log collector Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Created OK configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' Expected results: After step 3, the debug flag should be true in the logcollector container in all the aide-ds pods. And there should be debug info in the logcollector container in the logs of all aide-ds pods. Additional info: If the user delete the fileintegrities and recreate it. the debug flag is true in the logcollector container in all the aide-ds pods. And there is debug info in the logcollector container in the logs of any aide-ds pod. $ oc delete fileintegrities/example-fileintegrity fileintegrity.fileintegrity.openshift.io "example-fileintegrity" deleted [xiyuan@MiWiFi-R3G-srv securitycompliance]$ oc apply -f - <<EOF apiVersion: fileintegrity.openshift.io/v1alpha1 kind: FileIntegrity metadata: name: example-fileintegrity namespace: openshift-file-integrity spec: debug: true config: name: myconf namespace: openshift-file-integrity key: aide-conf gracePeriod: 15 EOF fileintegrity.fileintegrity.openshift.io/example-fileintegrity created $ oc logs pod/aide-ds-example-fileintegrity-9br5g -c logcollector Starting the file-integrity log collector ... logcollector: Container ID: cri-o://6b670bdf504603c716228042e2f7d6a7800e478eaa8e0646ff9ed08fc43b98cf Image: quay.io/file-integrity-operator/file-integrity-operator:latest Image ID: quay.io/file-integrity-operator/file-integrity-operator@sha256:60a35963ec7be42ea7e33c59ca946b679444f8cfd3a9551935ecd7131abdf1f9 Port: <none> Host Port: <none> Args: logcollector --file=/hostroot/etc/kubernetes/aide.log --config-map-prefix=aide-ds-example-fileintegrity --owner=example-fileintegrity --namespace=openshift-file-integrity --interval=15 --debug=true --<<debug flag $ oc logs pod/aide-ds-example-fileintegrity-6cgp7 -c logcollector Starting the file-integrity log collector debug: Waiting for /hostroot/etc/kubernetes/aide.latest-result.log debug: File '/hostroot/etc/kubernetes/aide.latest-result.log' found debug: Integrity check failed, continuing to collect log file debug: Waiting for /hostroot/etc/kubernetes/aide.log debug: File '/hostroot/etc/kubernetes/aide.log' found debug: Creating configMap 'aide-ds-example-fileintegrity-ip-10-0-58-148.us-east-2.compute.internal' to collect logs debug: Getting FileIntegrity openshift-file-integrity/example-fileintegrity debug: added 1 changed 1 removed 1