Bug 1631087 - Cannot see basic audit log
Summary: Cannot see basic audit log
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Master
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.9.z
Assignee: Maciej Szulik
QA Contact: ge liu
URL:
Whiteboard:
Depends On:
Blocks: 1632154 1632155
TreeView+ depends on / blocked
 
Reported: 2018-09-19 20:46 UTC by Renato Puccini
Modified: 2019-07-09 07:52 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Default log format for audit was set to json. Consequence: Audit log was always printed using json format. Fix: Allow setting log format as specified in master-config.yaml Result: Audit log contains values per configured log format.
Clone Of:
: 1632154 1632155 (view as bug list)
Environment:
Last Closed: 2018-11-20 03:12:03 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2908 None None None 2018-11-20 03:12:44 UTC

Description Renato Puccini 2018-09-19 20:46:25 UTC
Created attachment 1484915 [details]
master-config.yaml

Description of problem:
Setting the audit file as the basic mode following in the docs, resulted in having advanced audit logs (json).

Version-Release number of selected component (if applicable):
~$ rpm -aq | grep openshift
atomic-openshift-3.9.40-1.git.0.0c9824a.el7.x86_64
atomic-openshift-node-3.9.40-1.git.0.0c9824a.el7.x86_64
openshift-ansible-playbooks-3.9.40-1.git.0.188c954.el7.noarch
atomic-openshift-docker-excluder-3.9.40-1.git.0.0c9824a.el7.noarch
atomic-openshift-utils-3.9.40-1.git.0.188c954.el7.noarch
atomic-openshift-master-3.9.40-1.git.0.0c9824a.el7.x86_64
openshift-ansible-3.9.40-1.git.0.188c954.el7.noarch
openshift-ansible-docs-3.9.40-1.git.0.188c954.el7.noarch
atomic-openshift-sdn-ovs-3.9.40-1.git.0.0c9824a.el7.x86_64
atomic-openshift-excluder-3.9.40-1.git.0.0c9824a.el7.noarch
openshift-ansible-roles-3.9.40-1.git.0.188c954.el7.noarch
atomic-openshift-clients-3.9.40-1.git.0.0c9824a.el7.x86_64

How reproducible:
https://access.redhat.com/solutions/1748893
https://docs.openshift.com/container-platform/3.9/install_config/master_node_configuration.html#master-node-config-audit-config

Steps to Reproduce:
1.Add the following into the master-config.yaml of all masters
auditConfig:
  auditFilePath: "/var/log/audit-ocp.log"
  enabled: true
  maximumFileRetentionDays: 10
  maximumFileSizeMegabytes: 10
  maximumRetainedFiles: 10
2. Restart master services
3. Resulted expected : 
AUDIT: id="5c3b8227-4af9-4322-8a71-542231c3887b" ip="127.0.0.1" method="GET" user="admin" as="<self>" asgroups="<lookup>" namespace="default" uri="/api/v1/namespaces/default/pods"
AUDIT: id="5c3b8227-4af9-4322-8a71-542231c3887b" response="200"

Actual results:
 tail -n100 /var/log/audit-ocp.log 
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-09-19T20:41:19Z"},"level":"Metadata","timestamp":"2018-09-19T20:41:19Z","auditID":"fa89ca70-4609-4182-9fac-f94571562123","stage":"ResponseComplete","requestURI":"/apis/apiregistration.k8s.io/v1beta1/apiservices/v1.apps.openshift.io/status","verb":"update","user":{"username":"system:apiserver","uid":"529ec0a1-ac41-48a3-aa32-005bfd253424","groups":["system:masters"]},"sourceIPs":["127.0.0.1"],"objectRef":{"resource":"apiservices","name":"v1.apps.openshift.io","uid":"4b8ba55c-bab8-11e8-a78b-005056aea42f","apiGroup":"apiregistration.k8s.io","apiVersion":"v1beta1","resourceVersion":"12","subresource":"status"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2018-09-19T20:41:19.609099Z","stageTimestamp":"2018-09-19T20:41:19.611612Z"}
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-09-19T20:41:19Z"},"level":"Metadata","timestamp":"2018-09-19T20:41:19Z","auditID":"fc950075-fed5-45cb-920b-fbb26d7df35d","stage":"ResponseComplete","requestURI":"/apis/apiregistration.k8s.io/v1beta1/apiservices/v1beta1.apiextensions.k8s.io/status","verb":"update","user":{"username":"system:apiserver","uid":"529ec0a1-ac41-48a3-aa32-005bfd253424","groups":["system:masters"]},"sourceIPs":["127.0.0.1"],"objectRef":{"resource":"apiservices","name":"v1beta1.apiextensions.k8s.io","uid":"4b8b9f86-bab8-11e8-a78b-005056aea42f","apiGroup":"apiregistration.k8s.io","apiVersion":"v1beta1","resourceVersion":"11","subresource":"status"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2018-09-19T20:41:19.609509Z","stageTimestamp":"2018-09-19T20:41:19.611808Z"}
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-09-19T20:41:19Z"},"level":"Metadata","timestamp":"2018-09-19T20:41:19Z","auditID":"9815770d-23e5-4f55-b189-cf4d8b27379d","stage":"RequestReceived","requestURI":"/apis/apiregistration.k8s.io/v1beta1/apiservices/v1beta2.apps/status","verb":"update","user":{"username":"system:apiserver","uid":"529ec0a1-ac41-48a3-aa32-005bfd253424","groups":["system:masters"]},"sourceIPs":["127.0.0.1"],"objectRef":{"resource":"apiservices","name":"v1beta2.apps","apiGroup":"apiregistration.k8s.io","apiVersion":"v1beta1","subresource":"status"},"requestReceivedTimestamp":"2018-09-19T20:41:19.612197Z","stageTimestamp":"2018-09-19T20:41:19.612197Z"}
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-09-19T20:41:19Z"},"level":"Metadata","timestamp":"2018-09-19T20:41:19Z","auditID":"0773f3a5-d3df-4640-8c5d-9fdbad8e6714","stage":"RequestReceived","requestURI":"/apis/apiregistration.k8s.io/v1beta1/apiservices/v1.autoscaling/status","verb":"update","user":{"username":"system:apiserver","uid":"529ec0a1-ac41-48a3-aa32-005bfd253424","groups":["system:masters"]},"sourceIPs":["127.0.0.1"],"objectRef":{"resource":"apiservices","name":"v1.autoscaling","apiGroup":"apiregistration.k8s.io","apiVersion":"v1beta1","subresource":"status"},"requestReceivedTimestamp":"2018-09-19T20:41:19.612596Z","stageTimestamp":"2018-09-19T20:41:19.612596Z"}
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-09-19T20:41:19Z"},"level":"Metadata","timestamp":"2018-09-19T20:41:19Z","auditID":"96c42d95-1efa-4e25-8662-70a817a43aa6","stage":"ResponseComplete","requestURI":"/apis/apiregistration.k8s.io/v1beta1/apiservices/v1.storage.k8s.io/status","verb":"update","user":{"username":"system:apiserver","uid":"529ec0a1-ac41-48a3-aa32-005bfd253424","groups":["system:masters"]},"sourceIPs":["127.0.0.1"],"objectRef":{"resource":"apiservices","name":"v1.storage.k8s.io","uid":"4ba299ac-bab8-11e8-a78b-005056aea42f","apiGroup":"apiregistration.k8s.io","apiVersion":"v1beta1","resourceVersion":"44","subresource":"status"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2018-09-19T20:41:19.610941Z","stageTimestamp":"2018-09-19T20:41:19.613207Z"}


Expected results:
Expecting basic audit logs, instead getting advanced audit logs

Additional info:

Comment 4 ge liu 2018-10-15 04:52:31 UTC
Failed to verify it with latest ocp vesion:

oc v3.9.47
kubernetes v1.9.1+a0ce1bc657
features: Basic-Auth GSSAPI Kerberos SPNEGO

basic audit config in master-config.yaml:

auditConfig:
  auditFilePath: "/etc/origin/master/audit-ocp.log"
  enabled: true
  maximumFileRetentionDays: 10
  maximumFileSizeMegabytes: 10
  maximumRetainedFiles: 10


audit-ocp.log:

{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-10-15T04:48:54Z"},"level":"Metadata","timestamp":"2018-10-15T04:48:54Z","auditID":"f96a325c-8b85-4c2e-bc7b-d623fad28a15","stage":"RequestReceived","requestURI":"/api/v1/namespaces/kube-service-catalog/configmaps/service-catalog-controller-manager","verb":"update","user":{"username":"system:serviceaccount:kube-service-catalog:service-catalog-controller","uid":"aaecac90-d02a-11e8-b631-0e95e51c6da0","groups":["system:serviceaccounts","system:serviceaccounts:kube-service-catalog","system:authenticated"]},"sourceIPs":["10.129.0.6"],"objectRef":{"resource":"configmaps","namespace":"kube-service-catalog","name":"service-catalog-controller-manager","apiVersion":"v1"},"requestReceivedTimestamp":"2018-10-15T04:48:54.646813Z","stageTimestamp":"2018-10-15T04:48:54.646813Z"}

Comment 5 Maciej Szulik 2018-10-15 10:19:36 UTC
Ge liu you need to explicitly specify logFormat parameter, if you don't we default to json for backwards compatibility. So this is working as expected.

Comment 6 ge liu 2018-10-15 11:32:36 UTC
Verified in ocp:

openshift v3.9.47
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.16

master-config.yaml:

auditConfig:
  auditFilePath: "/etc/origin/master/1audit-ocp.log"
  logFormat: legacy
  enabled: true
  maximumFileRetentionDays: 10
  maximumFileSizeMegabytes: 10
  maximumRetainedFiles: 10


2018-10-15T07:30:33.97529349-04:00 AUDIT: id="6c7b2a34-2a03-4b54-82f1-b382d0ebcb0d" stage="RequestReceived" ip="172.18.13.11" method="list" user="system:node:ip-172-18-13-11.ec2.internal" groups="\"system:nodes\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="<none>" uri="/apis/network.openshift.io/v1/egressnetworkpolicies?limit=500&resourceVersion=0" response="<deferred>"

Comment 7 ge liu 2018-10-15 12:25:18 UTC
@maszulik, as we discussed on irc, I will file a new doc bug to trace the doc issues, thx

Comment 8 openshift-github-bot 2018-10-15 21:59:17 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/2a2b7f254f858d83b4ad038f8db6cb923501a943
Merge pull request #21081 from soltysh/bug1631087

Bug 1632155 - Accept logFormat when passed to audit config

Comment 10 errata-xmlrpc 2018-11-20 03:12:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2908


Note You need to log in before you can comment on or make changes to this bug.