Bug 2008223

can't reproduce the issue now:

[root@localhost ~]# oc version --client
Client Version: 4.10.0-0.nightly-2021-10-19-210507


[root@localhost oc adm must-gather -- /usr/bin/gather_audit_logs ; echo $?
[must-gather      ] OUT Using must-gather plug-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2af566cedaab422336ade987354e0221419d76283b43dac0354607c37592baa8
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information.
ClusterID: dd8b8dcc-9944-4a40-a9c3-a16ba54db839
ClusterVersion: Stable at "4.9.0"
ClusterOperators:
	All healthy and stable


[must-gather      ] OUT namespace/openshift-must-gather-g7xwm created
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-h6s8b created
[must-gather      ] OUT pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2af566cedaab422336ade987354e0221419d76283b43dac0354607c37592baa8 created
[must-gather-zrmwr] POD 2021-10-20T02:14:10.295728843Z WARNING: Collecting one or more audit logs on ALL masters in your cluster. This could take a large amount of time.
[must-gather-zrmwr] POD 2021-10-20T02:14:10.433260440Z INFO: Started  downloading openshift-apiserver/audit.log from ip-10-0-145-240.us-east-2.compute.internal
......
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information.
ClusterID: dd8b8dcc-9944-4a40-a9c3-a16ba54db839
ClusterVersion: Stable at "4.9.0"
ClusterOperators:
	All healthy and stable


0

$ oc version
Client Version: 4.10.0-0.nightly-2021-10-16-173656
Server Version: 4.10.0-0.nightly-2021-10-20-193037
Kubernetes Version: v1.22.1+d767194

$ oc patch apiserver cluster -p '{"spec": {"audit": {"profile": "None"}}}' --type merge

After the kube-apiserver degradation is finished.
$ oc get pods -n openshift-kube-apiserver -l apiserver --show-labels

$ oc adm must-gather -- /usr/bin/gather_audit_logs;echo $?
$ oc adm must-gather -- /usr/bin/gather_audit_logs;echo $?

[must-gather      ] OUT Using must-gather plug-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6bdc74183cf2e2632b0ee40ff6939ce97ee222576eab2308434784587273a637
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information.
ClusterID: 74d849f6-c326-41fc-b754-c9816955df23
ClusterVersion: Stable at "4.10.0-0.nightly-2021-10-20-193037"
ClusterOperators:
	All healthy and stable


[must-gather      ] OUT namespace/openshift-must-gather-p5clv created
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-z2l2x created
[must-gather      ] OUT pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6bdc74183cf2e2632b0ee40ff6939ce97ee222576eab2308434784587273a637 created
[must-gather-jz6fq] POD 2021-10-21T03:17:28.472216228Z ERROR: To raise a Red Hat support request, it is required to set the top level audit policy to
[must-gather-jz6fq] POD 2021-10-21T03:17:28.472216228Z        Default, WriteRequestBodies, or AllRequestBodies to generate audit log events that can
[must-gather-jz6fq] POD 2021-10-21T03:17:28.472216228Z        be analyzed by support. Try 'oc edit apiservers' and set 'spec.audit.profile' back to
[must-gather-jz6fq] POD 2021-10-21T03:17:28.472216228Z        "Default" and reproduce the issue while gathering audit logs. You can use "--force" to
[must-gather-jz6fq] POD 2021-10-21T03:17:28.472216228Z        override this error.
[must-gather-jz6fq] OUT waiting for gather to complete
[must-gather-jz6fq] OUT downloading gather output
[must-gather-jz6fq] OUT receiving incremental file list
[must-gather-jz6fq] OUT ./
[must-gather-jz6fq] OUT 
[must-gather-jz6fq] OUT sent 27 bytes  received 41 bytes  19.43 bytes/sec
[must-gather-jz6fq] OUT total size is 0  speedup is 0.00
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-z2l2x deleted
[must-gather      ] OUT namespace/openshift-must-gather-p5clv deleted


When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information.
ClusterID: 74d849f6-c326-41fc-b754-c9816955df23
ClusterVersion: Stable at "4.10.0-0.nightly-2021-10-20-193037"
ClusterOperators:
	All healthy and stable

0

per the Doc
https://github.com/openshift/enhancements/blob/master/enhancements/kube-apiserver/audit-policy.md, the command will return with an error (non-zero exit code), so assign it back.

> $ oc adm must-gather -- /usr/bin/gather_audit_logs;echo $?

`;` is being interpreted by your local command line and not passed to container, check the actual result of the script by rsh-ing into it and verifying it manually. 

Moving back to qa.

$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2021-10-25-190146   True        False         13h     Cluster version is 4.10.0-0.nightly-2021-10-25-190146

In terminal A, make the must-gather pod running,
$ oc adm must-gather -- tail -f /dev/null

In terminal B, run /usr/bin/gather_audit_logs and check the output and return code.
$ oc rsh -n openshift-must-gather-rktb7 must-gather-mlk9b
Defaulted container "gather" out of: gather, copy
sh-4.4# /usr/bin/gather_audit_logs
ERROR: To raise a Red Hat support request, it is required to set the top level audit policy to
       Default, WriteRequestBodies, or AllRequestBodies to generate audit log events that can
       be analyzed by support. Try 'oc edit apiservers' and set 'spec.audit.profile' back to
       "Default" and reproduce the issue while gathering audit logs. You can use "--force" to
       override this error.
sh-4.4# echo  $?
1

From above, as expected, so move the bug VERIFIED.