Bug 2142929

Summary: Permission denied when try get instancestypes
Product: Container Native Virtualization (CNV) Reporter: Ohad <orevah>
Component: VirtualizationAssignee: Ohad <orevah>
Status: CLOSED ERRATA QA Contact: zhe peng <zpeng>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.12.0CC: dholler, nunnatsa
Target Milestone: ---   
Target Release: 4.12.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: CNV v4.12.0-714 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-01-24 13:42:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ohad 2022-11-15 14:35:23 UTC 
Description of problem:
When running oc adm must-gather --image=quay.io/kubevirt/must-gather -- /usr/bin/gather --instancetypes
I get the following output:

[must-gather      ] OUT Using must-gather plug-in image: quay.io/kubevirt/must-gather
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information:
ClusterID: bac2cac4-f347-4bd8-a5b4-5afe3c517d0d
ClusterVersion: Stable at "4.12.0-ec.5"
ClusterOperators:
	clusteroperator/kube-apiserver is not upgradeable because UnsupportedConfigOverridesUpgradeable: setting: [admission.pluginConfig.PodSecurity.configuration.apiVersion admission.pluginConfig.PodSecurity.configuration.defaults.audit admission.pluginConfig.PodSecurity.configuration.defaults.audit-version admission.pluginConfig.PodSecurity.configuration.defaults.enforce admission.pluginConfig.PodSecurity.configuration.defaults.enforce-version admission.pluginConfig.PodSecurity.configuration.defaults.warn admission.pluginConfig.PodSecurity.configuration.defaults.warn-version admission.pluginConfig.PodSecurity.configuration.kind]


[must-gather      ] OUT namespace/openshift-must-gather-kdpwl created
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-2kk9x created
[must-gather      ] OUT pod for plug-in image quay.io/kubevirt/must-gather created
[must-gather-n2wmj] POD 2022-11-15T12:28:39.976245081Z running gather_instancetypes
[must-gather-n2wmj] POD 2022-11-15T12:28:39.977810803Z /usr/bin/gather: line 167: /usr/bin/gather_instancetypes: Permission denied
[must-gather-n2wmj] POD 2022-11-15T12:28:39.978216846Z running logs
[must-gather-n2wmj] POD 2022-11-15T12:28:39.982116987Z + export BASE_COLLECTION_PATH=/must-gather
[must-gather-n2wmj] POD 2022-11-15T12:28:39.982116987Z + BASE_COLLECTION_PATH=/must-gather
[must-gather-n2wmj] POD 2022-11-15T12:28:39.982116987Z + NAMESPACE_FILE=/var/run/secrets/kubernetes.io/serviceaccount/namespace
[must-gather-n2wmj] POD 2022-11-15T12:28:39.982116987Z + [[ -f /var/run/secrets/kubernetes.io/serviceaccount/namespace ]]
[must-gather-n2wmj] POD 2022-11-15T12:28:40.014180888Z ++ sed -E 's|pod/([^ ]+).*|\1|'
[must-gather-n2wmj] POD 2022-11-15T12:28:40.014180888Z ++ grep '^pod'
[must-gather-n2wmj] POD 2022-11-15T12:28:40.014180888Z ++ oc status
[must-gather-n2wmj] POD 2022-11-15T12:28:43.030072231Z + POD=must-gather-n2wmj
[must-gather-n2wmj] POD 2022-11-15T12:28:43.045117383Z ++ cat /var/run/secrets/kubernetes.io/serviceaccount/namespace
[must-gather-n2wmj] POD 2022-11-15T12:28:43.045327286Z + oc logs --timestamps=true -n openshift-must-gather-kdpwl must-gather-n2wmj -c gather
[must-gather-n2wmj] OUT waiting for gather to complete
[must-gather-n2wmj] OUT downloading gather output
[must-gather-n2wmj] OUT receiving incremental file list
[must-gather-n2wmj] OUT ./
[must-gather-n2wmj] OUT must-gather.log
[must-gather-n2wmj] OUT 
[must-gather-n2wmj] OUT sent 46 bytes  received 515 bytes  224.40 bytes/sec
[must-gather-n2wmj] OUT total size is 974  speedup is 1.74
[must-gather      ] OUT namespace/openshift-must-gather-kdpwl deleted
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-2kk9x deleted


Reprinting Cluster State:
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information:
ClusterID: bac2cac4-f347-4bd8-a5b4-5afe3c517d0d
ClusterVersion: Stable at "4.12.0-ec.5"
ClusterOperators:
	clusteroperator/kube-apiserver is not upgradeable because UnsupportedConfigOverridesUpgradeable: setting: [admission.pluginConfig.PodSecurity.configuration.apiVersion admission.pluginConfig.PodSecurity.configuration.defaults.audit admission.pluginConfig.PodSecurity.configuration.defaults.audit-version admission.pluginConfig.PodSecurity.configuration.defaults.enforce admission.pluginConfig.PodSecurity.configuration.defaults.enforce-version admission.pluginConfig.PodSecurity.configuration.defaults.warn admission.pluginConfig.PodSecurity.configuration.defaults.warn-version admission.pluginConfig.PodSecurity.configuration.kind]
Comment 2 Kedar Bidarkar 2022-11-16 13:21:47 UTC 
While running must_gather we see the below message.

[must-gather-n2wmj] POD 2022-11-15T12:28:39.977810803Z /usr/bin/gather: line 167: /usr/bin/gather_instancetypes: Permission denied
Comment 3 Dominik Holler 2022-11-16 13:46:54 UTC 
thanks @nunnatsa
Comment 4 zhe peng 2022-11-24 05:35:08 UTC 
verify with build:
OCP-4.12.0-rc.1
CNV-v4.12.0-736

step:
1. run  cmd:
oc adm must-gather --image=quay.io/kubevirt/must-gather -- /usr/bin/gather --instancetypes

[must-gather      ] OUT Using must-gather plug-in image: quay.io/kubevirt/must-gather
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information.
ClusterID: 1b4d4c3f-cd36-406e-b913-a5abedb8b1a1
ClusterVersion: Stable at "4.12.0-rc.1"
ClusterOperators:
	All healthy and stable


[must-gather      ] OUT namespace/openshift-must-gather-xpp9f created
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-wmhp5 created
W1124 00:27:57.997556   17022 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (containers "gather", "copy" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (containers "gather", "copy" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or containers "gather", "copy" must set securityContext.runAsNonRoot=true), seccompProfile (pod or containers "gather", "copy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
[must-gather      ] OUT pod for plug-in image quay.io/kubevirt/must-gather created
[must-gather-z72c4] POD 2022-11-24T05:27:59.437305206Z running gather_instancetypes
[must-gather-z72c4] POD 2022-11-24T05:27:59.437305206Z inspecting virtualmachineinstancetype
[must-gather-z72c4] POD 2022-11-24T05:27:59.437305206Z inspecting virtulmachineclusterinstancetype
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z + export BASE_COLLECTION_PATH=/must-gather
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z + BASE_COLLECTION_PATH=/must-gather
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z + PROS=5
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z + resources=(virtualmachineinstancetype virtulmachineclusterinstancetype virtualmachinepreference virtualmachineclusterpreference)
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z + echo virtualmachineinstancetype virtulmachineclusterinstancetype virtualmachinepreference virtualmachineclusterpreference
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z + tr ' ' '\n'
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z + xargs -t '-I{}' -P 5 --max-args=1 sh -c 'echo "inspecting $1" && oc adm inspect --dest-dir ${BASE_COLLECTION_PATH} --all-namespaces $1' -- '{}'
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z sh -c echo "inspecting $1" && oc adm inspect --dest-dir ${BASE_COLLECTION_PATH} --all-namespaces $1 -- virtualmachineinstancetype 
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z sh -c echo "inspecting $1" && oc adm inspect --dest-dir ${BASE_COLLECTION_PATH} --all-namespaces $1 -- virtulmachineclusterinstancetype 
[must-gather-z72c4] POD 2022-11-24T05:27:59.437458073Z sh -c echo "inspecting $1" && oc adm inspect --dest-dir ${BASE_COLLECTION_PATH} --all-namespaces $1 -- virtualmachinepreference 
[must-gather-z72c4] POD 2022-11-24T05:27:59.443358251Z inspecting virtualmachinepreference
[must-gather-z72c4] POD 2022-11-24T05:27:59.443358251Z inspecting virtualmachineclusterpreference
[must-gather-z72c4] POD 2022-11-24T05:27:59.443417632Z sh -c echo "inspecting $1" && oc adm inspect --dest-dir ${BASE_COLLECTION_PATH} --all-namespaces $1 -- virtualmachineclusterpreference 
[must-gather-z72c4] POD 2022-11-24T05:27:59.811942359Z Wrote inspect data to /must-gather.
[must-gather-z72c4] POD 2022-11-24T05:27:59.860179899Z Wrote inspect data to /must-gather.
[must-gather-z72c4] POD 2022-11-24T05:28:00.730305742Z I1124 05:28:00.730038      17 request.go:665] Waited for 1.171848535s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/tektontasks.kubevirt.io/v1alpha1?timeout=32s
[must-gather-z72c4] POD 2022-11-24T05:28:00.981180248Z I1124 05:28:00.980949      16 request.go:665] Waited for 1.145741722s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/upload.cdi.kubevirt.io/v1alpha1?timeout=32s
[must-gather-z72c4] POD 2022-11-24T05:28:02.007613187Z Wrote inspect data to /must-gather.
[must-gather-z72c4] POD 2022-11-24T05:28:02.185880510Z error: the server doesn't have a resource type "virtulmachineclusterinstancetype"
[must-gather-z72c4] POD 2022-11-24T05:28:02.195460431Z running logs
[must-gather-z72c4] POD 2022-11-24T05:28:02.197617418Z + export BASE_COLLECTION_PATH=/must-gather
[must-gather-z72c4] POD 2022-11-24T05:28:02.197617418Z + BASE_COLLECTION_PATH=/must-gather
[must-gather-z72c4] POD 2022-11-24T05:28:02.197617418Z + NAMESPACE_FILE=/var/run/secrets/kubernetes.io/serviceaccount/namespace
[must-gather-z72c4] POD 2022-11-24T05:28:02.197655309Z + [[ -f /var/run/secrets/kubernetes.io/serviceaccount/namespace ]]
[must-gather-z72c4] POD 2022-11-24T05:28:02.198222042Z ++ oc status
[must-gather-z72c4] POD 2022-11-24T05:28:02.198345584Z ++ grep '^pod'
[must-gather-z72c4] POD 2022-11-24T05:28:02.200188470Z ++ sed -E 's|pod/([^ ]+).*|\1|'
[must-gather-z72c4] POD 2022-11-24T05:28:02.360896073Z + POD=must-gather-z72c4
[must-gather-z72c4] POD 2022-11-24T05:28:02.361197037Z ++ cat /var/run/secrets/kubernetes.io/serviceaccount/namespace
[must-gather-z72c4] POD 2022-11-24T05:28:02.363191699Z + oc logs --timestamps=true -n openshift-must-gather-xpp9f must-gather-z72c4 -c gather
[must-gather-z72c4] OUT waiting for gather to complete
[must-gather-z72c4] OUT downloading gather output
[must-gather-z72c4] OUT receiving incremental file list
[must-gather-z72c4] OUT ./
[must-gather-z72c4] OUT event-filter.html
[must-gather-z72c4] OUT must-gather.log
[must-gather-z72c4] OUT timestamp
[must-gather-z72c4] OUT 
[must-gather-z72c4] OUT sent 84 bytes  received 2,537 bytes  5,242.00 bytes/sec
[must-gather-z72c4] OUT total size is 6,998  speedup is 2.67
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-wmhp5 deleted
[must-gather      ] OUT namespace/openshift-must-gather-xpp9f deleted


When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information.
ClusterID: 1b4d4c3f-cd36-406e-b913-a5abedb8b1a1
ClusterVersion: Stable at "4.12.0-rc.1"
ClusterOperators:
	All healthy and stable


no Permission denied found, move to verified.
Comment 8 errata-xmlrpc 2023-01-24 13:42:07 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.12.0 Images security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0408