Description of problem: On an aws cluster-- Downloaded latest installer/oc $ oc version Client Version: 4.5.0-0.ci-2020-06-19-043726 Server Version: 4.5.0-0.ci-2020-06-19-043726 Kubernetes Version: v1.18.3 [pcameron@new-host-14 iii (log01)]$ oc adm must-gather [must-gather ] OUT Using must-gather plugin-in image: registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472 [must-gather ] OUT namespace/openshift-must-gather-d5bzb created [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-q7rtr created [must-gather ] OUT pod for plug-in image registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472 created [must-gather-7tjx5] OUT gather did not start: timed out waiting for the condition [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-q7rtr deleted [must-gather ] OUT namespace/openshift-must-gather-d5bzb deleted error: gather did not start for pod must-gather-7tjx5: timed out waiting for the condition Version-Release number of selected component (if applicable): How reproducible: always. shows up in ci runs as well Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Can you provide more details about the pod or re-run the command with --v=10 ?
$ oc adm must-gather --v=10 -- /usr/bin/gather_network_ovn_trace I0619 08:45:43.779495 8768 loader.go:375] Config loaded from file: /home/pcameron/data/projects/src/github.com/openshift/cluster-network-operator/MY-OVN-1/auth/kubeconfig I0619 08:45:43.780411 8768 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/apis/image.openshift.io/v1/namespaces/openshift/imagestreams/must-gather' I0619 08:45:43.953588 8768 round_trippers.go:443] GET https://api.pcameron0619a.devcluster.openshift.com:6443/apis/image.openshift.io/v1/namespaces/openshift/imagestreams/must-gather 200 OK in 173 milliseconds I0619 08:45:43.953633 8768 round_trippers.go:449] Response Headers: I0619 08:45:43.953665 8768 round_trippers.go:452] Audit-Id: a57d9cf1-ac97-431f-be16-b5fbc5f54d7c I0619 08:45:43.953689 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:45:43.953711 8768 round_trippers.go:452] Cache-Control: no-store I0619 08:45:43.953733 8768 round_trippers.go:452] Content-Type: application/json I0619 08:45:43.953754 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:45:37 GMT I0619 08:45:43.953774 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:45:43.953795 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:45:43.953815 8768 round_trippers.go:452] Content-Length: 1522 I0619 08:45:43.955688 8768 request.go:1068] Response Body: {"kind":"ImageStream","apiVersion":"image.openshift.io/v1","metadata":{"name":"must-gather","namespace":"openshift","selfLink":"/apis/image.openshift.io/v1/namespaces/openshift/imagestreams/must-gather","uid":"f27abdbc-128d-4bb8-b957-974ddf07440d","resourceVersion":"30812","generation":2,"creationTimestamp":"2020-06-19T12:08:36Z","managedFields":[{"manager":"cluster-version-operator","operation":"Update","apiVersion":"image.openshift.io/v1","time":"2020-06-19T12:42:21Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:tags":{".":{},"k:{\"name\":\"latest\"}":{".":{},"f:annotations":{},"f:from":{".":{},"f:kind":{},"f:name":{}},"f:generation":{},"f:importPolicy":{".":{},"f:scheduled":{}},"f:name":{},"f:referencePolicy":{".":{},"f:type":{}}}}}}}]},"spec":{"lookupPolicy":{"local":false},"tags":[{"name":"latest","annotations":null,"from":{"kind":"DockerImage","name":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472"},"generation":2,"importPolicy":{"scheduled":true},"referencePolicy":{"type":"Source"}}]},"status":{"dockerImageRepository":"image-registry.openshift-image-registry.svc:5000/openshift/must-gather","tags":[{"tag":"latest","items":[{"created":"2020-06-19T12:09:39Z","dockerImageReference":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","image":"sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","generation":2}]}]}} [must-gather ] OUT Using must-gather plugin-in image: registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472 I0619 08:45:43.958814 8768 request.go:1068] Request Body: {"kind":"Namespace","apiVersion":"v1","metadata":{"generateName":"openshift-must-gather-","creationTimestamp":null,"labels":{"openshift.io/run-level":"0"},"annotations":{"oc.openshift.io/command":"oc adm must-gather"}},"spec":{},"status":{}} I0619 08:45:43.958921 8768 round_trippers.go:423] curl -k -v -XPOST -H "Content-Type: application/json" -H "Accept: application/json, */*" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces' I0619 08:45:43.997395 8768 round_trippers.go:443] POST https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces 201 Created in 38 milliseconds I0619 08:45:43.997419 8768 round_trippers.go:449] Response Headers: I0619 08:45:43.997429 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:45:43.997436 8768 round_trippers.go:452] Content-Type: application/json I0619 08:45:43.997444 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:45:43.997450 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:45:43.997457 8768 round_trippers.go:452] Content-Length: 779 I0619 08:45:43.997464 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:45:37 GMT I0619 08:45:43.997470 8768 round_trippers.go:452] Audit-Id: 7abe3b1e-5e0d-4732-8973-5696ea3818f3 I0619 08:45:43.999370 8768 request.go:1068] Response Body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"openshift-must-gather-6vtq8","generateName":"openshift-must-gather-","selfLink":"/api/v1/namespaces/openshift-must-gather-6vtq8","uid":"f859c775-3f93-43fb-80f3-93c56d29c869","resourceVersion":"31903","creationTimestamp":"2020-06-19T12:45:37Z","labels":{"openshift.io/run-level":"0"},"annotations":{"oc.openshift.io/command":"oc adm must-gather"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:oc.openshift.io/command":{}},"f:generateName":{},"f:labels":{".":{},"f:openshift.io/run-level":{}}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} [must-gather ] OUT namespace/openshift-must-gather-6vtq8 created I0619 08:45:44.000027 8768 request.go:1068] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"generateName":"must-gather-","creationTimestamp":null,"annotations":{"oc.openshift.io/command":"oc adm must-gather"}},"subjects":[{"kind":"ServiceAccount","name":"default","namespace":"openshift-must-gather-6vtq8"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"cluster-admin"}} I0619 08:45:44.000074 8768 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings' I0619 08:45:44.045379 8768 round_trippers.go:443] POST https://api.pcameron0619a.devcluster.openshift.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings 201 Created in 45 milliseconds I0619 08:45:44.045430 8768 round_trippers.go:449] Response Headers: I0619 08:45:44.045448 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:45:44.045468 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:45:44.045480 8768 round_trippers.go:452] Content-Length: 928 I0619 08:45:44.045497 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:45:37 GMT I0619 08:45:44.045511 8768 round_trippers.go:452] Audit-Id: 3614e76a-0d2e-4cc7-9c79-0b916b30491f I0619 08:45:44.045526 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:45:44.045540 8768 round_trippers.go:452] Content-Type: application/json I0619 08:45:44.047214 8768 request.go:1068] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"must-gather-2dw8m","generateName":"must-gather-","selfLink":"/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/must-gather-2dw8m","uid":"90306c1a-32f0-44c4-bd25-420a241c0a3d","resourceVersion":"31916","creationTimestamp":"2020-06-19T12:45:37Z","annotations":{"oc.openshift.io/command":"oc adm must-gather"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:oc.openshift.io/command":{}},"f:generateName":{}},"f:roleRef":{"f:apiGroup":{},"f:kind":{},"f:name":{}},"f:subjects":{}}}]},"subjects":[{"kind":"ServiceAccount","name":"default","namespace":"openshift-must-gather-6vtq8"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"cluster-admin"}} [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-2dw8m created I0619 08:45:44.052889 8768 request.go:1068] Request Body: {"kind":"Pod","apiVersion":"v1","metadata":{"generateName":"must-gather-","creationTimestamp":null,"labels":{"app":"must-gather"}},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}}],"initContainers":[{"name":"gather","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/usr/bin/gather_network_ovn_trace"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"}]}],"containers":[{"name":"copy","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"}]}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"tolerations":[{"operator":"Exists"}]},"status":{}} I0619 08:45:44.053042 8768 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8/pods' I0619 08:45:44.110257 8768 round_trippers.go:443] POST https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8/pods 201 Created in 57 milliseconds I0619 08:45:44.110308 8768 round_trippers.go:449] Response Headers: I0619 08:45:44.110322 8768 round_trippers.go:452] Content-Type: application/json I0619 08:45:44.110334 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:45:44.110346 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:45:44.110361 8768 round_trippers.go:452] Content-Length: 3029 I0619 08:45:44.110374 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:45:37 GMT I0619 08:45:44.110390 8768 round_trippers.go:452] Audit-Id: 9fe59beb-b2d0-4311-9d32-cf80d86bce70 I0619 08:45:44.110404 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:45:44.113727 8768 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-45s7d","generateName":"must-gather-","namespace":"openshift-must-gather-6vtq8","selfLink":"/api/v1/namespaces/openshift-must-gather-6vtq8/pods/must-gather-45s7d","uid":"0837be2d-03c2-4a94-a7f6-85606866bf22","resourceVersion":"31928","creationTimestamp":"2020-06-19T12:45:37Z","labels":{"app":"must-gather"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-b8gxv","secret":{"secretName":"default-token-b8gxv","defaultMode":420}}],"initContainers":[{"name":"gather","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/usr/bin/gather_network_ovn_trace"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-b8gxv","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-b8gxv","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","qosClass":"BestEffort"}} [must-gather ] OUT pod for plug-in image registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472 created I0619 08:45:44.128012 8768 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8/pods/must-gather-45s7d' I0619 08:45:44.164934 8768 round_trippers.go:443] GET https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8/pods/must-gather-45s7d 200 OK in 36 milliseconds I0619 08:45:44.164960 8768 round_trippers.go:449] Response Headers: I0619 08:45:44.164968 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:45:37 GMT I0619 08:45:44.164975 8768 round_trippers.go:452] Audit-Id: cf3961a8-b08f-45b5-863c-d63fbba0f813 I0619 08:45:44.164982 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:45:44.164990 8768 round_trippers.go:452] Content-Type: application/json I0619 08:45:44.165000 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:45:44.165023 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:45:44.167803 8768 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-45s7d","generateName":"must-gather-","namespace":"openshift-must-gather-6vtq8","selfLink":"/api/v1/namespaces/openshift-must-gather-6vtq8/pods/must-gather-45s7d","uid":"0837be2d-03c2-4a94-a7f6-85606866bf22","resourceVersion":"31935","creationTimestamp":"2020-06-19T12:45:37Z","labels":{"app":"must-gather"},"managedFields":[{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:initContainerStatuses":{},"f:startTime":{}}}},{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-b8gxv","secret":{"secretName":"default-token-b8gxv","defaultMode":420}}],"initContainers":[{"name":"gather","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/usr/bin/gather_network_ovn_trace"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-b8gxv","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-b8gxv","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"ip-10-0-139-163.us-east-2.compute.internal","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z","reason":"ContainersNotInitialized","message":"containers with incomplete status: [gather]"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z"}],"hostIP":"10.0.139.163","startTime":"2020-06-19T12:45:37Z","initContainerStatuses":[{"name":"gather","state":{"waiting":{"reason":"PodInitializing"}},"lastState":{},"ready":false,"restartCount":0,"image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","imageID":""}],"containerStatuses":[{"name":"copy","state":{"waiting":{"reason":"PodInitializing"}},"lastState":{},"ready":false,"restartCount":0,"image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","imageID":"","started":false}],"qosClass":"BestEffort"}} I0619 08:46:44.168859 8768 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8/pods/must-gather-45s7d' I0619 08:46:44.208769 8768 round_trippers.go:443] GET https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8/pods/must-gather-45s7d 200 OK in 39 milliseconds I0619 08:46:44.208795 8768 round_trippers.go:449] Response Headers: I0619 08:46:44.208801 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:46:44.208809 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:46:44.208815 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:46:37 GMT I0619 08:46:44.208821 8768 round_trippers.go:452] Audit-Id: 1654efcb-44da-4fd1-a03d-aca20561fb85 I0619 08:46:44.208827 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:46:44.208832 8768 round_trippers.go:452] Content-Type: application/json I0619 08:46:44.211196 8768 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-45s7d","generateName":"must-gather-","namespace":"openshift-must-gather-6vtq8","selfLink":"/api/v1/namespaces/openshift-must-gather-6vtq8/pods/must-gather-45s7d","uid":"0837be2d-03c2-4a94-a7f6-85606866bf22","resourceVersion":"32261","creationTimestamp":"2020-06-19T12:45:37Z","labels":{"app":"must-gather"},"annotations":{"k8s.ovn.org/pod-networks":"{\"default\":{\"ip_addresses\":[\"10.131.0.16/23\"],\"mac_address\":\"f6:0a:65:83:00:11\",\"gateway_ips\":[\"10.131.0.1\"],\"ip_address\":\"10.131.0.16/23\",\"gateway_ip\":\"10.131.0.1\"}}","k8s.v1.cni.cncf.io/network-status":"[{\n \"name\": \"ovn-kubernetes\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.131.0.16\"\n ],\n \"mac\": \"f6:0a:65:83:00:11\",\n \"default\": true,\n \"dns\": {}\n}]","k8s.v1.cni.cncf.io/networks-status":"[{\n \"name\": \"ovn-kubernetes\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.131.0.16\"\n ],\n \"mac\": \"f6:0a:65:83:00:11\",\n \"default\": true,\n \"dns\": {}\n}]"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}},{"manager":"ovnkube","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:k8s.ovn.org/pod-networks":{}}}}},{"manager":"multus","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:40Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:46:35Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:initContainerStatuses":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.131.0.16\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-b8gxv","secret":{"secretName":"default-token-b8gxv","defaultMode":420}}],"initContainers":[{"name":"gather","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/usr/bin/gather_network_ovn_trace"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-b8gxv","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-b8gxv","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"ip-10-0-139-163.us-east-2.compute.internal","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z","reason":"ContainersNotInitialized","message":"containers with incomplete status: [gather]"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-19T12:45:37Z"}],"hostIP":"10.0.139.163","podIP":"10.131.0.16","podIPs":[{"ip":"10.131.0.16"}],"startTime":"2020-06-19T12:45:37Z","initContainerStatuses":[{"name":"gather","state":{"waiting":{"reason":"CreateContainerError","message":"container create failed: time=\"2020-06-19T12:46:24Z\" level=error msg=\"container_linux.go:348: starting container process caused \\\"exec: \\\\\\\"/usr/bin/gather_network_ovn_trace\\\\\\\": stat /usr/bin/gather_network_ovn_trace: no such file or directory\\\"\"\ncontainer_linux.go:348: starting container process caused \"exec: \\\"/usr/bin/gather_network_ovn_trace\\\": stat /usr/bin/gather_network_ovn_trace: no such file or directory\"\n"}},"lastState":{},"ready":false,"restartCount":0,"image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","imageID":""}],"containerStatuses":[{"name":"copy","state":{"waiting":{"reason":"PodInitializing"}},"lastState":{},"ready":false,"restartCount":0,"image":"registry.svc.ci.openshift.org/ocp/4.5-2020-06-19-043726@sha256:ef4afb2bc8d802294401b3a112b49d4f5b5727498e2a75f2181f612168ba8472","imageID":"","started":false}],"qosClass":"BestEffort"}} [must-gather-45s7d] OUT gather did not start: timed out waiting for the condition I0619 08:46:44.211694 8768 request.go:1068] Request Body: {"kind":"DeleteOptions","apiVersion":"rbac.authorization.k8s.io/v1"} I0619 08:46:44.211753 8768 round_trippers.go:423] curl -k -v -XDELETE -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/must-gather-2dw8m' I0619 08:46:44.258063 8768 round_trippers.go:443] DELETE https://api.pcameron0619a.devcluster.openshift.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/must-gather-2dw8m 200 OK in 46 milliseconds I0619 08:46:44.258101 8768 round_trippers.go:449] Response Headers: I0619 08:46:44.258128 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:46:44.258143 8768 round_trippers.go:452] Content-Length: 218 I0619 08:46:44.258153 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:46:37 GMT I0619 08:46:44.258160 8768 round_trippers.go:452] Audit-Id: 71fb980a-7a67-43ba-bb29-b87debc8f8c4 I0619 08:46:44.258166 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:46:44.258173 8768 round_trippers.go:452] Content-Type: application/json I0619 08:46:44.258204 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:46:44.260278 8768 request.go:1068] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"must-gather-2dw8m","group":"rbac.authorization.k8s.io","kind":"clusterrolebindings","uid":"90306c1a-32f0-44c4-bd25-420a241c0a3d"}} [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-2dw8m deleted I0619 08:46:44.260399 8768 request.go:1068] Request Body: {"kind":"DeleteOptions","apiVersion":"v1"} I0619 08:46:44.260456 8768 round_trippers.go:423] curl -k -v -XDELETE -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/711c56a" 'https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8' I0619 08:46:44.300449 8768 round_trippers.go:443] DELETE https://api.pcameron0619a.devcluster.openshift.com:6443/api/v1/namespaces/openshift-must-gather-6vtq8 200 OK in 39 milliseconds I0619 08:46:44.300476 8768 round_trippers.go:449] Response Headers: I0619 08:46:44.300486 8768 round_trippers.go:452] Date: Fri, 19 Jun 2020 12:46:37 GMT I0619 08:46:44.300492 8768 round_trippers.go:452] Audit-Id: b7cf9321-b4c6-4b2f-8f65-74a94f05ea8e I0619 08:46:44.300499 8768 round_trippers.go:452] Cache-Control: no-cache, private I0619 08:46:44.300506 8768 round_trippers.go:452] Content-Type: application/json I0619 08:46:44.300513 8768 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: 28afc46b-0982-4537-a5af-3cda1611f621 I0619 08:46:44.300520 8768 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: d6ec7dc1-599d-4495-987b-9c8660234180 I0619 08:46:44.300527 8768 round_trippers.go:452] Content-Length: 1271 I0619 08:46:44.304209 8768 request.go:1068] Response Body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"openshift-must-gather-6vtq8","generateName":"openshift-must-gather-","selfLink":"/api/v1/namespaces/openshift-must-gather-6vtq8","uid":"f859c775-3f93-43fb-80f3-93c56d29c869","resourceVersion":"32279","creationTimestamp":"2020-06-19T12:45:37Z","deletionTimestamp":"2020-06-19T12:46:37Z","labels":{"openshift.io/run-level":"0"},"annotations":{"oc.openshift.io/command":"oc adm must-gather","openshift.io/sa.scc.mcs":"s0:c25,c5","openshift.io/sa.scc.supplemental-groups":"1000610000/10000","openshift.io/sa.scc.uid-range":"1000610000/10000"},"managedFields":[{"manager":"cluster-policy-controller","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:openshift.io/sa.scc.mcs":{},"f:openshift.io/sa.scc.supplemental-groups":{},"f:openshift.io/sa.scc.uid-range":{}}}}},{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-19T12:45:37Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:oc.openshift.io/command":{}},"f:generateName":{},"f:labels":{".":{},"f:openshift.io/run-level":{}}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}} [must-gather ] OUT namespace/openshift-must-gather-6vtq8 deleted F0619 08:46:44.304321 8768 helpers.go:115] error: gather did not start for pod must-gather-45s7d: timed out waiting for the condition
oc adm must-gather is also failing on ci https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_release/8654/rehearse-8654-pull-ci-openshift-cluster-network-operator-master-e2e-ovn-step-registry/1273585759516889088/artifacts/e2e-ovn-step-registry/gather-must-gather/container-logs/test.log Running must-gather... error: gather did not start for pod must-gather-ck2r8: timed out waiting for the condition error: failed to execute wrapped command: exit status 1 https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_release/8654/rehearse-8654-pull-ci-openshift-cluster-network-operator-master-e2e-ovn-step-registry/1273585759516889088/artifacts/e2e-ovn-step-registry/gather-network-trace/container-logs/test.log Gathering network ovn artifacts ... [must-gather ] OUT the server could not find the requested resource (get imagestreams.image.openshift.io must-gather) [must-gather ] OUT [must-gather ] OUT Using must-gather plugin-in image: quay.io/openshift/origin-must-gather:latest [must-gather ] OUT namespace/openshift-must-gather-zmqxq created [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-nwzxt created [must-gather ] OUT pod for plug-in image quay.io/openshift/origin-must-gather:latest created [must-gather-bjzpx] OUT gather did not start: timed out waiting for the condition [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-nwzxt deleted error: gather did not start for pod must-gather-bjzpx: timed out waiting for the condition [must-gather ] OUT namespace/openshift-must-gather-zmqxq deleted error: failed to execute wrapped command: exit status 1
same with 4.5.0-rc2 # ./oc version Client Version: 4.5.0-rc.2 Server Version: 4.5.0-rc.2 Kubernetes Version: v1.18.3+91d0edd # # ./oc adm must-gather --dest-dir=./mg -v10 I0620 20:00:49.144211 18453 loader.go:375] Config loaded from file: /root/rhv/45-rc2/cluster/auth/kubeconfig I0620 20:00:49.157789 18453 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/apis/image.openshift.io/v1/namespaces/openshift/imagestreams/must-gather' I0620 20:00:49.195566 18453 round_trippers.go:443] GET https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/apis/image.openshift.io/v1/namespaces/openshift/imagestreams/must-gather 200 OK in 37 milliseconds I0620 20:00:49.195622 18453 round_trippers.go:449] Response Headers: I0620 20:00:49.195634 18453 round_trippers.go:452] Audit-Id: ae5a9d23-1894-44b7-b3ce-f9f9ac00f37d I0620 20:00:49.195645 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:00:49.195655 18453 round_trippers.go:452] Cache-Control: no-store I0620 20:00:49.195665 18453 round_trippers.go:452] Content-Type: application/json I0620 20:00:49.195676 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:00:49 GMT I0620 20:00:49.195686 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:00:49.195696 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:00:49.195707 18453 round_trippers.go:452] Content-Length: 1434 I0620 20:00:49.195805 18453 request.go:1068] Response Body: {"kind":"ImageStream","apiVersion":"image.openshift.io/v1","metadata":{"name":"must-gather","namespace":"openshift","selfLink":"/apis/image.openshift.io/v1/namespaces/openshift/imagestreams/must-gather","uid":"407c66da-2f41-45be-ba1f-10b95698dcbe","resourceVersion":"21238","generation":2,"creationTimestamp":"2020-06-20T17:46:17Z","managedFields":[{"manager":"cluster-version-operator","operation":"Update","apiVersion":"image.openshift.io/v1","time":"2020-06-20T17:55:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:tags":{".":{},"k:{\"name\":\"latest\"}":{".":{},"f:annotations":{},"f:from":{".":{},"f:kind":{},"f:name":{}},"f:generation":{},"f:importPolicy":{".":{},"f:scheduled":{}},"f:name":{},"f:referencePolicy":{".":{},"f:type":{}}}}}}}]},"spec":{"lookupPolicy":{"local":false},"tags":[{"name":"latest","annotations":null,"from":{"kind":"DockerImage","name":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8"},"generation":2,"importPolicy":{"scheduled":true},"referencePolicy":{"type":"Source"}}]},"status":{"dockerImageRepository":"","tags":[{"tag":"latest","items":[{"created":"2020-06-20T17:47:04Z","dockerImageReference":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","image":"sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","generation":2}]}]}} [must-gather ] OUT Using must-gather plugin-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8 I0620 20:00:49.242935 18453 request.go:1068] Request Body: {"kind":"Namespace","apiVersion":"v1","metadata":{"generateName":"openshift-must-gather-","creationTimestamp":null,"labels":{"openshift.io/run-level":"0"},"annotations":{"oc.openshift.io/command":"oc adm must-gather"}},"spec":{},"status":{}} I0620 20:00:49.243038 18453 round_trippers.go:423] curl -k -v -XPOST -H "Content-Type: application/json" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" -H "Accept: application/json, */*" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces' I0620 20:00:49.257329 18453 round_trippers.go:443] POST https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces 201 Created in 14 milliseconds I0620 20:00:49.257381 18453 round_trippers.go:449] Response Headers: I0620 20:00:49.257393 18453 round_trippers.go:452] Audit-Id: 68704aff-19eb-4ba4-9d1d-4c4a65c58132 I0620 20:00:49.257404 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:00:49.257414 18453 round_trippers.go:452] Content-Type: application/json I0620 20:00:49.257425 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:00:49.257435 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:00:49.257446 18453 round_trippers.go:452] Content-Length: 779 I0620 20:00:49.257456 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:00:49 GMT I0620 20:00:49.257525 18453 request.go:1068] Response Body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"openshift-must-gather-k5xf7","generateName":"openshift-must-gather-","selfLink":"/api/v1/namespaces/openshift-must-gather-k5xf7","uid":"e5a37071-01d5-4a10-8168-858d72b70035","resourceVersion":"23257","creationTimestamp":"2020-06-20T18:00:49Z","labels":{"openshift.io/run-level":"0"},"annotations":{"oc.openshift.io/command":"oc adm must-gather"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:49Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:oc.openshift.io/command":{}},"f:generateName":{},"f:labels":{".":{},"f:openshift.io/run-level":{}}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} [must-gather ] OUT namespace/openshift-must-gather-k5xf7 created I0620 20:00:49.278257 18453 request.go:1068] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"generateName":"must-gather-","creationTimestamp":null,"annotations":{"oc.openshift.io/command":"oc adm must-gather"}},"subjects":[{"kind":"ServiceAccount","name":"default","namespace":"openshift-must-gather-k5xf7"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"cluster-admin"}} I0620 20:00:49.278464 18453 round_trippers.go:423] curl -k -v -XPOST -H "Content-Type: application/json" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" -H "Accept: application/json, */*" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings' I0620 20:00:49.683215 18453 round_trippers.go:443] POST https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings 201 Created in 404 milliseconds I0620 20:00:49.683250 18453 round_trippers.go:449] Response Headers: I0620 20:00:49.683262 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:00:49.683272 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:00:49.683283 18453 round_trippers.go:452] Content-Length: 928 I0620 20:00:49.683293 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:00:49 GMT I0620 20:00:49.683304 18453 round_trippers.go:452] Audit-Id: 5bc3c37a-a712-4f9b-bfdf-c3d60acb32f6 I0620 20:00:49.683314 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:00:49.683325 18453 round_trippers.go:452] Content-Type: application/json I0620 20:00:49.683384 18453 request.go:1068] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"must-gather-s4gfx","generateName":"must-gather-","selfLink":"/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/must-gather-s4gfx","uid":"4631e0ce-125a-44e6-ad14-38719c6e6200","resourceVersion":"23262","creationTimestamp":"2020-06-20T18:00:49Z","annotations":{"oc.openshift.io/command":"oc adm must-gather"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2020-06-20T18:00:49Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:oc.openshift.io/command":{}},"f:generateName":{}},"f:roleRef":{"f:apiGroup":{},"f:kind":{},"f:name":{}},"f:subjects":{}}}]},"subjects":[{"kind":"ServiceAccount","name":"default","namespace":"openshift-must-gather-k5xf7"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"cluster-admin"}} [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-s4gfx created I0620 20:00:49.702563 18453 request.go:1068] Request Body: {"kind":"Pod","apiVersion":"v1","metadata":{"generateName":"must-gather-","creationTimestamp":null,"labels":{"app":"must-gather"}},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}}],"initContainers":[{"name":"gather","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/usr/bin/gather"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"}]}],"containers":[{"name":"copy","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"}]}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"tolerations":[{"operator":"Exists"}]},"status":{}} I0620 20:00:49.702702 18453 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods' I0620 20:00:49.735561 18453 round_trippers.go:443] POST https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods 500 Internal Server Error in 32 milliseconds I0620 20:00:49.735634 18453 round_trippers.go:449] Response Headers: I0620 20:00:49.735649 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:00:49.735660 18453 round_trippers.go:452] Content-Length: 328 I0620 20:00:49.735671 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:00:49 GMT I0620 20:00:49.735682 18453 round_trippers.go:452] Audit-Id: a328a0fd-729b-4bb3-8db8-f03ed7950c74 I0620 20:00:49.735692 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:00:49.735703 18453 round_trippers.go:452] Content-Type: application/json I0620 20:00:49.735714 18453 round_trippers.go:452] Retry-After: 1 I0620 20:00:49.735724 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:00:49.735858 18453 request.go:907] Got a Retry-After 1s response for attempt 1 to https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods I0620 20:00:50.736457 18453 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" -H "Content-Type: application/json" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods' I0620 20:00:50.748959 18453 round_trippers.go:443] POST https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods 201 Created in 12 milliseconds I0620 20:00:50.749018 18453 round_trippers.go:449] Response Headers: I0620 20:00:50.749044 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:00:50 GMT I0620 20:00:50.749059 18453 round_trippers.go:452] Audit-Id: 064cf1cd-ebc0-492b-a395-886be2d9378d I0620 20:00:50.749070 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:00:50.749086 18453 round_trippers.go:452] Content-Type: application/json I0620 20:00:50.749126 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:00:50.749148 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:00:50.749164 18453 round_trippers.go:452] Content-Length: 3049 I0620 20:00:50.749253 18453 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-84z7z","generateName":"must-gather-","namespace":"openshift-must-gather-k5xf7","selfLink":"/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z","uid":"0aaeba9c-fc56-40b5-9d9e-343fd674a579","resourceVersion":"23297","creationTimestamp":"2020-06-20T18:00:50Z","labels":{"app":"must-gather"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:50Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-42m4w","secret":{"secretName":"default-token-42m4w","defaultMode":420}}],"initContainers":[{"name":"gather","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/usr/bin/gather"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"imagePullSecrets":[{"name":"default-dockercfg-t9vwg"}],"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","qosClass":"BestEffort"}} [must-gather ] OUT pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8 created I0620 20:00:50.760244 18453 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z' I0620 20:00:50.771518 18453 round_trippers.go:443] GET https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z 200 OK in 11 milliseconds I0620 20:00:50.771547 18453 round_trippers.go:449] Response Headers: I0620 20:00:50.771560 18453 round_trippers.go:452] Content-Type: application/json I0620 20:00:50.771571 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:00:50.771582 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:00:50.771593 18453 round_trippers.go:452] Content-Length: 3211 I0620 20:00:50.771603 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:00:50 GMT I0620 20:00:50.771614 18453 round_trippers.go:452] Audit-Id: 77326488-eae6-4683-b37b-e2342546678a I0620 20:00:50.771625 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:00:50.771829 18453 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-84z7z","generateName":"must-gather-","namespace":"openshift-must-gather-k5xf7","selfLink":"/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z","uid":"0aaeba9c-fc56-40b5-9d9e-343fd674a579","resourceVersion":"23298","creationTimestamp":"2020-06-20T18:00:50Z","labels":{"app":"must-gather"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:50Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-42m4w","secret":{"secretName":"default-token-42m4w","defaultMode":420}}],"initContainers":[{"name":"gather","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/usr/bin/gather"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"cluster-84qpt-worker-0-shz5p","securityContext":{},"imagePullSecrets":[{"name":"default-dockercfg-t9vwg"}],"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","conditions":[{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z"}],"qosClass":"BestEffort"}} I0620 20:01:50.772983 18453 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z' I0620 20:01:50.778979 18453 round_trippers.go:443] GET https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z 200 OK in 5 milliseconds I0620 20:01:50.779024 18453 round_trippers.go:449] Response Headers: I0620 20:01:50.779044 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:01:50.779062 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:01:50.779076 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:01:50 GMT I0620 20:01:50.779088 18453 round_trippers.go:452] Audit-Id: 247d62d8-d4fd-43d4-917e-bdb5540354c4 I0620 20:01:50.779124 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:01:50.779135 18453 round_trippers.go:452] Content-Type: application/json I0620 20:01:50.779289 18453 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-84z7z","generateName":"must-gather-","namespace":"openshift-must-gather-k5xf7","selfLink":"/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z","uid":"0aaeba9c-fc56-40b5-9d9e-343fd674a579","resourceVersion":"23355","creationTimestamp":"2020-06-20T18:00:50Z","labels":{"app":"must-gather"},"annotations":{"k8s.v1.cni.cncf.io/network-status":"[{\n \"name\": \"openshift-sdn\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.4\"\n ],\n \"default\": true,\n \"dns\": {}\n}]","k8s.v1.cni.cncf.io/networks-status":"[{\n \"name\": \"openshift-sdn\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.4\"\n ],\n \"default\": true,\n \"dns\": {}\n}]"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:50Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}},{"manager":"multus","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:59Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:initContainerStatuses":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.129.2.4\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-42m4w","secret":{"secretName":"default-token-42m4w","defaultMode":420}}],"initContainers":[{"name":"gather","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/usr/bin/gather"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"cluster-84qpt-worker-0-shz5p","securityContext":{},"imagePullSecrets":[{"name":"default-dockercfg-t9vwg"}],"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotInitialized","message":"containers with incomplete status: [gather]"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z"}],"hostIP":"10.32.111.4","podIP":"10.129.2.4","podIPs":[{"ip":"10.129.2.4"}],"startTime":"2020-06-20T18:00:50Z","initContainerStatuses":[{"name":"gather","state":{"running":{"startedAt":"2020-06-20T18:00:58Z"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","imageID":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","containerID":"cri-o://ea00f84b68c3323872a102e4dadba6d13e2610c084a7ebd7c062c87fc3813f23"}],"containerStatuses":[{"name":"copy","state":{"waiting":{"reason":"PodInitializing"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","imageID":"","started":false}],"qosClass":"BestEffort"}} I0620 20:01:50.781381 18453 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z/log?container=gather&follow=true' I0620 20:01:50.792475 18453 round_trippers.go:443] GET https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z/log?container=gather&follow=true 500 Internal Server Error in 11 milliseconds I0620 20:01:50.792504 18453 round_trippers.go:449] Response Headers: I0620 20:01:50.792534 18453 round_trippers.go:452] Content-Length: 236 I0620 20:01:50.792548 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:01:50 GMT I0620 20:01:50.792562 18453 round_trippers.go:452] Audit-Id: a925fcec-7404-406b-a2ac-633e8ac3546e I0620 20:01:50.792572 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:01:50.792582 18453 round_trippers.go:452] Content-Type: application/json I0620 20:01:50.792607 18453 request.go:1068] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Get https://10.32.111.4:10250/containerLogs/openshift-must-gather-k5xf7/must-gather-84z7z/gather?follow=true: remote error: tls: internal error","code":500} [must-gather-84z7z] OUT gather logs unavailable: Get https://10.32.111.4:10250/containerLogs/openshift-must-gather-k5xf7/must-gather-84z7z/gather?follow=true: remote error: tls: internal error [must-gather-84z7z] OUT waiting for gather to complete I0620 20:01:50.793564 18453 round_trippers.go:423] curl -k -v -XGET -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" -H "Accept: application/json, */*" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z' I0620 20:01:50.798995 18453 round_trippers.go:443] GET https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z 200 OK in 5 milliseconds I0620 20:01:50.799025 18453 round_trippers.go:449] Response Headers: I0620 20:01:50.799037 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:01:50 GMT I0620 20:01:50.799049 18453 round_trippers.go:452] Audit-Id: dbe8f952-b78c-4f55-ac01-304c2b4ebe20 I0620 20:01:50.799059 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:01:50.799070 18453 round_trippers.go:452] Content-Type: application/json I0620 20:01:50.799080 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:01:50.799122 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:01:50.799242 18453 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-84z7z","generateName":"must-gather-","namespace":"openshift-must-gather-k5xf7","selfLink":"/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z","uid":"0aaeba9c-fc56-40b5-9d9e-343fd674a579","resourceVersion":"23355","creationTimestamp":"2020-06-20T18:00:50Z","labels":{"app":"must-gather"},"annotations":{"k8s.v1.cni.cncf.io/network-status":"[{\n \"name\": \"openshift-sdn\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.4\"\n ],\n \"default\": true,\n \"dns\": {}\n}]","k8s.v1.cni.cncf.io/networks-status":"[{\n \"name\": \"openshift-sdn\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.4\"\n ],\n \"default\": true,\n \"dns\": {}\n}]"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:50Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}},{"manager":"multus","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:59Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:initContainerStatuses":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.129.2.4\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-42m4w","secret":{"secretName":"default-token-42m4w","defaultMode":420}}],"initContainers":[{"name":"gather","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/usr/bin/gather"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"cluster-84qpt-worker-0-shz5p","securityContext":{},"imagePullSecrets":[{"name":"default-dockercfg-t9vwg"}],"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotInitialized","message":"containers with incomplete status: [gather]"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z"}],"hostIP":"10.32.111.4","podIP":"10.129.2.4","podIPs":[{"ip":"10.129.2.4"}],"startTime":"2020-06-20T18:00:50Z","initContainerStatuses":[{"name":"gather","state":{"running":{"startedAt":"2020-06-20T18:00:58Z"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","imageID":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","containerID":"cri-o://ea00f84b68c3323872a102e4dadba6d13e2610c084a7ebd7c062c87fc3813f23"}],"containerStatuses":[{"name":"copy","state":{"waiting":{"reason":"PodInitializing"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","imageID":"","started":false}],"qosClass":"BestEffort"}} I0620 20:02:50.801135 18453 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z' I0620 20:02:50.812032 18453 round_trippers.go:443] GET https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z 200 OK in 10 milliseconds I0620 20:02:50.812107 18453 round_trippers.go:449] Response Headers: I0620 20:02:50.812123 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:02:50 GMT I0620 20:02:50.812134 18453 round_trippers.go:452] Audit-Id: 7874ffa0-5dcf-4b79-b386-98f4ad6accfb I0620 20:02:50.812145 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:02:50.812156 18453 round_trippers.go:452] Content-Type: application/json I0620 20:02:50.812166 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:02:50.812177 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:02:50.812616 18453 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"must-gather-84z7z","generateName":"must-gather-","namespace":"openshift-must-gather-k5xf7","selfLink":"/api/v1/namespaces/openshift-must-gather-k5xf7/pods/must-gather-84z7z","uid":"0aaeba9c-fc56-40b5-9d9e-343fd674a579","resourceVersion":"23355","creationTimestamp":"2020-06-20T18:00:50Z","labels":{"app":"must-gather"},"annotations":{"k8s.v1.cni.cncf.io/network-status":"[{\n \"name\": \"openshift-sdn\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.4\"\n ],\n \"default\": true,\n \"dns\": {}\n}]","k8s.v1.cni.cncf.io/networks-status":"[{\n \"name\": \"openshift-sdn\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.4\"\n ],\n \"default\": true,\n \"dns\": {}\n}]"},"managedFields":[{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:50Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"copy\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:initContainers":{".":{},"k:{\"name\":\"gather\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/must-gather\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"must-gather-output\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}}},{"manager":"multus","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:59Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:initContainerStatuses":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.129.2.4\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},"spec":{"volumes":[{"name":"must-gather-output","emptyDir":{}},{"name":"default-token-42m4w","secret":{"secretName":"default-token-42m4w","defaultMode":420}}],"initContainers":[{"name":"gather","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/usr/bin/gather"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"copy","image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"resources":{},"volumeMounts":[{"name":"must-gather-output","mountPath":"/must-gather"},{"name":"default-token-42m4w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":0,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"cluster-84qpt-worker-0-shz5p","securityContext":{},"imagePullSecrets":[{"name":"default-dockercfg-t9vwg"}],"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotInitialized","message":"containers with incomplete status: [gather]"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z","reason":"ContainersNotReady","message":"containers with unready status: [copy]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-20T18:00:50Z"}],"hostIP":"10.32.111.4","podIP":"10.129.2.4","podIPs":[{"ip":"10.129.2.4"}],"startTime":"2020-06-20T18:00:50Z","initContainerStatuses":[{"name":"gather","state":{"running":{"startedAt":"2020-06-20T18:00:58Z"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","imageID":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","containerID":"cri-o://ea00f84b68c3323872a102e4dadba6d13e2610c084a7ebd7c062c87fc3813f23"}],"containerStatuses":[{"name":"copy","state":{"waiting":{"reason":"PodInitializing"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:271b725dc5eaa4aba565bd9f7065472f94ca8311f528385ccef32c15248debc8","imageID":"","started":false}],"qosClass":"BestEffort"}} [must-gather-84z7z] OUT gather never finished: timed out waiting for the condition I0620 20:02:50.813478 18453 request.go:1068] Request Body: {"kind":"DeleteOptions","apiVersion":"rbac.authorization.k8s.io/v1"} I0620 20:02:50.813572 18453 round_trippers.go:423] curl -k -v -XDELETE -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/must-gather-s4gfx' I0620 20:02:50.835367 18453 round_trippers.go:443] DELETE https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/must-gather-s4gfx 200 OK in 21 milliseconds I0620 20:02:50.835406 18453 round_trippers.go:449] Response Headers: I0620 20:02:50.835419 18453 round_trippers.go:452] Audit-Id: 5168013e-e6fd-4314-bfaa-da6187f82ee4 I0620 20:02:50.835430 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:02:50.835441 18453 round_trippers.go:452] Content-Type: application/json I0620 20:02:50.835452 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:02:50.835462 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:02:50.835473 18453 round_trippers.go:452] Content-Length: 218 I0620 20:02:50.835483 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:02:50 GMT I0620 20:02:50.835547 18453 request.go:1068] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"must-gather-s4gfx","group":"rbac.authorization.k8s.io","kind":"clusterrolebindings","uid":"4631e0ce-125a-44e6-ad14-38719c6e6200"}} [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-s4gfx deleted I0620 20:02:50.835704 18453 request.go:1068] Request Body: {"kind":"DeleteOptions","apiVersion":"v1"} I0620 20:02:50.835778 18453 round_trippers.go:423] curl -k -v -XDELETE -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/711c56a" 'https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7' I0620 20:02:50.848955 18453 round_trippers.go:443] DELETE https://api.cluster.ocp4-csa.coe.muc.redhat.com:6443/api/v1/namespaces/openshift-must-gather-k5xf7 200 OK in 13 milliseconds I0620 20:02:50.848983 18453 round_trippers.go:449] Response Headers: I0620 20:02:50.849002 18453 round_trippers.go:452] Content-Type: application/json I0620 20:02:50.849021 18453 round_trippers.go:452] X-Kubernetes-Pf-Flowschema-Uid: bc2676b1-fb6b-47e7-9470-0dcc110e041c I0620 20:02:50.849039 18453 round_trippers.go:452] X-Kubernetes-Pf-Prioritylevel-Uid: c9019cf6-2d16-413c-a717-1ba39342ae4b I0620 20:02:50.849053 18453 round_trippers.go:452] Content-Length: 1271 I0620 20:02:50.849064 18453 round_trippers.go:452] Date: Sat, 20 Jun 2020 18:02:50 GMT I0620 20:02:50.849075 18453 round_trippers.go:452] Audit-Id: 26861d25-aefd-43e0-8f20-afc40da624ca I0620 20:02:50.849085 18453 round_trippers.go:452] Cache-Control: no-cache, private I0620 20:02:50.849138 18453 request.go:1068] Response Body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"openshift-must-gather-k5xf7","generateName":"openshift-must-gather-","selfLink":"/api/v1/namespaces/openshift-must-gather-k5xf7","uid":"e5a37071-01d5-4a10-8168-858d72b70035","resourceVersion":"23984","creationTimestamp":"2020-06-20T18:00:49Z","deletionTimestamp":"2020-06-20T18:02:50Z","labels":{"openshift.io/run-level":"0"},"annotations":{"oc.openshift.io/command":"oc adm must-gather","openshift.io/sa.scc.mcs":"s0:c24,c9","openshift.io/sa.scc.supplemental-groups":"1000570000/10000","openshift.io/sa.scc.uid-range":"1000570000/10000"},"managedFields":[{"manager":"cluster-policy-controller","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:49Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:openshift.io/sa.scc.mcs":{},"f:openshift.io/sa.scc.supplemental-groups":{},"f:openshift.io/sa.scc.uid-range":{}}}}},{"manager":"oc","operation":"Update","apiVersion":"v1","time":"2020-06-20T18:00:49Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:oc.openshift.io/command":{}},"f:generateName":{},"f:labels":{".":{},"f:openshift.io/run-level":{}}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}} [must-gather ] OUT namespace/openshift-must-gather-k5xf7 deleted F0620 20:02:50.849509 18453 helpers.go:115] error: gather never finished for pod must-gather-84z7z: timed out waiting for the condition # and had the same with -rc1
The problem here is that /usr/bin/gather_network_ovn_trace script is not available in 4.5 must-gather image. It was added only in 4.6 https://github.com/openshift/must-gather/pull/156. Phil already opened a 4.5 backport here: https://github.com/openshift/must-gather/pull/160 so when that merges only then it'll be possible for oc adm must-gather to run that script. I'm closing since this is not a bug in oc adm must-gather but in must-gather image, which is already tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1849244