Attachment 1440648 Details for Bug 1581305 – Log from the node filtered by pod/container/uid

Log from the node filtered by pod/container/uid

nodelog.txt (text/plain), 43.00 KB, created by Joel Smith on 2018-05-23 14:55:59 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Joel Smith

Created: 2018-05-23 14:55:59 UTC

Size: 43.00 KB

patch

obsolete

>May 21 23:26:18 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:18.347973   95549 kubelet.go:1856] SyncLoop (ADD, "api"): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)"
>May 21 23:26:18 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:18.471255   95549 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:18 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:18.571569   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:18 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:18.594549   95549 operation_generator.go:555] MountVolume.SetUp succeeded for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:18 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:18.697437   95549 kuberuntime_manager.go:385] No sandbox for pod "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)" can be found. Need to start a new one
>May 21 23:26:19 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:19.475526   95549 kubelet.go:1901] SyncLoop (PLEG): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)", event: &pleg.PodLifecycleEvent{ID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", Type:"ContainerStarted", Data:"ae32adefb7e27a76359c4bab387f4280134adde535c063b3a650e891e59d34d9"}
>May 21 23:26:42 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:42.330650   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc9248418fe0", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:"spec.containers{git-server}"}, Reason:"Pulled", Message:"Successfully pulled image \"aosqe/ssh-git-server-openshift\"", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34c938f1be0, ext:46641835886492, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34c938f1be0, ext:46641835886492, loc:(*time.Location)(0x8fb0260)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc9248418fe0" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:42 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:42.341773   95549 kubelet.go:1872] SyncLoop (DELETE, "api"): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)"
>May 21 23:26:42 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:42.465291   95549 reconciler.go:181] operationExecutor.UnmountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "5cf5704e-5d4e-11e8-850d-0a46c474dfe0" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:42 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:42.474862   95549 operation_generator.go:684] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq" (OuterVolumeSpecName: "default-token-8b2xq") pod "5cf5704e-5d4e-11e8-850d-0a46c474dfe0" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0"). InnerVolumeSpecName "default-token-8b2xq". PluginName "kubernetes.io/secret", VolumeGidValue ""
>May 21 23:26:42 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:42.566135   95549 reconciler.go:285] Volume detached for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") on node "ip-172-31-63-68.ec2.internal" DevicePath ""
>May 21 23:26:42 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:42.700030   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc925e4a592e", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:"spec.containers{git-server}"}, Reason:"Created", Message:"Created container", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ca997e52e, ext:46642205560994, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ca997e52e, ext:46642205560994, loc:(*time.Location)(0x8fb0260)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc925e4a592e" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:42 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:42.984349   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc926f38204f", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:"spec.containers{git-server}"}, Reason:"Started", Message:"Started container", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34cba85ac4f, ext:46642489579447, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34cba85ac4f, ext:46642489579447, loc:(*time.Location)(0x8fb0260)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc926f38204f" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:43 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:43.407111   95549 kubelet.go:1901] SyncLoop (PLEG): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)", event: &pleg.PodLifecycleEvent{ID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", Type:"ContainerStarted", Data:"9adabb17fb445540f631c4d9e499514089716d6016966046111e4d97ba60df67"}
>May 21 23:26:43 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:43.569334   95549 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:43 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:43.669687   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:43 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:43.683980   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:26:44.183930159 +0000 UTC m=+46643.691670738 (durationBeforeRetry 500ms). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:26:43 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:43.685673   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:44 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:44.271360   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:44 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:44.275488   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:26:45.275445023 +0000 UTC m=+46644.783185430 (durationBeforeRetry 1s). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:26:44 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:44.280516   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34d106ab83f, ext:46643783169892, loc:(*time.Location)(0x8fb0260)}}, Count:2, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:45 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:45.374403   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:45 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:45.378528   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:26:47.37846543 +0000 UTC m=+46646.886206052 (durationBeforeRetry 2s). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:26:45 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:45.383254   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34d568e9be3, ext:46644886185408, loc:(*time.Location)(0x8fb0260)}}, Count:3, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:47.380114   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:47.384428   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:26:51.38438389 +0000 UTC m=+46650.892124456 (durationBeforeRetry 4s). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:26:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:47.389252   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34dd6e916e1, ext:46646892115025, loc:(*time.Location)(0x8fb0260)}}, Count:4, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:51 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:51.390775   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:51 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:51.394757   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:26:59.394730429 +0000 UTC m=+46658.902470851 (durationBeforeRetry 8s). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:26:51 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:51.399359   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ed786ff5a, ext:46650902463582, loc:(*time.Location)(0x8fb0260)}}, Count:5, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:26:59 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:26:59.420423   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:26:59 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:59.434254   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:27:15.434215296 +0000 UTC m=+46674.941955809 (durationBeforeRetry 16s). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:26:59 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:26:59.440757   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f350d9e17400, ext:46658941946298, loc:(*time.Location)(0x8fb0260)}}, Count:6, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:27:15 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:27:15.487306   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:27:15 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:27:15.492122   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:27:47.492089285 +0000 UTC m=+46706.999829781 (durationBeforeRetry 32s). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:27:15 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:27:15.497321   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f354dd54705c, ext:46674999813605, loc:(*time.Location)(0x8fb0260)}}, Count:7, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:27:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:27:47.586519   95549 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "git-server-2-j2pxd" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 21 23:27:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:27:47.591070   95549 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\" (\"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\")" failed. No retries permitted until 2018-05-21 23:28:51.591024576 +0000 UTC m=+46771.098765172 (durationBeforeRetry 1m4s). Error: "MountVolume.SetUp failed for volume \"default-token-8b2xq\" (UniqueName: \"kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq\") pod \"git-server-2-j2pxd\" (UID: \"5cf5704e-5d4e-11e8-850d-0a46c474dfe0\") : secrets \"default-token-8b2xq\" not found"
>May 21 23:27:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:27:47.596424   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530cc929910f0b3", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"default-token-8b2xq\" : secrets \"default-token-8b2xq\" not found", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f34ce8c3b2b3, ext:46643191654551, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f35ce33a060b, ext:46707098745721, loc:(*time.Location)(0x8fb0260)}}, Count:8, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530cc929910f0b3" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:28:46 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:28:46.407929   95549 kubelet.go:1635] Unable to mount volumes for pod "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)": timeout expired waiting for volumes to attach or mount for pod "38ua8"/"git-server-2-j2pxd". list of unmounted volumes=[default-token-8b2xq]. list of unattached volumes=[default-token-8b2xq]; skipping pod
>May 21 23:28:46 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:28:46.408068   95549 pod_workers.go:186] Error syncing pod 5cf5704e-5d4e-11e8-850d-0a46c474dfe0 ("git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)"), skipping: timeout expired waiting for volumes to attach or mount for pod "38ua8"/"git-server-2-j2pxd". list of unmounted volumes=[default-token-8b2xq]. list of unattached volumes=[default-token-8b2xq]
>May 21 23:28:46 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:28:46.410109   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530ccaf2bfc895f", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:""}, Reason:"FailedMount", Message:"Unable to mount volumes for pod \"git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)\": timeout expired waiting for volumes to attach or mount for pod \"38ua8\"/\"git-server-2-j2pxd\". list of unmounted volumes=[default-token-8b2xq]. list of unattached volumes=[default-token-8b2xq]", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f36b98503d5f, ext:46765915652195, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f36b98503d5f, ext:46765915652195, loc:(*time.Location)(0x8fb0260)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530ccaf2bfc895f" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:28:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:28:47.230507   95549 kuberuntime_container.go:547] Killing container "docker://9adabb17fb445540f631c4d9e499514089716d6016966046111e4d97ba60df67" with 10 second grace period
>May 21 23:28:47 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:28:47.315827   95549 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"git-server-2-j2pxd.1530ccaf61fb95a9", GenerateName:"", Namespace:"38ua8", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"38ua8", Name:"git-server-2-j2pxd", UID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", APIVersion:"v1", ResourceVersion:"221155766", FieldPath:"spec.containers{git-server}"}, Reason:"Killing", Message:"Killing container with id docker://git-server:Need to kill Pod", Source:v1.EventSource{Component:"kubelet", Host:"ip-172-31-63-68.ec2.internal"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f36bd2b47fa9, ext:46766821559447, loc:(*time.Location)(0x8fb0260)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbeb8f36bd2b47fa9, ext:46766821559447, loc:(*time.Location)(0x8fb0260)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "git-server-2-j2pxd.1530ccaf61fb95a9" is forbidden: unable to create new content in namespace 38ua8 because it is being terminated' (will not retry!)
>May 21 23:28:48 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:28:48.288891   95549 kubelet.go:1901] SyncLoop (PLEG): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)", event: &pleg.PodLifecycleEvent{ID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", Type:"ContainerDied", Data:"9adabb17fb445540f631c4d9e499514089716d6016966046111e4d97ba60df67"}
>May 21 23:28:48 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:28:48.289027   95549 kubelet.go:1901] SyncLoop (PLEG): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)", event: &pleg.PodLifecycleEvent{ID:"5cf5704e-5d4e-11e8-850d-0a46c474dfe0", Type:"ContainerDied", Data:"ae32adefb7e27a76359c4bab387f4280134adde535c063b3a650e891e59d34d9"}
>May 21 23:28:48 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: E0521 23:28:48.327692   95549 remote_runtime.go:278] ContainerStatus "9adabb17fb445540f631c4d9e499514089716d6016966046111e4d97ba60df67" from runtime service failed: rpc error: code = Unknown desc = Error: No such container: 9adabb17fb445540f631c4d9e499514089716d6016966046111e4d97ba60df67
>May 21 23:28:48 ip-172-31-63-68.ec2.internal atomic-openshift-node[95549]: I0521 23:28:48.414790   95549 reconciler.go:285] Volume detached for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") on node "ip-172-31-63-68.ec2.internal" DevicePath ""
>May 22 19:08:49 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:08:49.249784    3083 kubelet.go:1856] SyncLoop (ADD, "api"): "run-once-test1-99_int2-yasun-int1006(f4cd37bd-5ced-11e8-adc7-0a46c474dfe0), run-once-test2-83_int2-yasun-int1006(72b02c68-5c66-11e8-ab04-0a46c474dfe0), run-once-test2-69_int2-yasun-int1006(dae6cf0b-5bef-11e8-a953-0a46c474dfe0), run-once-test1-76_int2-yasun-int1006(1fbf613e-5c2b-11e8-b94b-0a0ddbb52ad4), run-once-test2-73_int2-yasun-int1006(bceb2ea0-5c11-11e8-a953-0a46c474dfe0), run-once-test2-72_int2-yasun-int1006(43906eae-5c09-11e8-a953-0a46c474dfe0), git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0), logging-fluentd-jtmk9_logging(9e2197cb-5944-11e8-bcf7-0a46c474dfe0), run-once-test2-60_int2-yasun-int1006(a0b42c5f-5ba3-11e8-8088-0a0ddbb52ad4), run-once-test2-100_int2-yasun-int1006(74242c82-5cf6-11e8-a0f1-0a0ddbb52ad4), run-once-test2-82_int2-yasun-int1006(fa5832e5-5c5d-11e8-a4d9-0a0ddbb52ad4), run-once-test2-57_int2-yasun-int1006(38e80ab4-5b8a-11e8-8088-0a0ddbb52ad4), run-once-test1-78_int2-yasun-int1006(0fd9c5e6-5c3c-11e8-a953-0a46c474dfe0), sdn-hz4d8_openshift-sdn(e446db47-5930-11e8-8471-0a46c474dfe0), run-once-test2-70_int2-yasun-int1006(539036b5-5bf8-11e8-b94b-0a0ddbb52ad4), ovs-42sjb_openshift-sdn(9435d966-5d4d-11e8-850d-0a46c474dfe0), run-once-test2-95_int2-yasun-int1006(19ee5a7a-5ccc-11e8-a4d9-0a0ddbb52ad4), build-05191630z-uk-2-wr8xp_ops-health-monitoring(31d4fc84-5b82-11e8-a07f-0a46c474dfe0), sync-bsk2s_openshift-node(d3e1945d-5d57-11e8-ae00-0ac586c2eb16), prometheus-node-exporter-52stt_kube-system(5d991ea0-4282-11e8-ae20-0a0ddbb52ad4), run-once-test2-85_int2-yasun-int1006(619b964f-5c77-11e8-ab04-0a46c474dfe0), run-once-test2-59_int2-yasun-int1006(28e3b43a-5b9b-11e8-aa8e-0a46c474dfe0), run-once-test2-68_int2-yasun-int1006(61e2d3dc-5be7-11e8-b94b-0a0ddbb52ad4), build-05191630z-uk-1-build_ops-health-monitoring(0434b789-5b82-11e8-a07f-0a46c474dfe0), run-once-test2-77_int2-yasun-int1006(9f8cc7cd-5c33-11e8-a953-0a46c474dfe0), run-once-test2-76_int2-yasun-int1006(2789f03e-5c2b-11e8-a953-0a46c474dfe0), run-once-test1-77_int2-yasun-int1006(9849e3ad-5c33-11e8-b94b-0a0ddbb52ad4), run-once-test2-75_int2-yasun-int1006(aee25ed4-5c22-11e8-a953-0a46c474dfe0), prometheus-operator-c8d4497bf-8s9r8_openshift-monitoring(bd640c5a-5d5b-11e8-957c-0ac586c2eb16), run-once-test2-97_int2-yasun-int1006(0aa2f357-5cdd-11e8-a4d9-0a0ddbb52ad4), run-once-test2-87_int2-yasun-int1006(52fbf35f-5c88-11e8-a4d9-0a0ddbb52ad4), run-once-test2-74_int2-yasun-int1006(36403112-5c1a-11e8-a953-0a46c474dfe0), run-once-test2-80_int2-yasun-int1006(09595f14-5c4d-11e8-a4d9-0a0ddbb52ad4), run-once-test2-89_int2-yasun-int1006(448085b6-5c99-11e8-a4d9-0a0ddbb52ad4), run-once-test2-65_int2-yasun-int1006(f90aef31-5bcd-11e8-a953-0a46c474dfe0), run-once-test2-86_int2-yasun-int1006(da240376-5c7f-11e8-a4d9-0a0ddbb52ad4), run-once-test2-79_int2-yasun-int1006(8ffbf3bf-5c44-11e8-a4d9-0a0ddbb52ad4), run-once-test2-78_int2-yasun-int1006(17de0a63-5c3c-11e8-a953-0a46c474dfe0), run-once-test1-75_int2-yasun-int1006(a746b535-5c22-11e8-b94b-0a0ddbb52ad4), jenkins-14-gzv8z_ldimaggi-osiotest1(5768e720-5d60-11e8-a597-0a46c474dfe0), run-once-test2-71_int2-yasun-int1006(cb28d80d-5c00-11e8-a953-0a46c474dfe0), run-once-test2-88_int2-yasun-int1006(cb7f1c4c-5c90-11e8-ab04-0a46c474dfe0), run-once-test2-66_int2-yasun-int1006(71910408-5bd6-11e8-a953-0a46c474dfe0), run-once-test1-74_int2-yasun-int1006(2e459cda-5c1a-11e8-a953-0a46c474dfe0), run-once-test2-56_int2-yasun-int1006(c1516db2-5b81-11e8-b709-0a0ddbb52ad4), build-05220101z-xp-1-build_ops-health-monitoring(9a1468fb-5d5b-11e8-957c-0ac586c2eb16), run-once-test2-81_int2-yasun-int1006(81a1ffc2-5c55-11e8-ab04-0a46c474dfe0), run-once-test2-67_int2-yasun-int1006(ea17122c-5bde-11e8-b94b-0a0ddbb52ad4)"
>May 22 19:08:51 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:08:51.677274    3083 reconciler.go:396] Reconciler sync states: could not find pod information in desired state, update it in actual state: &{volumeName:kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq podName:5cf5704e-5d4e-11e8-850d-0a46c474dfe0 volumeSpec:0xc420381b20 outerVolumeSpecName:default-token-8b2xq pod:0xc421d77880 attachablePlugin:<nil> volumeGidValue: devicePath: reportedInUse:false mounter:0xc421b5a000 blockVolumeMapper:<nil>}
>May 22 19:08:53 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:08:53.357639    3083 reconciler.go:181] operationExecutor.UnmountVolume started for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") pod "5cf5704e-5d4e-11e8-850d-0a46c474dfe0" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0")
>May 22 19:08:53 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:08:53.369528    3083 operation_generator.go:684] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq" (OuterVolumeSpecName: "default-token-8b2xq") pod "5cf5704e-5d4e-11e8-850d-0a46c474dfe0" (UID: "5cf5704e-5d4e-11e8-850d-0a46c474dfe0"). InnerVolumeSpecName "default-token-8b2xq". PluginName "kubernetes.io/secret", VolumeGidValue ""
>May 22 19:08:53 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:08:53.458378    3083 reconciler.go:285] Volume detached for volume "default-token-8b2xq" (UniqueName: "kubernetes.io/secret/5cf5704e-5d4e-11e8-850d-0a46c474dfe0-default-token-8b2xq") on node "ip-172-31-63-68.ec2.internal" DevicePath ""
>May 22 19:09:02 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:09:02.968192    3083 kubelet.go:1872] SyncLoop (DELETE, "api"): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)"
>May 22 19:09:02 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:09:02.970676    3083 kubelet.go:1866] SyncLoop (REMOVE, "api"): "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)"
>May 22 19:09:02 ip-172-31-63-68.ec2.internal atomic-openshift-node[3083]: I0522 19:09:02.979973    3083 kubelet.go:2060] Failed to delete pod "git-server-2-j2pxd_38ua8(5cf5704e-5d4e-11e8-850d-0a46c474dfe0)", err: pod not found

Actions: View

Attachments on bug 1581305: 1440171 | 1440172 | 1440174 | 1440175 | 1440648 | 1440649