Description of problem: Pod is killed unexpectedly as DeadlineExceeded when activeDeadlineSeconds: 341547784951046144 This is wired that we have value overflow check, this yaml file passed check but not work properly. $ oc create -f pod.yaml Error from server (BadRequest): error when creating "pod.yaml": Pod in version "v1" cannot be handled as a Pod: [pos 139]: json: overflow integer Version-Release number of selected component (if applicable): oc v3.6.85 kubernetes v1.6.1+5115d708d7 How reproducible: Always Steps to Reproduce: 1. create pod with activeDeadlineSeconds: 341547784951046144 $ oc create -f pod.yaml pod created. cat pod.yaml apiVersion: v1 kind: Pod metadata: name: hello-pod1 spec: activeDeadlineSeconds: 341547784951046144 containers: - image: "docker.io/ocpqe/hello-pod" name: hello-pod 2. check pod $ oc get pod $ oc describe pod Actual results: NAME READY STATUS RESTARTS AGE hello-pod1 0/1 DeadlineExceeded 0 4s <---snip---> Status: Failed Reason: DeadlineExceeded Message: Pod was active on the node longer than the specified deadline <---snip---> Expected results: pod should be running for specified time, should not be killed too soon. Additional info: If invalid value set for activeDeadlineSeconds, we can get error $ oc create -f pod.yaml Error from server (BadRequest): error when creating "pod.yaml": Pod in version "v1" cannot be handled as a Pod: [pos 139]: json: overflow integer OCP 3.5 also has the same issue.
I opened the following upstream pull requests: https://github.com/kubernetes/kubernetes/pull/46642 has the kubelet detect overflows that result with the provided value and make it think the active deadline was actually negative. https://github.com/kubernetes/kubernetes/pull/46640 which restricts validation of activeDeadlineSeconds to max uint32 it's possible this will be considered backwards incompatible since it tightens validation, but i think its safe to do given the value space.
ORIGIN PR: https://github.com/openshift/origin/pull/14424 restricts validation of activeDeadlineSeconds to max uint32
correction, to fix cross build errors, it will be max int32.
Verified on openshift v3.6.94 Failed. # oc get pod NAME READY STATUS RESTARTS AGE hello-pod1 0/1 DeadlineExceeded 0 54s
tested on commit: 6473593c449ab84cfdf57361e41eae5b78e766ee $ oc create -f pod.yaml The Pod "hello-pod1" is invalid: spec.activeDeadlineSeconds: Invalid value: 341547784951046144: must be between 1 and 2147483647, inclusive
@wmeng can you include the exact version you are running and the pod definition you created if you still see the problem?
@epairs the exact version mentioned in comment 5, Verified on openshift v3.6.94 Failed. and pod definition in repo steps of description. and I tried again in latest OCP, openshift v3.6.96, still has this problem. Can you have a try?
Could you please indicate which OCP version fixed this bug?
openshift v3.6.114 $ cat ~/scripts/samples/kube/active-deadline-too-big.yaml apiVersion: v1 kind: Pod metadata: name: burstable spec: activeDeadlineSeconds: 341547784951046144 containers: - image: busybox name: busybox command: - sleep - "3600" resources: requests: memory: 1500Mi cpu: 300m restartPolicy: Always terminationGracePeriodSeconds: 0 $ oc create -f ~/scripts/samples/kube/active-deadline-too-big.yaml The Pod "burstable" is invalid: spec.activeDeadlineSeconds: Invalid value: 341547784951046144: must be between 1 and 2147483647, inclusive
Verified on openshift v3.6.114 kubernetes v1.6.1+5115d708d7 Fixed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1716