Description of problem: Originally reported bug : https://github.com/kubernetes/kubernetes/issues/61178 By creating a deployment where an init container creates a file in an emptydir and then the container tries to mount this file as a subpath will reproduce it. --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: subpath namespace: kube-system spec: replicas: 1 template: metadata: labels: app: subpath spec: initContainers: - name: init image: busybox command: - touch - /mount/test volumeMounts: - name: mount mountPath: /mount containers: - name: subtest image: busybox command: - ls - -l - /mount/test volumeMounts: - name: mount mountPath: /mount/test subPath: test volumes: - name: mount emptyDir: {} Version-Release number of selected component (if applicable): 3.9 Description of problem: Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info:
https://github.com/openshift/ose/pull/1135
Not in 3.9.9-1 yet, waiting for next build.
Just for the reference, I checked older releases and opened bugs + PRs for OSE 3.7 - 3.5. 3.4 and 3.3 does not suffer from this bug, because they does not support subpath with files, only with directories. Nobody has complained so far and I'd like to leave it as it is. 3.2 does not have subpath.
The PR is merged to 3.9.11-1, move this bug to QA status to check it in 3.9.11-1.
Tested on below version: openshift v3.9.11 kubernetes v1.9.1+a0ce1bc657 Tried to create a pod with comment #0, checked the upstream issue description of https://github.com/kubernetes/kubernetes/issues/61178#issuecomment-373135118, wait for 2 hours did not see the error of subpath of "Error: failed to prepare subPath for volumeMount "flannel-net-conf" of container "kube-flannel" # oc get pods NAME READY STATUS RESTARTS AGE subpath-6cc544864b-99l22 0/1 CrashLoopBackOff 7 14m # oc describe pods subpath-6cc544864b-99l22 Name: subpath-6cc544864b-99l22 Namespace: kube-system Node: wehe-node-registry-router-2/10.1.2.8 Start Time: Mon, 19 Mar 2018 03:14:08 +0000 Labels: app=subpath pod-template-hash=2771004206 Annotations: openshift.io/scc=restricted Status: Running IP: 10.129.0.14 Controlled By: ReplicaSet/subpath-6cc544864b Init Containers: init: Container ID: docker://92310c5052e045253e8d546aaaab4f3a83ce64b1bb64245690d68f382dc03f4f Image: busybox Image ID: docker-pullable://docker.io/busybox@sha256:2107a35b58593c58ec5f4e8f2c4a70d195321078aebfadfbfb223a2ff4a4ed21 Port: <none> Command: touch /mount/test State: Terminated Reason: Completed Exit Code: 0 Started: Mon, 19 Mar 2018 03:14:14 +0000 Finished: Mon, 19 Mar 2018 03:14:14 +0000 Ready: True Restart Count: 0 Environment: <none> Mounts: /mount from mount (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-c84sd (ro) Containers: subtest: Container ID: docker://48301a5294c6113edda36f0f5ad24173f5bdd426997d354782d38cc3ed81da7b Image: busybox Image ID: docker-pullable://docker.io/busybox@sha256:2107a35b58593c58ec5f4e8f2c4a70d195321078aebfadfbfb223a2ff4a4ed21 Port: <none> Command: ls -l /mount/test State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Completed Exit Code: 0 Started: Mon, 19 Mar 2018 03:25:06 +0000 Finished: Mon, 19 Mar 2018 03:25:06 +0000 Ready: False Restart Count: 7 Environment: <none> Mounts: /mount/test from mount (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-c84sd (ro) Conditions: Type Status Initialized True Ready False PodScheduled True Volumes: mount: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: default-token-c84sd: Type: Secret (a volume populated by a Secret) SecretName: default-token-c84sd Optional: false QoS Class: BestEffort Node-Selectors: node-role.kubernetes.io/compute=true Tolerations: <none> Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 14m default-scheduler Successfully assigned subpath-6cc544864b-99l22 to wehe-node-registry-router-2 Normal SuccessfulMountVolume 14m kubelet, wehe-node-registry-router-2 MountVolume.SetUp succeeded for volume "mount" Normal SuccessfulMountVolume 14m kubelet, wehe-node-registry-router-2 MountVolume.SetUp succeeded for volume "default-token-c84sd" Normal Pulling 14m kubelet, wehe-node-registry-router-2 pulling image "busybox" Normal Pulled 14m kubelet, wehe-node-registry-router-2 Successfully pulled image "busybox" Normal Created 14m kubelet, wehe-node-registry-router-2 Created container Normal Started 14m kubelet, wehe-node-registry-router-2 Started container Normal Created 13m (x3 over 14m) kubelet, wehe-node-registry-router-2 Created container Normal Started 13m (x3 over 14m) kubelet, wehe-node-registry-router-2 Started container Normal Pulling 13m (x4 over 14m) kubelet, wehe-node-registry-router-2 pulling image "busybox" Normal Pulled 13m (x4 over 14m) kubelet, wehe-node-registry-router-2 Successfully pulled image "busybox" Warning BackOff 4m (x44 over 13m) kubelet, wehe-node-registry-router-2 Back-off restarting failed container And per Jan's comment "the container is in CrashLoopBackOff because it finished successfully and its restart policy keeps restarting it and gives up eventually"