Bug 1318542

Summary: DB pod keeps in RunContainerError state and fail to create docker container with API error (500)
Product: OpenShift Online Reporter: Wenjing Zheng <wzheng>
Component: ContainersAssignee: Abhishek Gupta <abhgupta>
Status: CLOSED CURRENTRELEASE QA Contact: Wenjing Zheng <wzheng>
Severity: high Docs Contact:
Priority: high    
Version: 3.xCC: abhgupta, aos-bugs, jokerman, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-23 15:10:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Wenjing Zheng 2016-03-17 07:59:08 UTC
Description of problem:
Below error appears for db pod:
Events:
  FirstSeen    LastSeen    Count    From                    SubobjectPath            Type        Reason        Message
  ---------    --------    -----    ----                    -------------            --------    ------        -------
  <invalid>    <invalid>    1    {default-scheduler }                            Normal        Scheduled    Successfully assigned database-1-hook-pre to ip-172-31-15-140.ec2.internal
  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}    spec.containers{lifecycle}    Warning        Failed        Failed to create docker container with error: API error (500): mkdir /var/lib/docker/volumes/3e5355385db4ae06b649f81d4cedd28c3ef6268445b762984ce6257a72d03f96: permission denied

  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}        Warning    FailedSync    Error syncing pod, skipping: failed to "StartContainer" for "lifecycle" with RunContainerError: "runContainer: API error (500): mkdir /var/lib/docker/volumes/3e5355385db4ae06b649f81d4cedd28c3ef6268445b762984ce6257a72d03f96: permission denied\n"

Version-Release number of selected component (if applicable):
kubernetes v1.2.0-alpha.7-703-gbc4550d
Docker 1.8.2-el7, build a01dc02/1.8.2
kernel 3.10.0-327.10.1.el7.x86_64
https://console.dev-preview-int.openshift.com/console/

How reproducible:
always

Steps to Reproduce:
1. Create a project
2. oc new-app -f https://raw.githubusercontent.com/openshift/origin/master/examples/sample-app/application-template-stibuild.json
3. Check pod status, especially db pod

Actual results:
[wzheng@openshiftqe test]$ oc get pods
NAME                        READY     STATUS              RESTARTS   AGE
database-1-deploy           1/1       Running             0          1m
database-1-hook-pre         0/1       RunContainerError   0          58s
frontend-1-ha4bb            1/1       Running             0          <invalid>
frontend-1-uo058            1/1       Running             0          <invalid>
ruby-sample-build-1-build   0/1       Completed           0          1m
[wzheng@openshiftqe test]$ oc describe pods database-1-hook-pre
Name:        database-1-hook-pre
Namespace:    wzheng2
Image(s):    openshift/mysql-55-centos7:latest
Node:        ip-172-31-15-140.ec2.internal/172.31.15.140
Start Time:    Thu, 17 Mar 2016 15:16:27 +0800
Labels:        openshift.io/deployer-pod-for.name=database-1
Status:        Pending
Reason:        
Message:    
IP:        10.1.7.61
Controllers:    <none>
Containers:
  lifecycle:
    Container ID:    
    Image:        openshift/mysql-55-centos7:latest
    Image ID:        
    Port:        
    Command:
      /bin/true
    QoS Tier:
      cpu:        BestEffort
      memory:        BestEffort
    State:        Waiting
      Reason:        RunContainerError
    Ready:        False
    Restart Count:    0
    Environment Variables:
      CUSTOM_VAR1:            custom_value1
      MYSQL_USER:            user6YJ
      MYSQL_PASSWORD:            AYn1DNJn
      MYSQL_DATABASE:            root
      OPENSHIFT_DEPLOYMENT_NAME:    database-1
      OPENSHIFT_DEPLOYMENT_NAMESPACE:    wzheng2
Conditions:
  Type        Status
  Ready     False 
Volumes:
  default-token-ka49f:
    Type:    Secret (a secret that should populate this volume)
    SecretName:    default-token-ka49f
Events:
  FirstSeen    LastSeen    Count    From                    SubobjectPath            Type        Reason        Message
  ---------    --------    -----    ----                    -------------            --------    ------        -------
  <invalid>    <invalid>    1    {default-scheduler }                            Normal        Scheduled    Successfully assigned database-1-hook-pre to ip-172-31-15-140.ec2.internal
  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}    spec.containers{lifecycle}    Warning        Failed        Failed to create docker container with error: API error (500): mkdir /var/lib/docker/volumes/3e5355385db4ae06b649f81d4cedd28c3ef6268445b762984ce6257a72d03f96: permission denied

  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}        Warning    FailedSync    Error syncing pod, skipping: failed to "StartContainer" for "lifecycle" with RunContainerError: "runContainer: API error (500): mkdir /var/lib/docker/volumes/3e5355385db4ae06b649f81d4cedd28c3ef6268445b762984ce6257a72d03f96: permission denied\n"

  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}    spec.containers{lifecycle}    Warning    Failed    Failed to create docker container with error: API error (500): mkdir /var/lib/docker/volumes/55860555b9de6f13eb3aa62a9f24c1fb49900e9dbe007eb2c4d819bb167a17e8: permission denied

  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}        Warning    FailedSync    Error syncing pod, skipping: failed to "StartContainer" for "lifecycle" with RunContainerError: "runContainer: API error (500): mkdir /var/lib/docker/volumes/55860555b9de6f13eb3aa62a9f24c1fb49900e9dbe007eb2c4d819bb167a17e8: permission denied\n"

  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}    spec.containers{lifecycle}    Warning    Failed    Failed to create docker container with error: API error (500): mkdir /var/lib/docker/volumes/32c3cfc0b917ed78ff8d1ba7a196c35f4d7a22477cb2325c7c2aead91b44572e: permission denied

  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}        Warning    FailedSync    Error syncing pod, skipping: failed to "StartContainer" for "lifecycle" with RunContainerError: "runContainer: API error (500): mkdir /var/lib/docker/volumes/32c3cfc0b917ed78ff8d1ba7a196c35f4d7a22477cb2325c7c2aead91b44572e: permission denied\n"

  <invalid>    <invalid>    4    {kubelet ip-172-31-15-140.ec2.internal}    spec.containers{lifecycle}    Normal    Pulling        pulling image "openshift/mysql-55-centos7:latest"
  <invalid>    <invalid>    4    {kubelet ip-172-31-15-140.ec2.internal}    spec.containers{lifecycle}    Normal    Pulled        Successfully pulled image "openshift/mysql-55-centos7:latest"
  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}                    Warning    FailedSync    Error syncing pod, skipping: failed to "StartContainer" for "lifecycle" with RunContainerError: "runContainer: API error (500): mkdir /var/lib/docker/volumes/5789239d348e46e0f226d34b3b925e916160ebf9a77e932fdf80932acb0746cf: permission denied\n"

  <invalid>    <invalid>    1    {kubelet ip-172-31-15-140.ec2.internal}    spec.containers{lifecycle}    Warning    Failed    Failed to create docker container with error: API error (500): mkdir /var/lib/docker/volumes/5789239d348e46e0f226d34b3b925e916160ebf9a77e932fdf80932acb0746cf: permission denied


Expected results:
The db pod should be running and succeed to create docker container.

Additional info:

Comment 1 Daniel Walsh 2016-03-18 18:26:12 UTC
Don't see where this is a docker issue?

Comment 2 Abhishek Gupta 2016-03-21 07:13:41 UTC
This is specific to Online Developer Preview, where we are specifically blocking docker from provisioning volumes within /var/lib/docker/volumes for pods that specify a VOLUME directive but the pod spec does not include a volume mount.

Assigning this bug to myself for now to avoid confusion.

Comment 3 Abhishek Gupta 2016-03-21 20:25:31 UTC
The deployment "pre" hook uses the openshift/mysql-55-centos7:latest image and I believe the image relies on some VOLUME for its data. Since the "pre" hook does not specify a PVC or an emptyDir volume to mount against it, docker falls back to provisioning a volume directory on its own. This is something that is explicitly prevented in Online and hence the hook pod fails.

You need to provide a PVC or an emptyDir for your database image for it to work.

Comment 4 Abhishek Gupta 2016-03-21 20:29:48 UTC
Note from Dan Mace: You need to explicitly specify which volumes you want to carry over to the hook pod via execNewPod.volumes. This should be a list of names of volumes from the template pod's volume list.

Comment 5 Wenjing Zheng 2016-03-24 03:41:07 UTC
Yes, after providing pvs or emptyDir, no such error, will verify this bug. Thanks for your clarification.