Description of problem:

Environment:

A Python 2.7, MongoDB application.   

If I do a git push which triggers a build via a Github webhook, via the console I can see the build complete, it then moves to the deploy state for a long period of time (~ 7 minutes) and then fails and the pod color turns orange.  

I can get around this by scaling down the pod before doing a git push, waiting for it to build, then deploy, and then I manually click the Scale Up button on the pod.  

The pod event messages are below, I believe they are somewhat misleading however as when I scale down the pod before git pushing, these event messages do not occur:

3:03:36 PM 
Warning 
Failed mount 
Unable to mount volumes for pod "python-app-77-1xmlr_my-app(d82bee7c-69b6-11e6-8f30-12d79454368d)": Could not attach EBS Disk "aws://us-east-1c/vol-abfa5979": Error attaching EBS volume: VolumeInUse: vol-abfa5979 is already attached to an instance status code: 400, request id:
8 times in the last 8 minutes

3:03:36 PM
Warning
Failed sync
Error syncing pod, skipping: Could not attach EBS Disk "aws://us-east-1c/vol-abfa5979": Error attaching EBS volume: VolumeInUse: vol-abfa5979 is already attached to an instance status code: 400, request id:
8 times in the last 8 minutes

2:54:31 PM
Normal
Scheduled
Successfully assigned python-app-77-1xmlr to ip-172-31-54-166.ec2.internal 

Version-Release number of selected component (if applicable):

Dev Preview

How reproducible:

I don't know.  

Steps to Reproduce:

1. In a Python 2.7, MongoDB application.

2. Login to console.

3. Do a git push.  

4. Watch what occurs in the console.  

Actual results:

The pod deployment is failing.  

Expected results:

The pod builds, deploys and scales up, whilst scaling down the previous pod.  

Additional info:

Troubleshooting Details

oc get pvc
NAME        STATUS    VOLUME         CAPACITY   ACCESSMODES   AGE
mongodb     Bound     pv-aws-1dj3b   4Gi        RWO           17d
pvc-nf0kl   Bound     pv-aws-e1agr   4Gi        RWO           16d


oc volume dc --all 
deploymentconfigs/mongodb
  pvc/mongodb (allocated 4GiB) as mongodb-data
    mounted at /var/lib/mongodb/data
deploymentconfigs/python-app
  pvc/pvc-nf0kl (allocated 4GiB) as mypythonvolume
    mounted at /opt/app-root/src/static/media_files


oc describe quota compute-resources -n my-app
Name:           compute-resources
Namespace:      my-app
Scopes:         NotTerminating
 * Matches all pods that do not have an active deadline.
Resource        Used    Hard
--------        ----    ----
limits.cpu      2       8
limits.memory   2G      4Gi


oc get pods
NAME                   READY     STATUS      RESTARTS   AGE
mongodb-11-2ln7o       1/1       Running     0          8d
python-app-24-build    0/1       Error       0          6d
python-app-36-build    0/1       Completed   0          22h
python-app-37-build    0/1       Completed   0          15h
python-app-38-build    0/1       Completed   0          20m
python-app-51-deploy   0/1       Error       0          8d
python-app-76-vfnmx    1/1       Running     0          21m
python-app-77-deploy   0/1       Error       0          17m

Browse > Deployments > python-app > Actions > Set Resource Limits is set to 1GB. 

Browse > Deployments > mongodb > Actions > Set Resource Limits is set to 1GB. 

Browse > Deployments > python-app > Edit YAML:

spec:
  strategy:
    type: Rolling
    rollingParams:
      updatePeriodSeconds: 1
      intervalSeconds: 1
      timeoutSeconds: 600
      maxUnavailable: 25%
      maxSurge: 25%
    resources: {  }

Browse > Deployments > mongodb > Edit YAML:
    
spec:
  strategy:
    type: Recreate
    recreateParams:
      timeoutSeconds: 600
    resources: {  }


It is my understanding that for dev preview I have 4GB memory available for the whole project and 10GB storage. 

Extensive troubleshooting around these issues is here:

https://groups.google.com/forum/#!topic/openshift/ZZBe8REFT2E