Bug 1315865 - ResourceQuota is preventing the deployment of pods
ResourceQuota is preventing the deployment of pods
Status: CLOSED WORKSFORME
Product: OpenShift Container Platform
Classification: Red Hat
Component: Deployments (Show other bugs)
3.1.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Paul Weil
zhou ying
: UpcomingRelease
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-08 15:04 EST by Eric Jones
Modified: 2016-04-08 08:10 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
OpenShift Enterprise 3.1
Last Closed: 2016-04-08 08:10:32 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Eric Jones 2016-03-08 15:04:53 EST
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create new project (mytest)
2. Docker pull/tag/push image (jenkins)
3. Set up the following resourcequota:
  apiVersion: "v1"
  kind: "ResourceQuota"
  metadata:
    name: "default-quota"
  spec:
    hard:
      memory: "2560Mi"
      cpu: "10"
      pods: "5"
      services: "5"
      replicationcontrollers: "5"
      resourcequotas: "1"
      persistentvolumeclaims: "5"
4. Try to deploy pod (oc new-app --image=mytest/jenkins-1-rhel7)


Actual results:
See the dc, rc, svc, and other components but no pods
Try to deploy newest version (oc deploy jenkins-1-rhel7 --latest) and receive the following error:


Expected results:
Deploy the pod as well.

Additional info:
As soon as the resourcequota is deleted from the project the pods start to deploy.
Comment 1 zhou ying 2016-03-09 05:07:07 EST
when the deployment can't be created due to a quota limit will get event from original report.

oc get events |grep Failed

3m          2m         6         deployment-example-2          ReplicationController                                         FailedCreate        {deployer }                              Error creating deployer pod for zhouy/deployment-example-1: Pod "deployment-example-1-deploy" is forbidden: must make a non-zero request for memory since it is tracked by quota.
Comment 2 Paul Weil 2016-03-09 08:13:28 EST
Eric, did you receive an event telling you that you must make a non-zero request for memory since you have enforced memory quota limits?
Comment 3 Boris Kurktchiev 2016-03-09 13:44:02 EST
So I originally reported this as a support case and Eric submitted the ticket on my behalf. When this is happening at no point do i see  Error creating deployer pod for zhouy/deployment-example-1: Pod "deployment-example-1-deploy" is forbidden: must make a non-zero request for memory since it is tracked by quota. in my logs

Running: oc v3.1.1.6-21-gcd70c35 kubernetes v1.1.0-origin-1107-g4c8e6f4

The only thing that the event log keeps repeating is: 
Failed for reason PodExceedsFreeCPU and possibly others 
Failed for reason MatchNodeSelector and possibly others
Failed for reason Region and possibly others

As soon as I oc delete default-quota; oc delete the-offending-pod, the pod deploys as it should.
Comment 4 Eric Jones 2016-03-09 14:36:35 EST
In my environment, I retested everything in a new project with the quota set up and when I tried to deploy the application I did have a similar error message in `oc get events`. I also tried to scale the rc up to 1 as it deployed with 0 replicas. 
Below is the `oc get events` output after both of those:

└──> oc get event
FIRSTSEEN   LASTSEEN   COUNT     NAME              KIND                    SUBOBJECT   REASON         SOURCE                      MESSAGE
2m          11s        15        jenkins-rhel7-1   ReplicationController               failedCreate   {deployer }                 Error creating deployer pod for ugh/jenkins-rhel7-1: Pod "jenkins-rhel7-1-deploy" is forbidden: must make a non-zero request for memory since it is tracked by quota.
13s         9s         3         jenkins-rhel7-1   ReplicationController               FailedCreate   {replication-controller }   Error creating: Pod "jenkins-rhel7-1-" is forbidden: must make a non-zero request for memory since it is tracked by quota.
Comment 5 Paul Weil 2016-03-14 17:14:56 EDT
Boris, the messages you say are repeating look like they are failures from scheduling rather than a quota admission error.  I was able to confirm that the message Eric and Zhou were seeing should be in the 3.1.1 release (it has been there for quite some time).  

Could you reproduce this and paste/gist/fpaste the full output of oc get events and the master log (running at least loglevel 4)?
Comment 6 Boris Kurktchiev 2016-04-07 14:33:12 EDT
Seems with the latest RPM release -25 I am no longer seeing this. I guess can close the bug and if I run into it again I will either reopen it here or send a new case to RH support.

Note You need to log in before you can comment on or make changes to this bug.