Bug 1338679 - Couldn't find any error from events when new deployer/builder pod failed due to quota limit
Summary: Couldn't find any error from events when new deployer/builder pod failed due ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Pod
Version: 3.x
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Michail Kargakis
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-23 09:10 UTC by zhou ying
Modified: 2016-09-19 13:49 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-19 13:49:17 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description zhou ying 2016-05-23 09:10:44 UTC
Description of problem:
when start new deployment, the deployer pod was not created, but  can't  get any event from original report, we should get events show the explicit forbidden info.

Version-Release number of selected component (if applicable):
openshift v1.3.0-alpha.0-631-gec0fb4d
kubernetes v1.3.0-alpha.1-331-g0522e63
etcd 2.3.0

How reproducible:
always

Steps to Reproduce:
1. Start openshift and login;
2. Create a dc:
`oc create -f deployment-with-resources.json`
3. After the deployment completed, create a quota/limits as cluster-admin:
`oc create -f /data/src/github.com/openshift/origin/examples/project-quota/quota.yaml -n zhouy --config=openshift.local.config/master/admin.kubeconfig`
`oc create -f  /data/src/github.com/openshift/origin/examples/project-quota/limits.yaml  -n zhouy --config=openshift.local.config/master/admin.kubeconfig`
4. Manuly start a deployment, check the result:
`oc deploy hooks --latest`
5. Check events from original reported

Actual results:
4. Prompted the new deployment started ,but check pods and dc , no deploy pod created.
5. Couldn't  find events about the failed deployment;

Expected results:
5. Can get events from original reported,show "hooks-1-deploy" is forbidden: XXX by quota

Additional info:
{
    "kind": "DeploymentConfig",
    "apiVersion": "v1",
    "metadata": {
        "name": "hooks",
        "creationTimestamp": null,
        "labels": {
            "name": "hello-openshift"
        }
    },
    "spec": {
        "strategy": {
            "type": "Recreate",
            "recreateParams": {
            },
            "resources": {
                 "limits": {
                     "cpu": "1400m",
                     "memory": "1500Mi" 
                 }
           }
        },
        "triggers": [
            {
                "type": "ConfigChange"
            }
        ],
        "replicas": 1,
        "selector": {
            "name": "hello-openshift"
        },
        "template": {
            "metadata": {
                "creationTimestamp": null,
                "labels": {
                    "name": "hello-openshift"
                }
            },
            "spec": {
                "volumes": [
                    {
                        "name": "dataem",
                        "emptyDir": {
                           "medium": ""
                        }
                    }
                ],
                "containers": [
                    {
                        "name": "hello-openshift",
                        "image": "openshift/hello-openshift",
                        "ports": [
                            {
                                "containerPort": 8080,
                                "protocol": "TCP"
                            }
                        ],
                        "env": [
                            {
                                "name": "MYSQL_USER",
                                "value": "user8Y2"
                            },
                            {
                                "name": "MYSQL_PASSWORD",
                                "value": "Plqe5Wev"
                            },
                            {
                                "name": "MYSQL_DATABASE",
                                "value": "root"
                            }
                        ],
                        "resources": {
                            "limits": {
                                "cpu": "400m",
                                "memory": "200Mi" 
                            }
                        },
                        "volumeMounts": [
                            {
                                "name": "dataem",
                                "mountPath": "/var/lib/etcd"
                            }
                        ],
                        "terminationMessagePath": "/dev/termination-log",
                        "imagePullPolicy": "Always",
                        "securityContext": {
                            "capabilities": {},
                            "privileged": false
                        }
                    }
                ],
                "restartPolicy": "Always",
                "dnsPolicy": "ClusterFirst"
            }
        }
    },
    "status": {}
}

Comment 1 openshift-github-bot 2016-05-23 12:45:39 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/fdbc0c1a242adca62cc36f0f7a29031fc7a96e22
Bug 1338679: emit events on failure to create a deployer pod

Comment 2 zhou ying 2016-05-24 02:09:34 UTC
Confirmed with ami:	
devenv-rhel7_4264, the issue has fixed.

[zhouy@zhouy v3]$ oc get events |grep Failed
10m         10m         1         deployment-example            DeploymentConfig                                                    Warning   FailedRetry         {deployments-controller }               deployment-example-2: About to stop retrying deployment-example-2: couldn't create deployer pod for zhouy/deployment-example-2: pods "deployment-example-2-deploy" is forbidden: Failed quota: quota: must specify cpu,memory
<invalid>   <invalid>   1         hooks                         DeploymentConfig                                                    Warning   FailedRetry         {deployments-controller }               hooks-2: About to stop retrying hooks-2: couldn't create deployer pod for zhouy/hooks-2: pods "hooks-2-deploy" is forbidden: [Maximum cpu usage per Pod is 500m, but limit is 1400m., Maximum memory usage per Pod is 750Mi, but limit is 1572864k., Maximum cpu usage per Container is 500m, but limit is 1400m., Maximum memory usage per Container is 750Mi, but limit is 1500Mi.]


Note You need to log in before you can comment on or make changes to this bug.