| Summary: | [online]Jenkins pod is CrashLoopBackOff when create jenkins application | ||
|---|---|---|---|
| Product: | OpenShift Online | Reporter: | XiuJuan Wang <xiuwang> |
| Component: | Image | Assignee: | Troy Dawson <tdawson> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Wang Haoran <haowang> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.x | CC: | aos-bugs, bmeng, bparees, cdigiovanni, dcbw, jokerman, mmccomas, rcarvalh, sspeiche, wewang, wzheng, xiuwang |
| Target Milestone: | --- | Flags: | xiuwang:
needinfo-
|
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1318510 | Environment: | |
| Last Closed: | 2016-05-23 15:08:46 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Bug Depends On: | 1318510 | ||
| Bug Blocks: | |||
|
Description
XiuJuan Wang
2016-04-20 07:41:15 UTC
Since the bug 1318510 in ose has been fixed almostly, so clone a new bug for online. The registry.access.redhat.com/openshift3/jenkins-1-rhel7(908b6dd3dafb) in online is too old, 7 weeks ago. Are you actually hitting this issue as described in online? @bparees indicates that there were changes made in between those versions that should have affected this. Yes,I am hitiing this issue in online. online is using registry.access.redhat.com/openshift3/jenkins-1-rhel7(908b6dd3dafb), It's 7 weeks ago,is too old. I know this bug #1318510 has been fixed partly with brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/jenkins-1-rhel7.Just open this new bug to track issue in online. today test in OSE-3.2 - RPM Install - RHEL-7.2 - Multitenant - GCE met the same problem registry.access.redhat.com/openshift3/jenkins-1-rhel7 908b6dd3dafb steps as follow: $oc new-app -f https://raw.githubusercontent.com/openshift/origin/master/examples/jenkins/jenkins-ephemeral-template.json # oc get pods NAME READY STATUS RESTARTS AGE jenkins-1-deploy 1/1 Running 0 1m jenkins-1-ijh0t 0/1 CrashLoopBackOff 1 1m [root@dhcp-128-91 backup]# oc describe pod jenkins-1-ijh0t Name: jenkins-1-ijh0t Namespace: wewang Node: qe-shared-master-registry-etcd-1/10.240.0.11 Start Time: Wed, 27 Apr 2016 10:04:59 +0800 Labels: deployment=jenkins-1,deploymentconfig=jenkins,name=jenkins Status: Running IP: 10.2.3.11 Controllers: ReplicationController/jenkins-1 Containers: jenkins: Container ID: docker://55b2c5688a615d0566f0b2183c52e81b8f70d1dbf8cf1914c57049cb31ab3846 Image: registry.access.redhat.com/openshift3/jenkins-1-rhel7:latest Image ID: docker://908b6dd3dafbabbb1cf38b60bb8a281988c04f4953854df19e0ed804fe9d4dfa Port: QoS Tier: cpu: BestEffort memory: Guaranteed Limits: memory: 512Mi Requests: memory: 512Mi State: Running Started: Wed, 27 Apr 2016 10:07:13 +0800 Last State: Terminated Reason: Error Exit Code: 137 Started: Wed, 27 Apr 2016 10:06:24 +0800 Finished: Wed, 27 Apr 2016 10:06:50 +0800 Ready: False Restart Count: 3 Liveness: http-get http://:8080/login delay=30s timeout=3s period=10s #success=1 #failure=3 Readiness: http-get http://:8080/login delay=3s timeout=3s period=10s #success=1 #failure=3 Environment Variables: JENKINS_PASSWORD: password Conditions: Type Status Ready False Volumes: jenkins-data: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: default-token-ll79s: Type: Secret (a volume populated by a Secret) SecretName: default-token-ll79s Events: FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 2m 2m 1 {default-scheduler } Normal Scheduled Successfully assigned jenkins-1-ijh0t to qe-shared-master-registry-etcd-1 2m 2m 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Pulling pulling image "registry.access.redhat.com/openshift3/jenkins-1-rhel7:latest" 1m 1m 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Pulled Successfully pulled image "registry.access.redhat.com/openshift3/jenkins-1-rhel7:latest" 1m 1m 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Created Created container with docker id 236c100c585e 1m 1m 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Started Started container with docker id 236c100c585e 1m 1m 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Created Created container with docker id 973805102a71 1m 1m 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Started Started container with docker id 973805102a71 1m 1m 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Warning Unhealthy Readiness probe failed: Get http://10.2.3.11:8080/login: dial tcp 10.2.3.11:8080: connection refused 1m 1m 2 {kubelet qe-shared-master-registry-etcd-1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with CrashLoopBackOff: "Back-off 10s restarting failed container=jenkins pod=jenkins-1-ijh0t_wewang(71f745e1-0c1c-11e6-a122-42010af0000e)" 59s 59s 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Created Created container with docker id e8579d9573a1 58s 58s 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Started Started container with docker id e8579d9573a1 1m 52s 2 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Warning Unhealthy Readiness probe failed: HTTP probe failed with statuscode: 503 1m 40s 2 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Warning Unhealthy Readiness probe failed: Get http://10.2.3.11:8080/login: read tcp 10.2.3.11:8080: use of closed network connection 32s 32s 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Warning Unhealthy Readiness probe failed: Get http://10.2.3.11:8080/login: read tcp 10.2.3.11:8080: connection reset by peer 1m 23s 4 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Warning BackOff Back-off restarting failed docker container 31s 23s 2 {kubelet qe-shared-master-registry-etcd-1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with CrashLoopBackOff: "Back-off 20s restarting failed container=jenkins pod=jenkins-1-ijh0t_wewang(71f745e1-0c1c-11e6-a122-42010af0000e)" 1m 9s 3 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Pulled Container image "registry.access.redhat.com/openshift3/jenkins-1-rhel7:latest" already present on machine 9s 9s 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Created Created container with docker id 55b2c5688a61 9s 9s 1 {kubelet qe-shared-master-registry-etcd-1} spec.containers{jenkins} Normal Started Started container with docker id 55b2c5688a61 I confirmed that brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/jenkins-1-rhel7 does not crashloop against the readiness probe with the current template. so this should be good to go once we publish the new image. assigning to Troy. *** Bug 1331617 has been marked as a duplicate of this bug. *** @Ben
registry.access.redhat.com/openshift3/jenkins-1-rhel7:latest has been updated to 37d7c8d851b9(13 days ago).
Jenkins pod with persistent volume could be running in online env now.
Just have some unhealthy messege in events.Please help to set the bug to on_qa.
Thanks!
7m 7m 1 {kubelet ip-172-31-14-21.ec2.internal} spec.containers{jenkins} Warning Unhealthy Readiness probe failed: Get http://10.1.11.10:8080/login: dial tcp 10.1.11.10:8080: connection refused
7m 6m 3 {kubelet ip-172-31-14-21.ec2.internal} spec.containers{jenkins} Warning Unhealthy Readiness probe failed: HTTP probe failed with statuscode: 503
6m 6m 1 {kubelet ip-172-31-14-21.ec2.internal} spec.containers{jenkins} Warning Unhealthy Liveness probe failed: HTTP probe failed with statuscode: 503
14m 14m 1 jenkins DeploymentConfig Warning FailedUpdate {deployment-controller } Cannot update deployment xiuwang/jenkins-1 status to Pending: replicationcontrollers "jenkins-1" cannot be updated: the object has been modified; please apply your changes to the latest version and try again
According to comment #7, move this bug to verified. |