Bug 1324900

Summary: Pod is waiting to start: ContainerCreating
Product: OpenShift Container Platform Reporter: Weibin Liang <weliang>
Component: NodeAssignee: Andy Goldstein <agoldste>
Status: CLOSED NOTABUG QA Contact: DeShuai Ma <dma>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.2.0CC: aos-bugs, jokerman, mmccomas, weliang
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-18 12:52:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Weibin Liang 2016-04-07 14:24:31 UTC
Description of problem:
All pods are in ContainerCreating status

Version-Release number of selected component (if applicable):
openshift v3.2.0.12 (from http://download.lab.bos.redhat.com/rcm-guest/puddles/RHAOS/AtomicOpenShift/3.2/latest/x86_64/os)
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5
docker-1.9.1-25.el7.x86_64
How reproducible:
Always

Steps to Reproduce:
1.ansible-playbook playbooks/byo/config.yml
[root@master3 ~]# oc get nodes
NAME                  STATUS                     AGE
master3.example.com   Ready,SchedulingDisabled   47s
node3.example.com     Ready                      48s

2. oc create services and pods
[root@master3 ~]# oc create -f ./hello-service1.json
service "hello-service1" created
[root@master3 ~]# oc create -f ./hello-service-pods1.json
pod "hello-openshift1-1" created
pod "hello-openshift1-2" created
pod "hello-openshift1-3" created

3.check services and pods
[root@master3 ~]# oc get services
NAME             CLUSTER-IP     EXTERNAL-IP   PORT(S)                 AGE
hello-service1   172.30.8.180   <none>        8080/TCP                10m
kubernetes       172.30.0.1     <none>        443/TCP,53/UDP,53/TCP   16m
[root@master3 ~]# oc get pods
NAME                 READY     STATUS              RESTARTS   AGE
hello-openshift1-1   0/1       ContainerCreating   0          14s
hello-openshift1-2   0/1       ContainerCreating   0          14s
hello-openshift1-3   0/1       ContainerCreating   0          14s
[root@master3 ~]# oc logs hello-openshift1-1
Error from server: container "hello-openshift1" in pod "hello-openshift1-1" is waiting to start: ContainerCreating
[root@master3 ~]# oc describe pod hello-openshift1-1
Name:		hello-openshift1-1
Namespace:	default
Node:		node3.example.com/10.18.41.202
Start Time:	Thu, 07 Apr 2016 10:09:47 -0400
Labels:		name=hello-openshift1
Status:		Pending
IP:		
Controllers:	<none>
Containers:
  hello-openshift1:
    Container ID:	
    Image:		openshift/hello-openshift:v1.0.6
    Image ID:		
    Port:		8080/TCP
    QoS Tier:
      cpu:	Guaranteed
      memory:	Guaranteed
    Limits:
      cpu:	10m
      memory:	16Mi
    Requests:
      memory:		16Mi
      cpu:		10m
    State:		Waiting
      Reason:		ContainerCreating
    Ready:		False
    Restart Count:	0
    Environment Variables:
Conditions:
  Type		Status
  Ready 	False 
Volumes:
  default-token-1lvdp:
    Type:	Secret (a volume populated by a Secret)
    SecretName:	default-token-1lvdp
Events:
  FirstSeen	LastSeen	Count	From				SubobjectPath	Type		Reason		Message
  ---------	--------	-----	----				-------------	--------	------		-------
  2m		2m		1	{default-scheduler }				Normal		Scheduled	Successfully assigned hello-openshift1-1 to node3.example.com
  2m		53s		4	{kubelet node3.example.com}			Warning		FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "Error: image openshift3/ose-pod:v3.2.0.12 not found"

  1m	5s	7	{kubelet node3.example.com}		Warning	FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "POD" with ImagePullBackOff: "Back-off pulling image \"openshift3/ose-pod:v3.2.0.12\""


Actual results:
Pod is in ContainerCreating status due to "Error: image openshift3/ose-pod:v3.2.0.12 not found"

Expected results:
Pod should be in running status

Additional info:

Comment 2 Weibin Liang 2016-04-07 15:30:45 UTC
[root@master3 openshift-ansible]# rpm -qa | grep openshift
tuned-profiles-atomic-openshift-node-3.2.0.11-1.git.0.6696e29.el7.x86_64
openshift-ansible-roles-3.0.47-6.git.0.7e39163.el7aos.noarch
openshift-ansible-3.0.47-6.git.0.7e39163.el7aos.noarch
openshift-ansible-filter-plugins-3.0.47-6.git.0.7e39163.el7aos.noarch
openshift-ansible-playbooks-3.0.47-6.git.0.7e39163.el7aos.noarch
atomic-openshift-clients-3.2.0.11-1.git.0.6696e29.el7.x86_64
atomic-openshift-master-3.2.0.11-1.git.0.6696e29.el7.x86_64
atomic-openshift-node-3.2.0.11-1.git.0.6696e29.el7.x86_64
atomic-openshift-sdn-ovs-3.2.0.11-1.git.0.6696e29.el7.x86_64
atomic-openshift-3.2.0.11-1.git.0.6696e29.el7.x86_64
openshift-ansible-lookup-plugins-3.0.47-6.git.0.7e39163.el7aos.noarch
atomic-openshift-utils-3.0.47-6.git.0.7e39163.el7aos.noarch
[root@master3 openshift-ansible]# docker version
Client:
 Version:         1.9.1
 API version:     1.21
 Package version: docker-1.9.1-25.el7.x86_64
 Go version:      go1.4.2
 Git commit:      78ee77d/1.9.1
 Built:           
 OS/Arch:         linux/amd64

Server:
 Version:         1.9.1
 API version:     1.21
 Package version: docker-1.9.1-25.el7.x86_64
 Go version:      go1.4.2
 Git commit:      78ee77d/1.9.1
 Built:           
 OS/Arch:         linux/amd64
[root@master3 openshift-ansible]#

Comment 3 Andy Goldstein 2016-04-07 15:32:13 UTC
What do you have in /etc/sysconfig/docker?

Comment 4 Weibin Liang 2016-04-07 15:41:41 UTC
[root@master3 ~]# cat /etc/sysconfig/docker
# /etc/sysconfig/docker

# Modify these options if you want to change the way the docker daemon runs
OPTIONS=' --selinux-enabled --insecure-registry=172.30.0.0/16 '
DOCKER_CERT_PATH=/etc/docker

# If you want to add your own registry to be used for docker search and docker
# pull use the ADD_REGISTRY option to list a set of registries, each prepended
# with --add-registry flag. The first registry added will be the first registry
# searched.
ADD_REGISTRY='--add-registry registry.access.redhat.com'

# If you want to block registries from being used, uncomment the BLOCK_REGISTRY
# option and give it a set of registries, each prepended with --block-registry
# flag. For example adding docker.io will stop users from downloading images
# from docker.io
# BLOCK_REGISTRY='--block-registry'

# If you have a registry secured with https but do not have proper certs
# distributed, you can tell docker to not look for full authorization by
# adding the registry to the INSECURE_REGISTRY line and uncommenting it.
# INSECURE_REGISTRY='--insecure-registry'

# On an SELinux system, if you remove the --selinux-enabled option, you
# also need to turn on the docker_transition_unconfined boolean.
# setsebool -P docker_transition_unconfined 1

# Location used for temporary files, such as those created by
# docker load and build operations. Default is /var/lib/docker/tmp
# Can be overriden by setting the following environment variable.
# DOCKER_TMPDIR=/var/tmp

# Controls the /etc/cron.daily/docker-logrotate cron job status.
# To disable, uncomment the line below.
# LOGROTATE=false
[root@master3 ~]#

Comment 7 DeShuai Ma 2016-04-15 04:59:41 UTC
Hi wei, could you help verify this bug?

Comment 8 Weibin Liang 2016-04-18 12:52:40 UTC
It works fine when I configured through Jenkins job.