Bug 1364017 - [infrastructure_public_278]Can't attach/debug a init container.
Summary: [infrastructure_public_278]Can't attach/debug a init container.
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Andy Goldstein
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-04 09:37 UTC by DeShuai Ma
Modified: 2016-09-19 14:32 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-19 14:32:45 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1933 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.3 Release Advisory 2016-09-27 13:24:36 UTC
Origin (Github) 10578 None None None 2016-08-22 20:10:24 UTC

Description DeShuai Ma 2016-08-04 09:37:45 UTC
Description of problem:
Use oc client can't attach or debug a init container.

Version-Release number of selected component (if applicable):
openshift v3.3.0.14
kubernetes v1.3.0+57fb9ac
etcd 2.3.0+git

How reproducible:
Always

Steps to Reproduce:
1.Create a pod with init container.
$ oc create -f https://raw.githubusercontent.com/mdshuai/testfile-openshift/master/initContainer/init-containers-sleep.yaml

2.When the init container is running, attach/debug the init container
$ oc attach hello-pod -c sleep 
$ oc debug pod/hello-pod -c sleep -- /bin/env


Actual results:
[root@dhcp-128-7 dma]# oc attach hello-pod -c sleep 
error: pod hello-pod is not running and cannot be attached to; current phase is Pending
[root@dhcp-128-7 dma]# oc debug pod/hello-pod -c sleep -- /bin/env
error: the container "sleep" is not a valid container name; must be one of [hello-pod]

Expected results:
2. Should attach/debug init container as normal app container.

Additional info:
[root@dhcp-128-7 dma]# oc describe pod/hello-pod
Name:			hello-pod
Namespace:		dma1
Security Policy:	restricted
Node:			openshift-123.lab.sjc.redhat.com/192.168.1.187
Start Time:		Thu, 04 Aug 2016 16:11:48 +0800
Labels:			name=hello-pod
Status:			Pending
IP:			10.1.1.2
Controllers:		<none>
Init Containers:
  sleep:
    Container ID:	docker://8bcac0650096391318a529991fd9787903aa5f78f61e71701b4b617b0b3d211a
    Image:		centos:centos7
    Image ID:		docker://sha256:97063303644439d9cea259b0e5f4b468633c90d88bf526acc67e5ae0a6e9427c
    Port:		
    Command:
      /bin/sh
      -c
      while :; do sleep 2; echo hello init container; done
    State:		Running
      Started:		Thu, 04 Aug 2016 16:11:52 +0800
    Ready:		False
    Restart Count:	0
    Volume Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-aur1j (ro)
    Environment Variables:	<none>
Containers:
  hello-pod:
    Container ID:	
    Image:		docker.io/deshuai/hello-pod:latest
    Image ID:		
    Port:		8080/TCP
    State:		Waiting
      Reason:		PodInitializing
    Ready:		False
    Restart Count:	0
    Volume Mounts:
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-aur1j (ro)
    Environment Variables:	<none>
Conditions:
  Type		Status
  Initialized 	False 
  Ready 	False 
  PodScheduled 	True 
Volumes:
  tmp:
    Type:	EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:	
  default-token-aur1j:
    Type:	Secret (a volume populated by a Secret)
    SecretName:	default-token-aur1j
QoS Tier:	BestEffort
Events:
  FirstSeen	LastSeen	Count	From						SubobjectPath			Type		Reason		Message
  ---------	--------	-----	----						-------------			--------	------		-------
  32m		32m		1	{default-scheduler }								Normal		Scheduled	Successfully assigned hello-pod to openshift-123.lab.sjc.redhat.com
  32m		32m		1	{kubelet openshift-123.lab.sjc.redhat.com}	spec.initContainers{sleep}	Normal		Pulled		Container image "centos:centos7" already present on machine
  32m		32m		1	{kubelet openshift-123.lab.sjc.redhat.com}	spec.initContainers{sleep}	Normal		Created		Created container with docker id 8bcac0650096
  32m		32m		1	{kubelet openshift-123.lab.sjc.redhat.com}	spec.initContainers{sleep}	Normal		Started		Started container with docker id 8bcac0650096

Comment 1 DeShuai Ma 2016-08-04 09:42:37 UTC
Before init container is complete, pod status is pending. In this status 'oc rsh,rsync,logs' all can success, only attach error with pod 'Pending'

Comment 2 Clayton Coleman 2016-08-15 22:00:15 UTC
https://github.com/openshift/origin/pull/10427

Comment 3 DeShuai Ma 2016-08-16 01:53:50 UTC
when pr merge will verify this bug.

Comment 4 DeShuai Ma 2016-08-19 07:20:09 UTC
this bug just merged in origin, still need wait to merge on ose.

Comment 5 DeShuai Ma 2016-08-22 05:22:33 UTC
Test on v3.3.0.23, oc attach has fixed, `oc debug` still can't debug init container.

[root@dhcp-128-7 Desktop]# oc debug pod/hello-pod -c sleep -- /bin/env
error: the container "sleep" is not a valid container name; must be one of [hello-pod]

Comment 6 Andy Goldstein 2016-08-22 20:10:24 UTC
Fixes for debugging init containers: https://github.com/openshift/origin/pull/10578

Comment 8 DeShuai Ma 2016-08-25 06:54:02 UTC
Test on v3.3.0.25+d2ac65e-dirty, verify this bug.
[root@qe-wmeng3-master-container-1 ~]# oc debug -n dma pod/hello-pod -c sleep -- /bin/env
Debugging with pod/hello-pod-debug, original command: /bin/sh -c while :; do sleep 2; echo hello init container; done
Waiting for pod to start ...
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=hello-pod-debug
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_53_UDP=udp://172.30.0.1:53
KUBERNETES_PORT_53_UDP_PROTO=udp
KUBERNETES_PORT_53_TCP=tcp://172.30.0.1:53
KUBERNETES_SERVICE_HOST=172.30.0.1
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT=tcp://172.30.0.1:443
KUBERNETES_PORT_53_UDP_ADDR=172.30.0.1
KUBERNETES_PORT_53_TCP_ADDR=172.30.0.1
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_ADDR=172.30.0.1
KUBERNETES_PORT_53_TCP_PORT=53
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_PORT_DNS_TCP=53
KUBERNETES_PORT_443_TCP=tcp://172.30.0.1:443
KUBERNETES_SERVICE_PORT_DNS=53
KUBERNETES_PORT_53_UDP_PORT=53
KUBERNETES_PORT_53_TCP_PROTO=tcp
HOME=/root

Removing debug pod ...


Note You need to log in before you can comment on or make changes to this bug.