Description of problem: 'oc attach' to a running pod hangs out and didn't return output of remote container Version-Release number of selected component (if applicable): oc v3.3.0.4 kubernetes v1.3.0+57fb9ac features: Basic-Auth GSSAPI Kerberos SPNEGO How reproducible: Always Steps to Reproduce: 1. Create a test pod # oc run mypod --image=aosqe/hello-openshift --generator=run-pod/v1 2.Check pod status # oc get pods NAME READY STATUS RESTARTS AGE mypod 1/1 Running 0 20s 3. Attach to the pod container and print logs # oc attach mypod --loglevel=6 I0713 18:43:04.143179 10365 loader.go:327] Config loaded from file /home/yapei/.kube/config I0713 18:43:05.805266 10365 round_trippers.go:318] GET https://master:8443/api 200 OK in 1661 milliseconds I0713 18:43:06.215094 10365 round_trippers.go:318] GET https://master:8443/apis 200 OK in 409 milliseconds I0713 18:43:06.558606 10365 round_trippers.go:318] GET https://master:8443/api 200 OK in 343 milliseconds I0713 18:43:06.902311 10365 round_trippers.go:318] GET https://master:8443/apis 200 OK in 343 milliseconds I0713 18:43:07.341622 10365 round_trippers.go:318] GET https://master:8443/api/v1/namespaces/yapei/pods/mypod 200 OK in 438 milliseconds I0713 18:43:07.342506 10365 attach.go:238] defaulting container name to mypod I0713 18:43:08.933794 10365 round_trippers.go:318] POST https://master:8443/api/v1/namespaces/yapei/pods/mypod/attach?container=mypod&stderr=true&stdin=true&stdout=true 101 Switching Protocols in 1591 milliseconds I0713 18:43:08.933820 10365 round_trippers.go:318] POST https://master:8443/api/v1/namespaces/yapei/pods/mypod/attach?container=mypod&stderr=true&stdin=true&stdout=true 101 Switching Protocols in 1591 milliseconds Actual results: 3. 'oc attach' hangs and didn't return any output from container Expected results: 3. Should get correct output from remote container Additional info:
This will be fixed when bug 1317784 is fixed. Additionally, we probably aren't going to be able to make this work with 100% of the images out there. For example, the hello-openshift image doesn't even work with `docker attach` using the example you have above. If you want to test attach once bug 1317784 is fixed, please use some other image like centos:centos7 instead of hello-openshift. I'm also removing the regression keyword because it isn't a regression.
That's ok, thanks for your info
Checked against v3.3.0.14 I tried image centos:centos7, but the pod can't be started # oc run testpod --image=centos:centos7 --generator=run-pod/v1 pod "testpod" created # oc get pods NAME READY STATUS RESTARTS AGE testpod 0/1 Completed 4 1m # oc get pods NAME READY STATUS RESTARTS AGE testpod 0/1 CrashLoopBackOff 4 2m Here is verification steps with another image(i'm sure this image docker.io/deshuai/hello-pod:latest supports docker attach, because in v3.2.1.12 "oc attach hello-pod" return correct strings "serving on 8888,serving on 8080"): 1. Download test file # curl -k -O https://raw.githubusercontent.com/mdshuai/testfile-openshift/master/k8s/hello-pod.yaml 2. Create pod # oc create -f hello-pod.yaml pod "hello-pod" created 3. Check pod is running # oc get pods NAME READY STATUS RESTARTS AGE hello-pod 1/1 Running 0 2m 4. Check attach # oc attach hello-pod If you don't see a command prompt, try pressing enter. Still couldn't get any output
"oc run testpod --image=centos:centos7 --generator=run-pod/v1" isn't a valid test case for this bz because: 1) You are using the default command for the image (bash), AND 2) You are not attaching stdin (-i) Therefore, the container exits immediately. This is the same as 'docker run centos:centos7' - you will see it exit immediately. You alternatively could do: (in one terminal) 1. oc run -it testpod --image=centos:centos7 --generator=run-pod/v1 2. yum -y install less 3. less /etc/services (in another terminal) 1. oc attach -it testpod (you should see the output from the "less" command) As for hello-pod, that does not work with either 'oc attach' or 'docker attach', so I don't think it's a valid test image.
Works fine with those steps: (in one terminal) 1. oc run -it testpod --image=centos:centos7 --generator=run-pod/v1 (in another terminal) 1. oc attach -it testpod openshift v3.3.0.16 kubernetes v1.3.0+507d3a7 etcd 2.3.0+git
Thanks wmeng, Andy Move to VERIFIED
It turns out there are two parts to this issue: 1) If you run a container with a TTY, it used to be that when you detached and reattached, you would have to press a key (e.g. enter) to get the screen to refresh and show you what was there. We fixed that with bug 1317784. 2) In 3.2, we used fsouza/go-dockerclient to talk to docker, and that had the option to ask docker to display the container's logs immediately after attaching. In 3.3, we switched to docker's engine-api, and the logs option is no longer available. This means that if you now attach to a container like hello-openshift and you want to see what was written to the terminal prior to attaching, you won't be able to any more (but you could get this data via 'oc logs'). I have opened a PR to docker (https://github.com/docker/docker/pull/26718) to add a logs option to the client API. Assuming it gets merged, we'd then need to update Kubernetes to use it, and then Origin. Derek, how do you want to handle this bz? Should treat it as a regression (because of 2) and move it to UpcomingRelease and target 3.5? Or something else?
It's not going to be fixed until docker accepts your PR, so let's wait and see what happens there first.
Reproduced on ded-int-aws OpenShift Master: v3.3.1.3 Kubernetes Master: v1.3.0+52492b4
FYI, searched this bug and did a regression check. Results are below. 3.4 (v3.4.1.27) still reproduces. 3.5 (v3.5.5.18), 3.6 (v3.6.84) fixed the bug (BTW, comment 7's PR was merged): $ oc run mypod --image=aosqe/hello-openshift --generator=run-pod/v1 $ oc attach mypod # after pod running If you don't see a command prompt, try pressing enter. serving on 8081 serving on 8888
Comment 7's PR was backported in https://github.com/openshift/origin/pull/12648 (3.5+).