Red Hat Bugzilla – Bug 1301199
docker adds distro tag to docker version which causes kubelet to fail
Last modified: 2016-03-31 19:23:32 EDT
Description of problem: Due to recent changes in kube upstream (https://github.com/kubernetes/kubernetes/pull/19675), the container run time status is being verified based on docker's Version instead of APIVersion as done before. Since docker on fedora or rhel appends distro string (1.9.1-fc22/el7) to its version, kube parser fails to parse as the last part 1-fc22 is not integer. It was not an issue before as I said because APIVersion was being used which does not have this issue. For example: $ docker version Client: Version: 1.9.1-fc22 API version: 1.21 Package version: docker-1.9.1-4.git64eb95e.fc22.x86_64 Go version: go1.5.3 Git commit: 64eb95e/1.9.1 Built: OS/Arch: linux/amd64 Server: Version: 1.9.1-fc22 API version: 1.21 Package version: docker-1.9.1-4.git64eb95e.fc22.x86_64 Go version: go1.5.3 Git commit: 64eb95e/1.9.1 Built: OS/Arch: linux/amd64 Here are some errors, I noticed on kube nodes: E0122 10:24:18.435492 27245 manager.go:986] docker: failed to parse docker server version "1.9.1-fc22": Unable to parse version "1.9.1-fc22": "1-fc22" is not an integer E0122 10:24:23.273679 27245 manager.go:986] docker: failed to parse docker server version "1.9.1-fc22": Unable to parse version "1.9.1-fc22": "1-fc22" is not an integer E0122 10:24:23.273692 27245 kubelet.go:2579] Container runtime sanity check failed: docker: failed to parse docker server version "1.9.1-fc22": Unable to parse version "1.9.1-fc22": "1-fc22" is not an integer Detailed logs: http://pastebin.test.redhat.com/342892 As container run time status fails, all nodes in kube cluster are going down from Ready to NotReady status (it takes almost 5m in my setup). Version-Release number of selected component (if applicable): docker-1.9.1-el7 How reproducible: always Steps to Reproduce: 1. git clone kubernetes which has https://github.com/kubernetes/kubernetes/pull/19675 2. cd kubernetes 3. make clean all 4. ./hack/local-up-cluster.sh 5. kubectl get nodes (wait for around 5 or more mins and see node status changes to NotReady 6. check kubelet logs: cat /tmp/kubelet.log and notice errors regarding 1-fc22 or 1-el7 not an integer. Actual results: docker version shows Version as 1.9.1-fc22/el7 Expected results: docker version shows Version as 1.9.1 Additional info:
This is now beleived to be a kubernetes bug. It should be able to handle the version.
I see there is a temporary fix in docker. Not blocker for 7.2.2.
Switching back to docker component as this bug is already attached in erratum.
In kubernetes-1.2.0-0.9.alpha1.gitb57e8bd.el7.x86_64 docker-1.9.1-23.el7.x86_64 The docker version is shown correct, move to vierfied kubelet: I0318 03:00:26.036165 29134 manager.go:169] Version: {KernelVersion:3.10.0-294.el7.x86_64 ContainerOsVersion:Red Hat Enterprise Linux DockerVersion:1.9.1 CadvisorVersion: CadvisorRevision:
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0536.html