Bug 1301199

Summary:	docker adds distro tag to docker version which causes kubelet to fail
Product:	Red Hat Enterprise Linux 7	Reporter:	Avesh Agarwal <avagarwa>
Component:	docker	Assignee:	Lokesh Mandvekar <lsm5>
Status:	CLOSED ERRATA	QA Contact:	atomic-bugs <atomic-bugs>
Severity:	high	Docs Contact:
Priority:	high
Version:	7.2	CC:	dwalsh, jchaloup, lsm5, lsu
Target Milestone:	rc	Keywords:	Extras
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	docker-1.9.1-13.el7	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1302028 (view as bug list)		Environment:
Last Closed:	2016-03-31 23:23:32 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1302028

Description Avesh Agarwal 2016-01-22 21:00:17 UTC

Description of problem:

Due to recent changes in kube upstream (https://github.com/kubernetes/kubernetes/pull/19675), the container run time status is being verified based on docker's Version instead of APIVersion as done before. Since docker on fedora or rhel appends distro string (1.9.1-fc22/el7) to its version, kube parser fails to parse as the last part 1-fc22 is not integer. It was not an issue before as I said because APIVersion was being used which does not have this issue. For example:

$ docker version
Client:
 Version:         1.9.1-fc22
 API version:     1.21
 Package version: docker-1.9.1-4.git64eb95e.fc22.x86_64
 Go version:      go1.5.3
 Git commit:      64eb95e/1.9.1
 Built:
 OS/Arch:         linux/amd64

Server:
 Version:         1.9.1-fc22
 API version:     1.21
 Package version: docker-1.9.1-4.git64eb95e.fc22.x86_64
 Go version:      go1.5.3
 Git commit:      64eb95e/1.9.1
 Built:
 OS/Arch:         linux/amd64

Here are some errors, I noticed on kube nodes:

E0122 10:24:18.435492   27245 manager.go:986] docker: failed to parse docker server version "1.9.1-fc22": Unable to parse version "1.9.1-fc22": "1-fc22" is not an integer
E0122 10:24:23.273679   27245 manager.go:986] docker: failed to parse docker server version "1.9.1-fc22": Unable to parse version "1.9.1-fc22": "1-fc22" is not an integer
E0122 10:24:23.273692   27245 kubelet.go:2579] Container runtime sanity check failed: docker: failed to parse docker server version "1.9.1-fc22": Unable to parse version "1.9.1-fc22": "1-fc22" is not an integer

Detailed logs: http://pastebin.test.redhat.com/342892

As container run time status fails, all nodes in kube cluster are going down from Ready to NotReady status (it takes almost 5m in my setup). 


Version-Release number of selected component (if applicable):
docker-1.9.1-el7

How reproducible:
always

Steps to Reproduce:
1. git clone kubernetes which has https://github.com/kubernetes/kubernetes/pull/19675
2. cd kubernetes
3. make clean all
4. ./hack/local-up-cluster.sh
5. kubectl get nodes (wait for around 5 or more mins and see node status changes to NotReady
6. check kubelet logs: cat /tmp/kubelet.log and notice errors regarding 1-fc22 or 1-el7 not an integer.

 

Actual results:
docker version shows Version as 1.9.1-fc22/el7

Expected results:
docker version shows Version as 1.9.1


Additional info:

Comment 3 Daniel Walsh 2016-01-26 14:40:06 UTC

This is now beleived to be a kubernetes bug.  It should be able to handle the version.

Comment 4 Jan Chaloupka 2016-01-26 14:56:01 UTC

I see there is a temporary fix in docker. Not blocker for 7.2.2.

Comment 5 Jan Chaloupka 2016-01-26 15:03:46 UTC

Switching back to docker component as this bug is already attached in erratum.

Comment 6 Luwen Su 2016-03-17 19:03:02 UTC

In 
kubernetes-1.2.0-0.9.alpha1.gitb57e8bd.el7.x86_64
docker-1.9.1-23.el7.x86_64

The docker version is shown correct, move to vierfied
kubelet: I0318 03:00:26.036165   29134 manager.go:169] Version: {KernelVersion:3.10.0-294.el7.x86_64 ContainerOsVersion:Red Hat Enterprise Linux DockerVersion:1.9.1 CadvisorVersion: CadvisorRevision:

Comment 8 errata-xmlrpc 2016-03-31 23:23:32 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0536.html