Bug 1190192

Summary: Kubelet service fails to start on Project Atomic host
Product: [Fedora] Fedora Reporter: Matt Micene <matt.micene>
Component: kubernetesAssignee: Jan Chaloupka <jchaloup>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 21CC: eparis, golang-updates, jchaloup, lsm5, nhorman, vbatts, wkq5325
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: kubernetes-0.9.1-0.2.git3623a01.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-02-21 04:24:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Change docker.socket dependency to docker.service none

Description Matt Micene 2015-02-06 15:43:26 UTC 
Description of problem:

The latest Project Atomic relese of docker-io-1.4.1-8.fc21.x86_64 deprecated docker.socket as a systemd service unit in favor of docker.service.  

https://git.fedorahosted.org/cgit/fedora-atomic.git/commit/?h=f21

sudo atomic status
  TIMESTAMP (UTC)         ID             OSNAME
* 2015-02-06 00:17:20     f0e46eaf14     fedora-atomic

The kublet.service systemd unit file uses docker.socket as a service dependency.  This missing dependency causes the kubelet service to enter a failed state.

Version-Release number of selected component (if applicable):
  kubernetes-0.7.0-18.0.git52e165a.fc21.x86_64

How reproducible:
  Every time the service attempts to start.

Steps to Reproduce:
  1. Update atomic host to current tree (atomic upgrade) and reboot
  2. Start kubelet service (systemctl start kublet)
  3. Check kubelet status (systemctl status kublet, journalctl -u kubelet.service)

Actual results:

  systemd[1]: Unit kubelet.service entered failed state.
  systemd[1]: kubelet.service failed.
  systemd[1]: kubelet.service failed toschedule restart job: Unit docker.socket   failed to load: No such file or direc

Expected results:

  Kubelet service starts

Additional info:

Changing the [Unit] dependency and ordering to use docker.service allows the service to start as normally.

[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service cadvisor.service
Requires=docker.service

Current workaround for Project Atomic hosts: create a local version of the distributed service unit in /etc/systemd/system and make the change listed above.  This will allow the service to start as expected.
Comment 1 Matt Micene 2015-02-06 15:59:23 UTC 
Created attachment 988941 [details]
Change docker.socket dependency to docker.service

Replicates fix from workaround
Comment 2 Jan Chaloupka 2015-02-06 17:52:39 UTC 
https://lists.fedoraproject.org/pipermail/scm-commits/Week-of-Mon-20150202/1505610.html
Comment 3 Fedora Update System 2015-02-06 17:55:03 UTC 
kubernetes-0.9.1-0.2.git3623a01.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/kubernetes-0.9.1-0.2.git3623a01.fc21
Comment 4 Fedora Update System 2015-02-06 18:08:08 UTC 
kubernetes-0.9.1-0.2.git3623a01.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/kubernetes-0.9.1-0.2.git3623a01.fc20
Comment 5 Fedora Update System 2015-02-08 08:56:51 UTC 
Package kubernetes-0.9.1-0.2.git3623a01.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kubernetes-0.9.1-0.2.git3623a01.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-1847/kubernetes-0.9.1-0.2.git3623a01.fc21
then log in and leave karma (feedback).
Comment 6 wkq5325 2015-02-09 23:59:50 UTC 
Hi everyone, I still hit that when change from docker.socket to docker.service ( I used fedora 20)

as below:

[fedora@test-k8s-minion ~]$ cat /usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service cadvisor.service
Requires=docker.service

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet \
	    ${KUBE_LOGTOSTDERR} \
	    ${KUBE_LOG_LEVEL} \
	    ${KUBE_ETCD_SERVERS} \
	    ${KUBELET_ADDRESS} \
	    ${KUBELET_PORT} \
	    ${KUBELET_HOSTNAME} \
	    ${KUBE_ALLOW_PRIV} \
	    ${KUBELET_ARGS}
Restart=on-failure

[Install]
WantedBy=multi-user.target


[fedora@test-k8s-minion ~]$ sudo systemctl start kubelet
Failed to issue method call: Unit docker.socket failed to load: No such file or directory.


I really can not figure out why it complain again and again.

Could you help that ? Thanks
Comment 7 wkq5325 2015-02-10 06:14:30 UTC 
I have solved it, it need reboot system. or else. it would not work.

Seems fedora bug. I am not sure the root cause.
Comment 8 Jan Chaloupka 2015-02-10 09:28:40 UTC 
Hi,

if you change a service file manually you must reload systemd service deamon by [1]:

# systemctl daemon-reload

With yum update this happen automatically.

[1] http://www.freedesktop.org/software/systemd/man/systemctl.html
Comment 9 Fedora Update System 2015-02-21 04:24:12 UTC 
kubernetes-0.9.1-0.2.git3623a01.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 10 Fedora Update System 2015-02-21 04:24:42 UTC 
kubernetes-0.9.1-0.2.git3623a01.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.