Bug 1322728

Summary: Failed to start node service during containerized installation
Product: OpenShift Container Platform Reporter: Gaoyun Pei <gpei>
Component: InstallerAssignee: Troy Dawson <tdawson>
Status: CLOSED ERRATA QA Contact: Ma xiaoqiang <xiama>
Severity: high Docs Contact:
Priority: high    
Version: 3.2.0CC: aos-bugs, bleanhar, jokerman, mmccomas, mwysocki, xtian
Target Milestone: ---Keywords: TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-12 16:39:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1318948    

Description Gaoyun Pei 2016-03-31 08:43:28 UTC
Description of problem:
When running a containerized installation, ansible failed at step:
TASK: [openshift_node | Start and enable node] ******************************** 
failed: [openshift-x.com] => {"failed": true}
msg: Job for atomic-openshift-node.service failed because the control process exited with error code. See "systemctl status atomic-openshift-node.service" and "journalctl -xe" for details.

Error "cp: cannot stat '/usr/lib/systemd/system/docker.service.d/docker-sdn-ovs.conf': No such file or directory" shown in the atomic-openshift-node log.


Checked the openshift3/node:v3.2.0.9 image, found there's no node&ovs related package in it.
[root@openshift-218 ~]# docker run -ti --entrypoint /bin/bash openshift3/node:v3.2.0.9
[root@6999136a794a origin]# rpm -qa|grep openshift
atomic-openshift-3.2.0.9-1.git.0.b99af7d.el7.x86_64
atomic-openshift-clients-3.2.0.9-1.git.0.b99af7d.el7.x86_64



Version-Release number of selected component (if applicable):
openshift-ansible-3.0.68-1.git.0.d80d45e.el7.noarch.rpm
openshift3/node:v3.2.0.9  (IMAGE ID  f76ffec37953)

How reproducible:
Always

Steps to Reproduce:
1.Start the atomic-openshift-node service on node host.
2.Monitor atomic-openshift-node log:
...
Mar 30 19:10:24 openshift-x.com systemd[1]: Starting atomic-openshift-node.service...
Mar 30 19:10:24 openshift-x.com atomic-openshift-node[15413]: Error response from daemon: no such id: atomic-openshift-node
Mar 30 19:10:24 openshift-x.com atomic-openshift-node[15413]: Error: failed to remove containers: [atomic-openshift-node]
Mar 30 19:10:26 openshift-x.com atomic-openshift-node[15419]: cp: cannot stat '/usr/lib/systemd/system/docker.service.d/docker-sdn-ovs.conf': No such file o
Mar 30 19:10:27 openshift-x.com systemd[1]: atomic-openshift-node.service: main process exited, code=exited, status=1/FAILURE
Mar 30 19:10:34 openshift-x.com atomic-openshift-node[15470]: Error response from daemon: no such id: atomic-openshift-node
Mar 30 19:10:34 openshift-x.com atomic-openshift-node[15470]: Error: failed to stop containers: [atomic-openshift-node]
Mar 30 19:10:34 openshift-x.com systemd[1]: atomic-openshift-node.service: control process exited, code=exited status=1
Mar 30 19:10:34 openshift-x.com systemd[1]: Failed to start atomic-openshift-node.service.


Actual results:


Expected results:
Should start node service successfully.

Additional info:

Comment 1 Brenton Leanhardt 2016-03-31 18:33:55 UTC
It definitely seems like there is a problem with the node image.  The last commit in dist-git for openshift-enterprise-node-docker removed atomic-openshift-sdn-ovs.  I'm pretty sure that was a mistake.

Comment 2 Troy Dawson 2016-03-31 18:53:51 UTC
Yep, that was a mistake.  The Dockerfiles in Origin had changed quite a bit and when I was syncing over to OSE I accidently removed the one package difference between the two.
A new image is currently building.  I'll update this bugzilla when it's ready.

Comment 3 Troy Dawson 2016-03-31 20:06:33 UTC
Updated image has been built and pushed to qe-registry.
Give it another try.

Comment 4 Gaoyun Pei 2016-04-01 01:53:29 UTC
Verify this bug with openshift3/ose:v3.2.0.9  1cee1ed5fca2

Install a a containerized env with openshift-ansible-3.0.68-1.git.0.d80d45e.el7.noarch, no error was made.

Comment 5 Marcel Wysocki 2016-05-11 09:45:05 UTC
Facing this as well on a containerized installation of 3.1.1.6
It happens for every master server.
I have to restart atomic-openshift-node manually and then re-run the ansible installer it it works fine.

Comment 7 errata-xmlrpc 2016-05-12 16:39:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1065