Bug 1309801

Summary: Containerized node doesn't mount /var/log so log aggregation components fail
Product: OpenShift Container Platform Reporter: Scott Dodson <sdodson>
Component: InstallerAssignee: Scott Dodson <sdodson>
Status: CLOSED ERRATA QA Contact: chunchen <chunchen>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.1.0CC: aos-bugs, gpei, jokerman, lmeyer, mmccomas, mwringe, wsun
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-12 16:38:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1310498    

Description Scott Dodson 2016-02-18 18:07:59 UTC
Description of problem:
The current containerized installation methods do not mount /var/log into the containerized node instance which prevents the log aggregation from working properly.

Version-Release number of selected component (if applicable):
Current

How reproducible:


Steps to Reproduce:
1. Deploy containerized OSE
2. Deploy aggregated logging components, observe failure
3.

Actual results:
Fluentd doesn't work because it needs read-write access to host level /var/log

Expected results:
Aggregated logging works properly

Additional info:

Comment 2 chunchen 2016-02-19 09:33:22 UTC
Will verify it when the bug [1] is fixed.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1309192#c9

Comment 3 Scott Dodson 2016-02-19 15:43:16 UTC
(In reply to chunchen from comment #2)
> Will verify it when the bug [1] is fixed.
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1309192#c9

Ok, I marked that one as a blocker even though I'm not really sure it's truly a blocker. I suspect this bug could be tested by running 3.1 OSE and 3.1.1 logging components.

Comment 4 Matt Wringe 2016-02-19 15:50:18 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=1309192 should have nothing to do with this issue. 1309192 deals with the Metrics components and this issue deals with the logging components.

These are two separate projects and need to have separate BZ listed for them. I am removing this issue as a dependency on 1309192

Comment 5 Luke Meyer 2016-02-19 16:12:29 UTC
(In reply to Scott Dodson from comment #0)

> Fluentd doesn't work because it needs read-write access to host level
> /var/log

Just to set the record straight, it's the node container that needs to write things to host /var/log because that's where fluentd is looking for them. Fluentd doesn't see that any docker logs exist if node isn't writing symlinks in /var/log.

Comment 6 chunchen 2016-02-24 05:32:35 UTC
It's fixed, checked with below latest logging images:

[root@openshift-154 ~]# docker images|grep logging| awk '{print $1"    "$3}' |awk -F'/' '{print $2"/"$3}'
openshift3/logging-elasticsearch    bd98af831d62
openshift3/logging-fluentd    0acd0a966bf4
openshift3/logging-kibana    32f915bc69ad
openshift3/logging-deployment    383767c6b100
openshift3/logging-auth-proxy    948429aebce9

[root@openshift-154 ~]# ps -ef |grep openshift3 |grep node
root     22690     1  0 2 23 ?       00:00:35 /usr/bin/docker run --name atomic-openshift-node --rm --privileged --net=host --pid=host -v /:/rootfs:ro -e CONFIG_FILE=/etc/origin/node/node-config.yaml -e OPTIONS=--loglevel=5 -e HOST=/rootfs -e HOST_ETC=/host-etc -v /var/lib/origin:/var/lib/origin -v /etc/origin/node:/etc/origin/node -v /etc/localtime:/etc/localtime:ro -v /etc/machine-id:/etc/machine-id:ro -v /run:/run -v /sys:/sys:ro -v /usr/bin/docker:/usr/bin/docker:ro -v /var/lib/docker:/var/lib/docker -v /lib/modules:/lib/modules -v /etc/origin/openvswitch:/etc/openvswitch -v /etc/origin/sdn:/etc/openshift-sdn -v /etc/systemd/system:/host-etc/systemd/system -v /var/log:/var/log openshift3/node

Comment 8 errata-xmlrpc 2016-05-12 16:38:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1065