Description of problem: After creating docker container, the size of the container's log in /var/lib/docker/containers/HASH/HASH-json.log is increasing. The issue is that we don't have any way to remove / tidy / rotate / set max size of the log. We have to destroy container to remove the log. Steps to Reproduce: 1. 2. 3. Actual results: During running the container, the container's log(/var/lib/docker/containers/HASH/HASH-json.log) would be increasing. Expected results / Additional info:: We would like to have some command to manage the log. Docker upstream has already merged the patch to solve this issue : https://github.com/docker/docker/pull/11485 Additional: I know we may be able to manage it by log aaggregation outside of individual docker containers[1]. However, we should be able to manage it, without log aggregation. [1] https://docs.openshift.com/enterprise/3.0/admin_guide/aggregate_logging.html
Actually log aggregation doesn't yet and probably won't deal with rotating the docker logs under the json-file logging driver. With docker-1.7 the driver will provide rotation options that should solve this. For now, your only recourse is: 1) "rotate" them separately, either with logrotate or similar, or just plain truncating them. 2) change the docker logging driver to send them to syslog; unfortunately this means you'll get nothing from "docker logs" or "oc logs", and I don't think the logs are adequately identified in the syslog such that you could narrow them down to a specific container/pod, so it's not a very useful option. I'm not certain when docker 1.7 is coming out but I don't think it's far away.
> With docker-1.7 the driver will provide rotation options that should solve this. For now, your only recourse is: Current OpenShift(v3.0.1.0) includes docker-1.7.1. So you mean 1.8? [root@ose3-master ~]# docker version Client version: 1.7.1 Client API version: 1.19 Package Version (client): docker-1.7.1-108.el7.x86_64 The fix in the upstream also says that milestone is "1.8.0": https://github.com/docker/docker/pull/11485
(In reply to Kenjiro Nakayama from comment #3) > Current OpenShift(v3.0.1.0) includes docker-1.7.1. So you mean 1.8? I expected it was coming out soon, didn't realize it was already out :) > The fix in the upstream also says that milestone is "1.8.0": > https://github.com/docker/docker/pull/11485 It seems you're right; I saw there were new logging features coming out in 1.7 (e.g. the journald driver) and saw the source code docs for rotation and assumed they were together. But I just tested adding the log rotation parameters and they did nothing. So, I guess we can expect that with docker 1.8 or a backport; in the meantime, the two options I gave seem to be what's available. We will have to evaluate whether our log aggregation solution can help manage log rotation in the short term.
Would this log parameter be something that we will be able to define in the build or deployment config if we use docker 1.8?
Could you please check Ryan comment and let us know how the log parameter will be defined?
Thought I already responded to this, somehow must have failed. See the following log options which should be available with Docker 1.8: https://docs.docker.com/reference/logging/overview/#the-json-file-options It's worth noting that while we will default to json-file driver for the next release, there is a journald driver available which also works with "docker logs" and thus "oc logs". I haven't investigated to see if this actually works under kubernetes (there could be e.g. formatting differences that require attention) or how log aggregation would be different, but it's promising.
With Docker 1.8 now shipping, log rotation with the json-file driver is just some options in Docker. What remains is to make customers aware of this via docs, and preferably to enable them to be set automatically during an ansible install. It may be appropriate to spawn two RFEs from this and close this CURRENTRELEASE.
(In reply to Luke Meyer from comment #19) I opened these two bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1286895 https://bugzilla.redhat.com/show_bug.cgi?id=1286893 Please feel free to close this.