Bug 1368426 - Docker stats not reporting metrics for containers that use another's network
Summary: Docker stats not reporting metrics for containers that use another's network
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: docker
Version: 7.2
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: rc
: ---
Assignee: Lokesh Mandvekar
QA Contact: atomic-bugs@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 1186913
TreeView+ depends on / blocked
 
Reported: 2016-08-19 11:10 UTC by Brendan Mchugh
Modified: 2020-05-14 15:16 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-22 22:00:03 UTC
Target Upstream Version:


Attachments (Terms of Use)
patch (3.03 KB, patch)
2016-08-19 11:10 UTC, Brendan Mchugh
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Github docker docker issues 21848 0 None None None 2016-08-19 11:12:07 UTC

Description Brendan Mchugh 2016-08-19 11:10:15 UTC
Created attachment 1192105 [details]
patch

Description of problem:

We have observed a regression in Docker 1.10.3 (used with OpenShift 3.2.1.x) wrt Docker 1.8.2 (with OpenShift 3.1): the `docker stats` command shows zero-values for metrics on containers that share the network of another container.  This is particularly visible in the context of OpenShift since all containers of a pods share the network namespace of the openshift3/ose-pod container.


This is a known bug, fixed in Docker 1.12.  See:

- Bug: https://github.com/docker/docker/issues/21848
- Fix: https://github.com/docker/docker/pull/21904


Version-Release number of selected component (if applicable):

All systems running up to date RHEL 7.2 / Docker 1.10.  This is limited to dev/test currently.


How reproducible:
The behaviour is repeatable.

Steps to Reproduce:

Test case taken from #21848:

```
[root@eric-rhel72 ~]# export PAUSE_ID=$(docker run -d -m 10M kubernetes/pause)
[root@eric-rhel72 ~]# CHILD_ID=$(docker run -d -m 100M --net container:$PAUSE_ID kubernetes/pause)
[root@eric-rhel72 ~]# docker stats $PAUSE_ID $CHILD_ID
CONTAINER                                                          CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O
2383fd90a8bca7788f32e16e72e4d868693880692fd37e62c5ed2426f43e601a   0.00%               2.367 MB / 10.49 MB   22.58%              648 B / 648 B       0 B / 0 B
7ede4312d597935e4a18e6078c20872e322cb4cb48447479662d41758906f8a5   0.00%               0 B / 0 B             0.00%               0 B / 0 B           0 B / 0 B
[root@eric-rhel72 ~]# rpm -qi docker
Name        : docker
Version     : 1.10.3
Release     : 46.el7.10
Architecture: x86_64
Install Date: Tue 16 Aug 2016 12:25:17 PM UTC
Group       : Unspecified
Size        : 46245001
License     : ASL 2.0
Signature   : RSA/SHA256, Fri 29 Jul 2016 07:26:38 PM UTC, Key ID 199e2f91fd431d51
Source RPM  : docker-1.10.3-46.el7.10.src.rpm
Build Date  : Fri 29 Jul 2016 05:46:27 PM UTC
Build Host  : x86-039.build.eng.bos.redhat.com
[...]
```

Additional info:

Fix:

Adapting and applying the patch applied to docker https://github.com/docker/docker/pull/21904, and building + installing the RPMs shows the patch resolves the issue:

```
[root@eric-rhel72 ~]# export PAUSE_ID=$(docker run -d -m 10M kubernetes/pause)
[root@eric-rhel72 ~]# CHILD_ID=$(docker run -d -m 100M --net container:$PAUSE_ID kubernetes/pause)
[root@eric-rhel72 ~]# docker stats $PAUSE_ID $CHILD_ID
CONTAINER                                                          CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O
1ddf815fb6b4f8418c1eca027b1e7a0f36c6b9e1f710642d70507d665a0aea37   0.00%               290.8 kB / 10.49 MB   2.77%               648 B / 648 B       0 B / 0 B
2d6425dabe73fb1724580ba74959591c4969c7735f19993389291f8ee23f650f   0.00%               290.8 kB / 104.9 MB   0.28%               648 B / 648 B       0 B / 0 B
[root@eric-rhel72 ~]# rpm -qi docker
Name        : docker
Version     : 1.10.3
Release     : 46.el7.10.patch
Architecture: x86_64
Install Date: Tue 16 Aug 2016 12:20:47 PM UTC
Group       : Unspecified
Size        : 46273796
License     : ASL 2.0
Signature   : (none)
Source RPM  : docker-1.10.3-46.el7.10.patch.src.rpm
Build Date  : Tue 16 Aug 2016 09:52:43 AM UTC
Build Host  : eric-rhel72.novalocal
[...]
```

Comment 2 Daniel Walsh 2016-08-19 20:43:43 UTC
Antonio anything on this bugzilla?

Comment 3 Antonio Murdaca 2016-08-22 09:16:13 UTC
(In reply to Daniel Walsh from comment #2)
> Antonio anything on this bugzilla?

Backported https://github.com/docker/docker/pull/21904 to projectatomic/docker 1.10.3 branches (Fedora and RHEL7). I'll rebuild for Fedora 24. Assigning Lokesh for RHEL.

Comment 4 Fedora Update System 2016-08-22 13:29:19 UTC
docker-1.10.3-27.git128f146.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-7c08a6a936

Comment 5 Fedora Update System 2016-08-22 20:23:00 UTC
docker-1.10.3-27.git128f146.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-7c08a6a936

Comment 6 Fedora Update System 2016-08-24 10:59:38 UTC
docker-1.10.3-28.git78aa320.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-89e0043ff5

Comment 7 Fedora Update System 2016-08-25 10:31:55 UTC
docker-1.10.3-28.git78aa320.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-89e0043ff5

Comment 8 Fedora Update System 2016-08-29 09:25:15 UTC
docker-1.10.3-29.gitc7e7694.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-a70d2ade7d

Comment 9 Fedora Update System 2016-08-29 22:52:41 UTC
docker-1.10.3-29.gitc7e7694.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-a70d2ade7d

Comment 10 Fedora Update System 2016-09-02 09:08:42 UTC
docker-1.10.3-50.gita612434.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-adecb6c677

Comment 11 Fedora Update System 2016-09-03 03:23:17 UTC
docker-1.10.3-50.gita612434.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-adecb6c677

Comment 12 Fedora Update System 2016-09-09 16:54:13 UTC
docker-1.10.3-50.gita612434.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.

Comment 15 Derrick Ornelas 2017-11-22 22:00:03 UTC
Closing CURRENTRELEASE as this looks fixed to me:

# rpm -q docker
docker-1.12.6-61.git85d7426.el7.x86_64

# export PAUSE_ID=$(docker run -d -m 10M kubernetes/pause)
# CHILD_ID=$(docker run -d -m 100M --net container:$PAUSE_ID kubernetes/pause)
# docker stats $PAUSE_ID $CHILD_ID
CONTAINER                                                          CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
10ee1e79e462d835a1be349c4064d32c89a64de088d43d26981ac7c7b9172111   0.00%               288 KiB / 10 MiB      2.81%               1.296 kB / 648 B    0 B / 0 B           4
13381af194b9ec3d37f020cb7e6a974df608ebddd86b7738815164a24dfc022f   0.00%               2.254 MiB / 100 MiB   2.25%               1.296 kB / 648 B    0 B / 0 B           4


Note You need to log in before you can comment on or make changes to this bug.