Bug 1463397
| Summary: | "docker stats" command reports inaccurate CPU usage | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Steven Walter <stwalter> |
| Component: | Containers | Assignee: | Jhon Honce <jhonce> |
| Status: | CLOSED WONTFIX | QA Contact: | DeShuai Ma <dma> |
| Severity: | low | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.2.1 | CC: | aos-bugs, jokerman, mmccomas |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-06-27 16:55:00 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Also of note, the machine only has 2 CPUs so customer is concerned over 300%+ (In reply to Steven Walter from comment #2) > Also of note, the machine only has 2 CPUs so customer is concerned over 300%+ Ignore this -- misread; only 2 CPUs are allocated in limits, but machine has more than 2 CPU |
Description of problem: `docker stats` command indicates CPU usage by a container consistently above 300% (in a multi-core system). However the `top` output indicates a sub-200% CPU usage, which is consistent with resource limits placed on the pod. Version-Release number of selected component (if applicable): 3.2.1 How reproducible: Confirmed Steps to Reproduce: 1. run docker stats in one terminal 2. run top in another 3. Observe the two Actual results: CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O cadaaad35832 314.27% 4.969 GB / 6.442 GB 77.14% 909.4 MB / 646 MB 49.76 MB / 0 B TOP for container: top - 15:53:01 up 111 days, 16:22, 0 users, load average: 10.51, 22.90, 28.23 Tasks: 12 total, 1 running, 11 sleeping, 0 stopped, 0 zombie %Cpu(s): 35.1 us, 39.6 sy, 0.0 ni, 22.0 id, 1.6 wa, 0.0 hi, 1.7 si, 0.0 st KiB Mem : 65802252 total, 22708220 free, 28383288 used, 14710744 buff/cache KiB Swap: 0 total, 0 free, 0 used. 36450864 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 1010010+ 20 0 8151860 3.587g 8996 S 169.0 5.7 2459:13 mysqld 15415 1010010+ 20 0 3348008 24024 2828 S 31.3 0.0 10:02.22 sysbench 19971 1010010+ 20 0 53988 2008 1448 R 0.3 0.0 0:00.01 top 24 1010010+ 20 0 4356 340 268 S 0.0 0.0 3:53.19 tail 970 1010010+ 20 0 11776 1720 1360 S 0.0 0.0 0:00.32 sh 15416 1010010+ 20 0 4316 360 284 S 0.0 0.0 0:00.00 tee 19938 1010010+ 20 0 11772 1712 1368 S 0.0 0.0 0:00.06 bash 65597 1010010+ 20 0 11772 1800 1444 S 0.0 0.0 0:00.10 sh 65876 1010010+ 20 0 4356 356 284 S 0.0 0.0 0:06.39 tail 120360 1010010+ 20 0 11776 1812 1448 S 0.0 0.0 0:00.13 sh 121383 1010010+ 20 0 13532 1792 1356 S 0.0 0.0 0:00.01 runtest.sh 123515 1010010+ 20 0 4356 356 284 S 0.0 0.0 0:00.72 tail Expected results: Expect that the CPU would be more closely aligned between the two outputs Additional info: # docker version Client: Version: 1.10.3 API version: 1.22 Package version: docker-common-1.10.3-44.el7.x86_64 Go version: go1.4.2 Git commit: 7ffc8ee-unsupported Built: Fri Jun 17 15:27:21 2016 OS/Arch: linux/amd64 Server: Version: 1.10.3 API version: 1.22 Package version: docker-common-1.10.3-44.el7.x86_64 Go version: go1.4.2 Git commit: 7ffc8ee-unsupported Built: Fri Jun 17 15:27:21 2016 OS/Arch: linux/amd64 root@apsrp6465:/root # openshift version openshift v3.2.1.4-1-g1864c8f kubernetes v1.2.0-36-g4a3f9c5 etcd 2.2.5 # oc get dc mysqldb -n mysql-pxc -o yaml apiVersion: v1 kind: DeploymentConfig metadata: name: mysqldb . . . spec: containers: - env: - name: MYSQL_USER value: admin - name: MYSQL_PASSWORD value: admin - name: MYSQL_ROOT_PASSWORD value: admin - name: MYSQL_DATABASE value: sampledb - name: MYSQL_MEM_REQUEST value: 6Gi - name: MYSQL_CPU_REQUEST value: 1500m . . . resources: limits: cpu: "2" memory: 6Gi requests: cpu: 1500m memory: 6Gi . . .