Bug 1511852
| Summary: | openshift ex dockergc encounter panic | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | DeShuai Ma <dma> |
| Component: | Node | Assignee: | Seth Jennings <sjenning> |
| Status: | CLOSED ERRATA | QA Contact: | DeShuai Ma <dma> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.7.0 | CC: | aos-bugs, jokerman, mmccomas |
| Target Milestone: | --- | ||
| Target Release: | 3.9.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: |
undefined
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-03-28 14:12:08 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Verify on v3.9.0-0.16.0, the panic is fixed.
# oc ex dockergc --image-gc-high-threshold=30 --image-gc-low-threshold=20 --dry-run=true
docker build garbage collection daemon
MinimumGCAge: {1h0m0s}, ImageGCHighThresholdPercent: 30, ImageGCLowThresholdPercent: 20
gathering disk usage data
usage exceeds high threshold (12500MB > 9303MB), attempting to free 6298MB
0 exited containers found
skipping infra image docker.io/openshift/mysql-55-centos7:latest
skipping infra image docker.io/openshift/php-55-centos7:latest
skipping infra image docker.io/openshift/origin-metrics-hawkular-metrics:latest
skipping infra image docker.io/openshift/wildfly-101-centos7:latest
removing image sha256:ff426288ea903fcf8d91aca97460c613348f7a27195606b45f19ae91776ca23d (size: 207198847, age: 34h11m29.619059176s)
skipping infra image docker.io/openshift/jenkins-2-centos7:latest
skipping infra image docker.io/openshift/origin-deployer:latest
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/openshift/origin-deployer latest bcfd389d2585 About an hour ago 1.277 GB
docker.io/openshift/jenkins-2-centos7 latest 4dcb08e555ad 5 hours ago 2.211 GB
docker.io/openshift/wildfly-101-centos7 latest fad1e0510edd 8 weeks ago 943.6 MB
docker.io/openshift/origin-metrics-hawkular-metrics latest 67c1503b2ae2 9 weeks ago 914.4 MB
docker.io/openshift/php-55-centos7 latest 089abeb67362 12 months ago 539.3 MB
docker.io/openshift/mysql-55-centos7 latest 968db52211da 13 months ago 384.6 MB
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0489 |
Description of problem: When run "openshift ex dockergc" to cleanup image for docker met panic Version-Release number of selected component (if applicable): openshift v3.7.5 kubernetes v1.7.6+a08f5eeb62 etcd 3.2.8 How reproducible: Sometime Steps to Reproduce: 1. Pull some image on node [root@qe-pod-node-registry-router-1 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE install-test/nodejs-mongodb-example-1 92e4ef4b 62057d7c378b 7 hours ago 490.3 MB docker.io/openshift/origin-deployer latest 5a630b4ed6a4 10 hours ago 1.099 GB docker.io/openshift/jenkins-2-centos7 latest 086e26565757 36 hours ago 2.021 GB docker.io/openshift/origin-metrics-hawkular-metrics latest 67c1503b2ae2 8 days ago 914.4 MB docker.io/openshift/wildfly-101-centos7 latest b0948ecacc39 2 weeks ago 943.6 MB registry.access.redhat.com/rhscl/nodejs-6-rhel7 <none> 96a89d880a8b 3 weeks ago 490.1 MB docker.io/openshift/php-55-centos7 latest 089abeb67362 10 months ago 539.3 MB docker.io/openshift/mysql-55-centos7 latest 968db52211da 11 months ago 384.6 MB 2. Check docker space usage [root@qe-pod-node-registry-router-1 ~]# df -k /var/lib/docker/ Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/rhel-root 24414208 14275132 10139076 59% / 3. Cleanup docker image space [root@qe-pod-node-registry-router-1 ~]# openshift ex dockergc --image-gc-high-threshold=40 --image-gc-low-threshold=30 --dry-run=true docker build garbage collection daemon MinimumGCAge: {1h0m0s}, ImageGCHighThresholdPercent: 40, ImageGCLowThresholdPercent: 30 gathering disk usage data usage exceeds high threshold (13940MB > 9536MB), attempting to free 6787MB 0 exited containers found skipping infra image docker.io/openshift/mysql-55-centos7:latest removing image sha256:968db52211da96767aa8c73854f4457b7c03bf80134cf4d35c085de9cf41ae10 (size: 384616280, age: 7980h12m53.345264645s) skipping infra image docker.io/openshift/php-55-centos7:latest removing image sha256:089abeb673622e4e2e7f2b32201b4e9bef7a54ad34ba2e176a3796e4f5e5d589 (size: 539328243, age: 7773h37m39.729339492s) panic: runtime error: index out of range goroutine 1 [running]: github.com/openshift/origin/pkg/oc/experimental/dockergc.doGarbageCollection(0xf2e2840, 0xc4201e34a0, 0xc4201e33e0, 0xc42123b970, 0xc4204af3a0, 0xf, 0x0, 0x0) /builddir/build/BUILD/atomic-openshift-git-0.f46092d/_output/local/go/src/github.com/openshift/origin/pkg/oc/experimental/dockergc/dockergc.go:215 +0x11c6 github.com/openshift/origin/pkg/oc/experimental/dockergc.Run(0xc420aaaff0, 0xc42123b970, 0xc421216480, 0xc420f7d470, 0x0, 0x3, 0x0, 0x0) /builddir/build/BUILD/atomic-openshift-git-0.f46092d/_output/local/go/src/github.com/openshift/origin/pkg/oc/experimental/dockergc/dockergc.go:258 +0x35b github.com/openshift/origin/pkg/oc/experimental/dockergc.NewCmdDockerGCConfig.func1(0xc421216480, 0xc420f7d470, 0x0, 0x3) /builddir/build/BUILD/atomic-openshift-git-0.f46092d/_output/local/go/src/github.com/openshift/origin/pkg/oc/experimental/dockergc/dockergc.go:83 +0x5f github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc421216480, 0xc420f7d3e0, 0x3, 0x3, 0xc421216480, 0xc420f7d3e0) /builddir/build/BUILD/atomic-openshift-git-0.f46092d/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:603 +0x22b github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4203bbd40, 0xc42000e020, 0xc4203bbd40, 0x1) /builddir/build/BUILD/atomic-openshift-git-0.f46092d/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:689 +0x339 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4203bbd40, 0x9, 0xc4203bbd40) /builddir/build/BUILD/atomic-openshift-git-0.f46092d/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:648 +0x2b main.main() /builddir/build/BUILD/atomic-openshift-git-0.f46092d/_output/local/go/src/github.com/openshift/origin/cmd/openshift/openshift.go:36 +0x1ae Actual results: Expected results: 3. Should cleanup image space successfully Additional info: