Bug 1538413

Summary: [3.9]Failed docker builds leave temporary containers on node
Product: OpenShift Container Platform Reporter: Dongbo Yan <dyan>
Component: BuildAssignee: Ben Parees <bparees>
Status: CLOSED NOTABUG QA Contact: Wenjing Zheng <wzheng>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.9.0CC: aos-bugs, bparees, cewong, clichybi, dma, dyan, wzheng, xtian
Target Milestone: ---   
Target Release: 3.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The OpenShift Docker builder invokes the Docker build API without the ForceRmTemp flag Consequence: Containers from failed builds remain on the node where the build ran. These containers are not recognized by the kubelet for gc and are therefore accumulated until the node runs out of space. Fix: Modified the Docker build API call from the OpenShift Docker builder to force the removal of temporary containers. Result: Failed containers no longer remain on the node where a Docker build ran.
 Story Points: ---
Clone Of: 1512679 Environment:
Last Closed: 2018-02-01 07:33:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 1 Dongbo Yan 2018-01-25 03:35:43 UTC 
openshift v3.9.0-0.23.0
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.8

# docker version
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-68.gitec8512b.el7.x86_64
 Go version:      go1.8.3
 Git commit:      ec8512b/1.12.6
 Built:           Thu Nov 16 15:19:17 2017
 OS/Arch:         linux/amd64

# docker ps -a |grep dyan
62a1db7c10d5        da93d29be1c0                                                                                                                                        "openshift-docker-bui"   16 minutes ago      Exited (1) 15 minutes ago                         k8s_docker-build_failing-build-1-build_dyan_6d1bf16a-017e-11e8-8ba9-0e478a6815d2_0
18e32dbd60d5        da93d29be1c0                                                                                                                                        "openshift-manage-doc"   16 minutes ago      Exited (0) 16 minutes ago                         k8s_manage-dockerfile_failing-build-1-build_dyan_6d1bf16a-017e-11e8-8ba9-0e478a6815d2_0
755ab068c818        registry.reg-aws.openshift.com:443/openshift3/ose-pod:v3.9.0-0.23.0                                                                                 "/usr/bin/pod"           16 minutes ago      Exited (137) 15 minutes ago                       k8s_POD_failing-build-1-build_dyan_6d1bf16a-017e-11e8-8ba9-0e478a6815d2_0
Comment 2 Ben Parees 2018-01-25 03:54:40 UTC 
Those are k8s containers associated with k8s pods (hence the k8s prefix in the name), they aren't the containers this bug referred to.
Comment 3 Dongbo Yan 2018-02-01 07:33:27 UTC 
Yes, the temporary containers should be like this:

ec33fb345a4ee32eef1ae228159bfa256cd8649ab303405dd11ee67dbaadf828   sha256:e4206c982b14df427e0becc54cc18d97772d601dbf90fbe82e83522282e71231             "/bin/sh -c 'exit 1'"        49 minutes ago      Exited (1) 49 minutes ago                            desperate_williams

so this is not a bug