Bug 1427691
| Summary: | Build fails with Driver devicemapper failed to remove root filesystem - Device Busy | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Ben Parees <bparees> |
| Component: | Build | Assignee: | Ben Parees <bparees> |
| Status: | CLOSED ERRATA | QA Contact: | Vikas Laad <vlaad> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 3.4.1 | CC: | amurdaca, aos-bugs, aship, bmchugh, bparees, dwalsh, erich, g.condello, haowang, imcleod, jhonce, jkaur, jokerman, lsm5, mifiedle, mmccomas, mpatel, penli, tatanaka, tdawson, twaugh, vgoyal, vlaad, wabouham, wmeng, wzheng, xtian |
| Target Milestone: | --- | ||
| Target Release: | 3.4.z | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: Source to image builds expect image commits to take no longer than 2 minutes.
Consequence: Commits which take longer than 2 minutes result in a timeout and a failed build.
Fix: The timeout will be removed so image commits can take indeterminate lengths of time.
Result: Commits which take an excessive amount of time will not result in a failed build.
|
Story Points: | --- |
| Clone Of: | 1391665 | Environment: | |
| Last Closed: | 2017-03-15 20:04:02 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1391665 | ||
| Bug Blocks: | |||
|
Comment 2
Ben Parees
2017-03-01 16:42:29 UTC
correct backport to 3.4.x pr: https://github.com/openshift/ose/pull/645 This has been merged into ocp and is in OCP v3.4.1.10 or newer. I am verifying the fix, here is what I am seeing when running out of iops root@ip-172-31-54-81: ~ # oc logs -n proj5 cakephp-mysql-example-40-build Cloning "https://github.com/redhat-performance/cakephp-ex.git" ... Commit: 0014ddebb91bc7dff3a1dabfbd7b51da762a6677 (made changes to enable database example) Author: ofthecure <robdean.smith> Date: Mon Apr 25 14:33:06 2016 -0400 DEPRECATED: Use .s2i/bin instead of .sti/bin error: build error: Cannot connect to the Docker daemon. Is the docker daemon running on this host? Other error is pod stuck in RunContainerError and description of pod has 12m 5m 4 {kubelet ip-172-31-31-237.us-west-2.compute.internal} spec.containers{sti-build} Warning Failed Failed to create docker container "sti-build" of pod "cakephp-mysql-example-40-build_proj4(16192de4-0439-11e7-8ba8-024b76e8e62f)" with error: operation timeout: context deadline exceeded 12m 5m 4 {kubelet ip-172-31-31-237.us-west-2.compute.internal} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "sti-build" with RunContainerError: "runContainer: operation timeout: context deadline exceeded" I think these are expected ? please confirm. yeah i'd say that's not surprising if you're out of iops. But if you want to actually recreate the bug at hand, you need to run out of iops during the commit, not before. The easiest way to make that happen might be to write an assemble script that does a sleep at the end. Once the assemble script hits the sleep, then trigger whatever you are doing to exhaust/limit the iops. Completed testing on 3.4.1.10, did not see this issue. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0512 |