Bug 1436395 - Some build failures do not show STARTED value
Summary: Some build failures do not show STARTED value
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Build
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.5.z
Assignee: Ben Parees
QA Contact: Vikas Laad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-27 20:31 UTC by Vikas Laad
Modified: 2017-04-26 05:36 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Builds that quickly progress from running to complete do not trigger the logic that sets the build start time. Consequence: Builds will be missing the build start time. Fix: If a build completes without having the start time set, set the start time equal to the completion time. Result: Builds will always have a start time even if they complete extremely quickly.
Clone Of:
Environment:
Last Closed: 2017-04-26 05:36:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
desc build complete (1.92 KB, text/plain)
2017-04-03 20:20 UTC, Vikas Laad
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:1129 0 normal SHIPPED_LIVE OpenShift Container Platform 3.5, 3.4, 3.3, and 3.2 bug fix update 2017-04-26 09:35:35 UTC

Description Vikas Laad 2017-03-27 20:31:31 UTC
Description of problem:
Some of the build failures do not show started time in the list when running 

oc get builds

proj9       cakephp-mysql-example-28   Source    Git@0014dde   Failed 
                                                   
root@ip-172-31-25-76: ~/svt/openshift_performance/ci/scripts # oc logs -n proj9       cakephp-mysql-example-28-build
Cloning "https://github.com/redhat-performance/cakephp-ex.git" ...
        Commit: 0014ddebb91bc7dff3a1dabfbd7b51da762a6677 (made changes to enable database example)
        Author: ofthecure <robdean.smith>
        Date:   Mon Apr 25 14:33:06 2016 -0400
DEPRECATED: Use .s2i/bin instead of .sti/bin
error: build error: Error response from daemon: {"message":"devmapper: Thin Pool has 5890 free data blocks which is less than minimum required 6083 free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior"}


Version-Release number of selected component (if applicable):
openshift v3.5.3
kubernetes v1.5.2+43a9be4
etcd 3.1.0

How reproducible:


Steps to Reproduce:
1. Run lots of builds and make sure worker nodes run out of disk space
2. see output of oc get builds 
3. some of the build failures missing started time

Actual results:
Missing STARTED column value

Expected results:
Should show STARTED value

Additional info:

Comment 2 Cesar Wong 2017-03-28 15:20:52 UTC
The build fails too quickly for the BuildPodController to process the Running event. It goes from Pending to Failed and we only set the start time on a Running event. We need to update the start time when we set the build to Complete or Failed if no start time has been set.

Comment 3 Vikas Laad 2017-04-03 20:20:32 UTC
Created attachment 1268485 [details]
desc build complete

Some of the Complete builds are also missing STARTED value, attaching logs for that. Please let me know if I need to create a separate bug for that.

Comment 4 Cesar Wong 2017-04-03 20:40:30 UTC
It's the same issue for both

Comment 5 Ben Parees 2017-04-07 12:10:02 UTC
Cesar it looks like maybe you fixed this as part of your pod handler work?
https://github.com/openshift/origin/blob/master/pkg/build/controller/common/util.go#L18

Comment 6 Ben Parees 2017-04-07 12:10:19 UTC
nevermind, I totally misread that code.

Comment 7 Ben Parees 2017-04-07 14:26:30 UTC
PR https://github.com/openshift/origin/pull/13670

Comment 9 Troy Dawson 2017-04-18 20:06:42 UTC
This has been merged into ocp and is in OCP v3.5.5.7 or newer.

Comment 12 Vikas Laad 2017-04-21 13:22:26 UTC
Completed multiple rounds concurrent builds, checked STARTED for Failed and Completed builds. No builds are missing STARTED value.

openshift v3.5.5.7
kubernetes v1.5.2+43a9be4
etcd 3.1.0

Comment 14 errata-xmlrpc 2017-04-26 05:36:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1129


Note You need to log in before you can comment on or make changes to this bug.