| Summary: | Build-Logs aren't provided during oc start-build | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Dmitry Zhukovski <dzhukous> |
| Component: | Build | Assignee: | Adam Kaplan <adam.kaplan> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | XiuJuan Wang <xiuwang> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.7.1 | CC: | aos-bugs, bleanhar, bparees, wsun |
| Target Milestone: | --- | ||
| Target Release: | 3.10.0 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: streaming of build logs failed due to a server-side timeout waiting for the build pod to start.
Consequence: oc start-build could hang if the --wait and --follow flags were set.
Fix:
1) Server-side timeout for a build pod to start was increased from 10 to 30 seconds.
2) If --follow flag is specified and the log streaming fails, return an error message to the user.
3) If --follow and --wait is specified, retry log streaming.
Result:
1) Log stream failures due to build pod wait timeouts are less likely to occur.
2) If --follow fails, user is presented with the message, "Failed to stream the build logs - to view the logs, run oc logs build/<build-name>"
3) If --follow and --wait flags are set, oc start-build will retry fetching the build logs until successful.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-08-27 18:25:25 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Dmitry Zhukovski
2018-05-08 13:38:01 UTC
Adam, I believe the current build log logic waits a bit for the build pod to be running before giving up and exiting. The --wait logic will wait basically forever for the build pod to complete. I do not recall why we don't also retry/wait forever for the build pod to be running so we can fetch the logs. It is worth revisiting. currently we wait 10s for the build to start running when we go to retrieve the logs: https://github.com/openshift/origin/blob/master/pkg/build/registry/buildlog/rest.go#L41 https://github.com/openshift/origin/blob/master/pkg/build/registry/buildlog/rest.go#L84-L91 I don't think we want to change the rest api behavior, but it would be reasonable to change the startbuild logic to retry fetching the logs: https://github.com/openshift/origin/blob/master/pkg/oc/cli/cmd/startbuild.go#L440-L449 (I also think the severity should be lowered. the simple workaround is to run: oc start-build --wait # will wait until the build is complete oc logs build/foo # retrieve the build logs once we know the build is complete Pull Request: https://github.com/openshift/origin/pull/19695 Commit pushed to master at https://github.com/openshift/origin https://github.com/openshift/origin/commit/7a1bf39a413ca5127b1ff09cff4dd27b883381f0 Improve resilience of oc start-build log streaming * Add retry when attempting to stream build logs. * Increase server-side build wait timeout to 30s. Fixes bug 1575990 Can't reproduce this bug with oc version oc v3.10.0-0.63.0 kubernetes v1.10.0+b81c8f8 features: Basic-Auth GSSAPI Kerberos SPNEGO Server https://**:8443 openshift v3.10.0-0.63.0 kubernetes v1.10.0+b81c8f8 # oc start-build dancer-mysql-example --follow --wait -n install-test --loglevel=4 build "dancer-mysql-example-5" started Cloning "https://github.com/openshift/dancer-ex.git" ... Commit: 950d3f52355c0c6989908419219d2c3cfdf7b8ff (Merge pull request #71 from bparees/httpcookies) Author: Ben Parees <bparees@users.noreply.github.com> Date: Fri Mar 9 12:56:51 2018 -0500 ---> Installing application source ... ---> Copying configuration files... |