Description of problem: While trying to deploy an application from a template if the ImageStream object appears before the BuildConfig then the builds fails with an error like this: Pushing image 172.30.220.103:5000/bfqc8526/workshop2:latest ... error: build error: Failed to push image: unauthorized: authentication required However if the order is swapped and the BuildConfig object appears first the build finishes successfully. Also when the build fails, if a new build is started through the CLI with oc start-build <bc> then it finishes successfully and through the web console further rebuilds also work, however if the new build is started from the web interface after it has failed previously from the CLI or web, then it keeps failing. Version-Release number of selected component (if applicable): 3.4 How reproducible: always Steps to Reproduce: 1. Import a template where BuildConfig appears before the ImageStream object 2. Deploy an application using this template Actual results: Build error while pushing the resulting image: error: build error: Failed to push image: unauthorized: authentication required Expected results: Build finished successfully. Additional info:
mostly this sounds like what is described here: https://github.com/openshift/origin/issues/4518#issuecomment-140460107 however in that scenario, subsequent build attempts should succeed. I would like to see level 5 build logs from the second failing build.
also i'm confused by this summary: "Also when the build fails, if a new build is started through the CLI with oc start-build <bc> then it finishes successfully and through the web console further rebuilds also work, however if the new build is started from the web interface after it has failed previously from the CLI or web, then it keeps failing." 1) the build fails (when the buildconfig precedes the imagestream in the template) 2) oc start-build or starting a build via the web console results in a successful build completion when does "after it has failed previously from the CLI or web" come into play? I think you might be saying: 1) the initial build fails 2) starting a build via the web console - build fails 3) starting a build via CLI or web will now always fail vs 1) the initial build fails 2) start a build via CLI - build succeeds 3) starting a build via CLI or web will now always succeed Is that accurate? If so it is extremely strange since the codepaths for starting builds should be identical for the CLI and webconsole. Again, full level 5 logs for the failing builds are needed.
After some discussion on IRC this is the known issue i linked, the confusion was due to using the "Rebuild" button which clones the existing broken build (it's missing the pushsecret) so it continues to fail. Starting a new build succeeds as expected. Copying the most relevant description of the problem from the github issue: during the generation of the build object, we try to resolve the output IST so we can decide what secret we need to use with it. the resolution fails so we don't setup any push secret, but that does not (and should not) fail the creation of the build object. So a build object gets created with no pushsecret. Then the build controller sees it and tries to process the build object. part of processing the build object requires resolving the output IST (so we know where to push). Since we can't do this (no IST yet), the build hangs in a retry state until the IST gets created. Finally the IST gets created and the build proceeds, but the push secret was never set up. The right fix for this is to resolve the output push location at the time we create the build (which means hanging the build object creation until the output destination can be resolved), but i think there are some flow challenges with doing that. Another option would be to update the push secret when we resolve the reference in the build controller, instead of when creating the build in the buildconfig controller. either way it's a real bug, you need to create your IST before launching the build if you don't want to hit it, or you need to run a second build after the IST is created.
Commit pushed to master at https://github.com/openshift/origin https://github.com/openshift/origin/commit/10f3a0181fefeaea3cdf34137dea5dd558fce1cf resolve image push secret in controller, not instantiate https://bugzilla.redhat.com/show_bug.cgi?id=1443163 bug 1443163
https://github.com/openshift/origin/pull/15102
Hello, just checking, would this fix come in 3.6 or 3.5 ? The target is marked as 3.7. Thank you
Neither, it's coming in 3.7 too risky a change to put in 3.6 this late and ot critical enough of an issue.
verified in openshift v3.7.0-0.143.3 kubernetes v1.7.0+80709908fd etcd 3.2.1 steps:# 1. Extract buildconfig and imagestream oc new-app https://github.com/wewang58/helloworld-v3.git -o json > helloworld-v3.json # cat helloworld-v3.json | jq .items[0] > imageStream.json # cat helloworld-v3.json | jq .items[1] > buildConfig.json 2. cretae build(buildconfig) before imagestream. # oc create -f buildConfig.json # oc create -f imageStream.json 3. Check the image pushed successfully [root@dhcp-128-91 .ssh]# oc get builds NAME TYPE FROM STATUS STARTED DURATION helloworld-v3-1 Source Git@f4be318 Complete Less than a second ago 7s [root@dhcp-128-91 .ssh]# oc build-logs helloworld-v3-1 Command "build-logs" is deprecated, use "oc logs build/<build-name>" instead. Cloning "https://github.com/wewang58/helloworld-v3.git" ... Commit: f4be3180053297f74bdfdecb5aec27f7ac229276 (empty) Author: Kenjiro Nakayama <nakayamakenjiro> Date: Fri Sep 4 14:06:33 2015 +0900 ---> Installing application source... Pushing image docker-registry.default.svc:5000/wewang/helloworld-v3:latest ... Pushed 1/6 layers, 21% complete Pushed 2/6 layers, 50% complete Pushed 3/6 layers, 67% complete Pushed 4/6 layers, 83% complete Pushed 5/6 layers, 83% complete Pushed 6/6 layers, 100% complete Push successful
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:3188