Red Hat Bugzilla – Bug 1475219
Failed to install service catalog with "oc cluster up" due to cannot pull image
Last modified: 2017-08-16 15:51 EDT
Description of problem: Failed to install service catalog with "oc cluster up" due to cannot pull image Version-Release number of selected component (if applicable): oc v3.6.170 kubernetes v1.6.1+5115d708d7 How reproducible: Always Steps to Reproduce: 1.Create cluster with oc cluster up with service-catalog flag $ sudo oc cluster up --image=brew-pulp.../openshift3/ose --version=v3.6 --service-catalog 2.Check service catalog related service pod 3. Actual results: Error: failed to start the service catalog apiserver: timed out waiting for the condition $ oc describe pod Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 15m 15m 5 default-scheduler Warning FailedScheduling no nodes available to schedule pods 15m 15m 1 default-scheduler Normal Scheduled Successfully assigned controller-manager-1631689040-m47l8 to localhost 15m 13m 4 kubelet, localhost spec.containers{controller-manager} Normal Pulling pulling image "openshift/origin-service-catalog:v3.6" 15m 13m 4 kubelet, localhost spec.containers{controller-manager} Warning Failed Failed to pull image "openshift/origin-service-catalog:v3.6": rpc error: code = 2 desc = Error response from daemon: {"message":"manifest for openshift/origin-service-catalog:v3.6 not found"} 15m 5m 45 kubelet, localhost Warning FailedSync Error syncing pod 15m 10s 60 kubelet, localhost spec.containers{controller-manager} Normal BackOff Back-off pulling image "openshift/origin-service-catalog:v3.6" Expected results: Could launch cluster without error Additional info:
There is no "v3.6" version available, thus there are no images w/ that tag to pull. The valid version would be "v3.6.0-rc.0" right now.
Yes, I check openshift/origin-service-catalog image in docker hub, there are 2 tags: latest, v3.6.0-rc.0. But refer to https://github.com/openshift/ose/blob/enterprise-3.6/examples/service-catalog/service-catalog.yaml , 'oc cluster up' should pull "latest" tag image, not "v3.6". is it normal?
the tag in that template is replaced by the value provided in --version or the version of the oc client binary: https://github.com/openshift/origin/blob/master/pkg/bootstrap/docker/up.go#L1036-L1037 https://github.com/openshift/origin/blob/master/pkg/bootstrap/docker/openshift/servicecatalog.go#L86 so yes, i'd expect "tag" to end up being v3.6 the way you ran it.
OK, so if the openshift/origin-service-catalog image tag is not consistent with 'version' flag, how can we deal with this situation
Cluster up needs to be updated to support the new image.
origin PR https://github.com/openshift/origin/pull/15486 will need backporting to 3.6 release.
Commits pushed to master at https://github.com/openshift/origin https://github.com/openshift/origin/commit/7c8cb1581da6195ea5f1852896373b87f2b334ef respect --image flag value for service catalog image name bug 1475219 https://github.com/openshift/origin/commit/ea2c299c9509a1ee1222ddc5c66ffbe42ca4c997 Merge pull request #15486 from bparees/ose_sc Automatic merge from submit-queue (batch tested with PRs 15486, 15482) respect --image flag value for service catalog image name bug 1475219 https://bugzilla.redhat.com/show_bug.cgi?id=1475219
https://github.com/openshift/origin/pull/15502 was the backport.
Test with oc v3.6.172.0.1 kubernetes v1.6.1+5115d708d7 cannot launch cluster log-- I0731 13:39:17.508529 9672 run.go:186] Container created with id "4fb77cfc1154405fba1792cdba51a77a94f254e4329a655e3dcb0387eb0e8e22" I0731 13:39:17.508548 9672 run.go:251] Starting container "4fb77cfc1154405fba1792cdba51a77a94f254e4329a655e3dcb0387eb0e8e22" I0731 13:39:17.673923 9672 run.go:258] Waiting for container "4fb77cfc1154405fba1792cdba51a77a94f254e4329a655e3dcb0387eb0e8e22" I0731 13:39:17.988825 9672 run.go:264] Done waiting for container "4fb77cfc1154405fba1792cdba51a77a94f254e4329a655e3dcb0387eb0e8e22", rc=0 I0731 13:39:17.988845 9672 run.go:269] Reading logs from container "4fb77cfc1154405fba1792cdba51a77a94f254e4329a655e3dcb0387eb0e8e22" I0731 13:39:17.989259 9672 run.go:277] Done reading logs from container "4fb77cfc1154405fba1792cdba51a77a94f254e4329a655e3dcb0387eb0e8e22" I0731 13:39:17.989272 9672 run.go:279] Stdout: openshift v3.6.172.0.3 kubernetes v1.6.1+5115d708d7 etcd 3.2.1 I0731 13:39:17.989278 9672 run.go:280] Stderr: I0731 13:39:17.989283 9672 run.go:284] Container run successful I0731 13:39:17.989287 9672 run.go:244] Deleting container "4fb77cfc1154405fba1792cdba51a77a94f254e4329a655e3dcb0387eb0e8e22" FAIL Error: Enabling the service catalog requires a newer server level than 3.6.0-alpha.2+3c221d5, this server is version 0.0.0
I don't see any indication that https://github.com/openshift/origin/pull/15502 has been merged into the ose fork. Sam, any idea why not?
There were at least two issues: 1) the PR for this bug was not part of the build (it will be part of the next one). 2) OCP moved to 5 digits versions which caused the "version 0.0.0" message, that is being fixed here: https://github.com/openshift/ose/pull/818
(and fixed in origin here: https://github.com/openshift/origin/pull/15567)
Verified oc v3.6.173.0.1 kubernetes v1.6.1+5115d708d7 After I docker pull ose-service-catalog:v3.6 image first, could install sevice-catalog successfully, move bug to verified $ sudo oc get pod -n service-catalog NAME READY STATUS RESTARTS AGE apiserver-1543693763-qlmmr 2/2 Running 0 3m controller-manager-414755520-b3hlr 1/1 Running 1 3m