Bug 1332301 - OpenShift 3.2 registry is missing the GCS Backend...
Summary: OpenShift 3.2 registry is missing the GCS Backend...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry
Version: 3.2.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: ---
Assignee: Michal Minar
QA Contact: Gan Huang
URL:
Whiteboard:
Depends On:
Blocks: OSOPS_V3
TreeView+ depends on / blocked
 
Reported: 2016-05-02 19:50 UTC by Thomas Wiest
Modified: 2016-08-11 18:38 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
The integrated registry now supports GCS storage driver. Please see https://docs.openshift.org/latest/install_config/install/docker_registry.html#docker-registry-configuration-reference-storage for configuration details.
Clone Of:
Environment:
Last Closed: 2016-08-11 18:38:04 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1608 0 normal SHIPPED_LIVE Red Hat OpenShift Enterprise 3.2.1.13 bug fix and enhancement update 2016-08-11 22:37:53 UTC

Description Thomas Wiest 2016-05-02 19:50:51 UTC
Description of problem:
The OpenShift 3.2 registry is missing the Google Cloud Storage (GCS) backend.

I spoke with clayton and this was a simple oversight.


Version-Release number of selected component (if applicable):

The registry currently deployed in dev-preview-int is:

registry.qe.openshift.com/openshift3/ose-docker-registry         v3.2.0.16           9340a6bd7ef2


How reproducible:
Very, it's missing.

Steps to Reproduce:
1. Ask Clayton. He knows the details.


Actual results:
It's missing.

Expected results:
It should be there.

Comment 1 Paul Weil 2016-05-02 19:53:24 UTC
storage driver enabled here: https://github.com/openshift/origin/pull/8692

Comment 4 Michal Minar 2016-05-06 14:33:26 UTC
tag v3.2.0.41 and higher

Comment 7 Thomas Wiest 2016-05-19 14:19:10 UTC
More info:  https://github.com/openshift/origin/issues/8826

Comment 8 Clayton Coleman 2016-06-14 21:09:30 UTC
Fixed in https://github.com/openshift/origin/pull/9211

Comment 10 Michal Minar 2016-07-01 08:34:51 UTC
Fixed in master by https://github.com/openshift/origin/pull/9211 and backported to 3.2.1 with https://github.com/openshift/ose/pull/271/commits.

Comment 11 Johnny Liu 2016-07-04 07:40:03 UTC
Re-test this bug with atomic-openshift-3.2.1.4-1.git.0.9fe156c.el7.x86_64 and ose-docker-registry:v3.2.1.4 (df8d5ff42e01), failed with the same error as comment 9.

# oc logs docker-registry-5-x8sp0
time="2016-07-04T03:34:27.769208666-04:00" level=info msg="version=v2.1.0+unknown" 
panic: StorageDriver not registered: gcs

goroutine 1 [running]:
github.com/docker/distribution/registry/handlers.NewApp(0x7f6037fb78d8, 0x2101a40, 0xc2082e0280, 0x7f6037fb78d8)
	/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/docker/distribution/registry/handlers/app.go:105 +0x3e1
github.com/openshift/origin/pkg/cmd/dockerregistry.Execute(0x7f6037faa798, 0xc20802c6e8)
	/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/dockerregistry/dockerregistry.go:60 +0x4d0
main.main()
	/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/cmd/dockerregistry/main.go:51 +0x3ea

goroutine 5 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x2102460)
	/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/golang/glog/glog.go:879 +0x78
created by github.com/golang/glog.init·1
	/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/golang/glog/glog.go:410 +0x2a7

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/lib/golang/src/runtime/asm_amd64.s:2232 +0x1

goroutine 13 [syscall]:
os/signal.loop()
	/usr/lib/golang/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/lib/golang/src/os/signal/signal_unix.go:27 +0x35

Comment 12 Clayton Coleman 2016-07-22 19:39:54 UTC
The dockerregistry binary that is put into the image needs to be built with the Go build tag.  build-cross.sh expects 

readonly OS_IMAGE_COMPILE_GOFLAGS="-tags include_gcs"

If that is not being used when the necessary binaries are brew built, GCS will not be enabled in the final image.

Comment 15 Kenny Woodson 2016-07-27 22:03:36 UTC
I tested this build today and was able to get passed the go panic error.

I attempted to test the registry by doing a simple build and push and received the following error:

I0727 17:52:32.343262       1 sti.go:334] Successfully built kwoodsontest/ruby-hello-world-4:ffd94ac6
I0727 17:52:32.368288       1 cleanup.go:23] Removing temporary directory /tmp/s2i-build129614529
I0727 17:52:32.368327       1 fs.go:156] Removing directory '/tmp/s2i-build129614529'
I0727 17:52:32.375616       1 sti.go:268] Using provided push secret for pushing 172.30.56.234:5000/kwoodsontest/ruby-hello-world:latest image
I0727 17:52:32.375635       1 sti.go:272] Pushing 172.30.56.234:5000/kwoodsontest/ruby-hello-world:latest image ...
I0727 17:52:38.139585       1 sti.go:277] Registry server Address: 
I0727 17:52:38.139747       1 sti.go:278] Registry server User Name: serviceaccount
I0727 17:52:38.139758       1 sti.go:279] Registry server Email: serviceaccount
I0727 17:52:38.139765       1 sti.go:284] Registry server Password: <<non-empty>>
F0727 17:52:38.139774       1 builder.go:204] Error: build error: Failed to push image. Response from registry is: Received unexpected HTTP status: 500 Internal Server Error




Here is my registry-config:

version: 0.1
log:
  level: debug
http:
  addr: :5000
storage:
  cache:
    layerinfo: inmemory
  delete:
    enabled: true
  gcs:
    bucket: <redacted>
    keyfile: /etc/registry_creds/creds.json
    rootdirectory: /registry
auth:
  openshift:
    realm: openshift
middleware:
  repository:
    - name: openshift
      options:
        pullthrough: true



Any info regarding this would be appreciated.

Comment 16 Michal Minar 2016-07-28 06:03:09 UTC
Kenny, could you please provide registry log from the time of failed push?

Comment 18 Gan Huang 2016-07-28 07:42:36 UTC
Test aganist 
# openshift version
openshift v3.2.1.12
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

Met the same error as comment 15.

Retrying in 1 seconds
Retrying in 1 seconds
Retrying in 1 seconds
Retrying in 1 seconds
I0728 03:32:10.942189       1 sti.go:277] Registry server Address: 
I0728 03:32:10.942209       1 sti.go:278] Registry server User Name: serviceaccount
I0728 03:32:10.942217       1 sti.go:279] Registry server Email: serviceaccount
I0728 03:32:10.942224       1 sti.go:284] Registry server Password: <<non-empty>>
F0728 03:32:10.942273       1 builder.go:204] Error: build error: Failed to push image. Response from registry is: Received unexpected HTTP status: 500 Internal Server Error

Comment 19 Michal Minar 2016-07-28 08:32:39 UTC
Copy&pasting error from Gan's registry log:

    time="2016-07-28T03:32:11.107829057-04:00" level=error msg="response completed with error" err.code=UNKNOWN err.detail="gcs: googleapi: Error 403: Forbidden, forbidden"

Related operation is a simple stat on the uploaded data blob:

    time="2016-07-28T03:32:11.10764685-04:00" level=debug msg="gcs.Stat(\"/docker/registry/v2/repositories/install-test/cakephp-example/_uploads/274054d6-f306-44eb-b9a4-834e27e65ab9/data\")"

It seems that the registry is either badly configured or the user is not authorized to store anything there?

Is it owned by someone else? Does the owner of the bucket need to set read-write permissions on /docker/registry to the pusher?

Comment 20 Kenny Woodson 2016-07-28 13:40:51 UTC
Thanks Michal for you reply.  I pinged Michal on IRC and we were able to get this working.  As Michal stated in comment #19 this was caused by a permission issue.

After running docker logs <containerid> I was able to find these logs:
time="2016-07-28T09:04:27.722237404-04:00" level=error msg="error canceling upload after error: gcs: googleapi: Error 403: Forbidden, forbidden" go.version=go1.4.2 http.request.host="172.30.56.234:5000"

In the gce console under the section IAM, select your service account/user.  Select the drop down on the right and grant the sa/user the Storage Object Admin.  (Don't forget to click save!)

From the docs https://cloud.google.com/iam/docs/managing-policies
- description: Full control of GCS objects.
  name: roles/storage.objectAdmin
  title: Storage Object Admin

I0728 09:36:08.287538       1 sti.go:268] Using provided push secret for pushing 172.30.56.234:5000/kwoodsontest/ruby-hello-world:latest image
I0728 09:36:08.287552       1 sti.go:272] Pushing 172.30.56.234:5000/kwoodsontest/ruby-hello-world:latest image ...
I0728 09:36:17.134119       1 sti.go:288] Successfully pushed 172.30.56.234:5000/kwoodsontest/ruby-hello-world:latest

The only take away is that it would be nice if the build logs included the errors from the docker-registry so that we could see the issue but the answer here is sufficient after minimal digging.  Internal server error is somewhat difficult to decipher.

Thanks again Michal, Gan, Johnny, and other qe team members.

Comment 21 Gan Huang 2016-07-29 01:31:12 UTC
I also successed to push to the registry after using another service account which has been granted to access the storage.

Thanks Michal and Kenny. Move to verified.

Comment 23 errata-xmlrpc 2016-08-11 18:38:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1608


Note You need to log in before you can comment on or make changes to this bug.