Bug 1232694 - Failed to start build after setting the push secret
Summary: Failed to start build after setting the push secret
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Build
Version: 3.x
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: ---
Assignee: Michal Fojtik
QA Contact: Wenjing Zheng
URL:
Whiteboard:
Depends On:
Blocks: 1245524
TreeView+ depends on / blocked
 
Reported: 2015-06-17 10:29 UTC by chunchen
Modified: 2019-08-15 04:43 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1245524 (view as bug list)
Environment:
Last Closed: 2015-09-08 18:00:55 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description chunchen 2015-06-17 10:29:50 UTC
Description of problem:
It's failed to start build after setting the push secret, met "Error creating: Pod "ruby-sample-build-1-build" is invalid: spec.volumes[2].name: duplicate value 'pushme'".

Version-Release number of selected component (if applicable):
devenv-fedora_1822 (ami-a74ebdcc)

How reproducible:
always

Steps to Reproduce:
1. Setup the OpenShift V3 environment according to: OpenShift 3 Application Lifecycle Sample  and login to server
   $ oc login --server=$master -u $user
   $ oc new-project $project-name

2. Get the '.dockercfg' file which contain account credential
1) register an account on DockerHub
2) log in to the registry to get the account credential
   $ docker login ## the credential file named ".dockercfg" will be generated under $HOME directory after login successfully

3. Create the secret
   $ oc secrets new pushme ~/.dockercfg

4. Update service account "builder" and secret "pushme" to make sure service account reference the secret
   $ oc get serviceaccount
   $ oc get secret
   $ oc edit serviceaccount/builder   # add secret "pushme" under "secrets" section as below
<---------snip-------->
secrets:
- name: pushme

   $ oc edit secret/pushme # add service account "builder" under "metadata" section as below
<---------snip-------->
metadata:
  annotations:
    kubernetes.io/service-account.name: builder
    kubernetes.io/service-account.uid: <builder-service-account-uid>

5. Update template json file to set the push secret like below
$ cat application-template-stibuild.json
     <----------snip--------->
       {
         "kind": "ImageStream",
         "apiVersion": "v1beta3",
         "metadata": {
           "name": "origin-ruby-sample"
         },
         "spec":{
         "dockerImageRepository": "chunyunchen/origin-ruby-sample"  ## my Dockerhub registry
       },
             "parameters": {
        "output": {
          "to": {
            "name": "origin-ruby-sample"
          },
          "pushSecret":{
             "name":"pushme"   ## the "pushSecret.name" is the same with above secret name
          }
        },
     <----------snip--------->
        "strategy": {
          "type": "Source"
        }
      },
     <----------snip--------->

6. Create the application with above template file
   $ oc process -f application-template-stibuild.json | oc create -f -

7. Check the build
   $ oc get build
   $ oc describe build/ruby-sample-build-1

Actual results:
the build's status is always "New" due to the pull secret is set automatically with same name of push secret, like below:

Name:			ruby-sample-build-1
Created:		9 minutes ago
Labels:			buildconfig=ruby-sample-build,name=ruby-sample-build,template=application-template-stibuild
Build Config:		ruby-sample-build
Status:			New
Duration:		waiting for 16m56s
Build Pod:		ruby-sample-build-1-build
Strategy:		Source
Image Reference:	DockerImage openshift/ruby-20-centos7:latest
Pull Secret Name:	pushme
Incremental Build:	yes
Source Type:		Git
URL:			git://github.com/openshift/ruby-hello-world.git
Output to:		origin-ruby-sample:latest
Output Spec:		<none>
Push Secret:		pushme
Events:
  FirstSeen				LastSeen			Count	From			SubobjectPath	Reason		Message
  Wed, 17 Jun 2015 17:24:29 +0800	Wed, 17 Jun 2015 17:33:50 +0800	6	{build-controller }			failedCreate	Error creating: Pod "ruby-sample-build-1-build" is invalid: spec.volumes[2].name: duplicate value 'pushme'

Expected results:
Should start build after setting the push secret successfully.

Additional info:

Comment 1 Michal Fojtik 2015-06-17 13:21:05 UTC
chunchen: The workaround for this is simple, just don't use the same secret name for both push/pull secrets. We have to make the volume names more unique to allow this.

Comment 2 chunchen 2015-06-18 08:12:57 UTC
@Michal Fojtik

On OSE, also meet "API error (403)" when pushing image to Dockerhub registry even have added option "--confirm-def-push=false" to docker instance like below:

root      14332      1  4 14:44 ?        00:00:15 /usr/bin/docker -d --selinux-enabled --confirm-def-push=false -b=lbr0 --mtu=1450 --insecure-registry 0.0.0.0/0
root      14821  14332  0 14:44 ?        00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 1936 -container-ip 10.1.0.6 -container-port 1936
root      14833  14332  0 14:44 ?        00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 10.1.0.6 -container-port 443
root      14845  14332  0 14:44 ?        00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 10.1.0.6 -container-port 80
1000000+  15309  14332  0 14:44 ?        00:00:00 /usr/bin/dockerregistry /config.yml

Some logs:
<------------snip---------->
I0618 04:00:02.442869       1 sti.go:246] Successfully built chunyunchen/ose-sti-0618
I0618 04:00:03.817883       1 cleanup.go:23] Removing temporary directory /tmp/sti066613609
I0618 04:00:03.817935       1 fs.go:99] Removing directory '/tmp/sti066613609'
I0618 04:00:03.821960       1 cfg.go:78] Found Docker authentication configuration in '/root/.dockercfg'
I0618 04:00:03.821988       1 cfg.go:46] PUSH_DOCKERCFG_PATH=/var/run/secrets/openshift.io/push/.dockercfg
I0618 04:00:03.822000       1 cfg.go:78] Found Docker authentication configuration in '/var/run/secrets/openshift.io/push/.dockercfg'
I0618 04:00:03.822217       1 cfg.go:64] Using chunyunchen user for Docker authentication
I0618 04:00:03.822235       1 sti.go:96] Using provided push secret for pushing chunyunchen/ose-sti-0618 image
I0618 04:00:03.822248       1 sti.go:99] Pushing chunyunchen/ose-sti-0618 image ...
E0618 04:00:03.824341       1 dockerutil.go:50] push for image chunyunchen/ose-sti-0618 failed, will retry in 10s ...
E0618 04:00:13.826163       1 dockerutil.go:50] push for image chunyunchen/ose-sti-0618 failed, will retry in 10s ...
F0618 04:00:24.818410       1 builder.go:70] Build error: Failed to push image: API error (403): Error: Status 403 trying to push repository chunyunchen/ose-sti-0618 to official registry: needs to be forced

Comment 3 chunchen 2015-06-18 10:02:38 UTC
Please ignore Comment #2, will try again when bug [1] is fixed, sorry for confused.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1224210

Comment 4 openshift-github-bot 2015-06-26 17:50:39 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/01afcf9008e319321e96a36a02f3e8334e11a154
Bug 1232694 - Make the secret volume for push/pull secrets unique

Comment 5 chunchen 2015-07-03 06:27:03 UTC
It's fixed, verified on devenv_fedora_1921, please refer to the below results:

[chunchen@F17-CCY cmd]$ oc describe build/ruby-sample-build-1
Name:			ruby-sample-build-1
Created:		Less than a second ago
Labels:			buildconfig=ruby-sample-build,name=ruby-sample-build,template=application-template-stibuild
Build Config:		ruby-sample-build
Status:			Running
Started:		2015-07-03 13:49:07 +0800 CST
Duration:		running for 24s
Build Pod:		ruby-sample-build-1-build
Strategy:		Source
Image Reference:	DockerImage openshift/ruby-20-centos7:latest
Pull Secret Name:	pushme
Source Type:		Git
URL:			https://github.com/openshift/ruby-hello-world.git
Output to:		origin-ruby-sample:latest
Output Spec:		<none>
Push Secret:		pushme
Events:
  FirstSeen				LastSeen			Count	From				SubobjectPath				Reason		Message
  Fri, 03 Jul 2015 13:49:04 +0800	Fri, 03 Jul 2015 13:49:04 +0800	1	{scheduler }								scheduled	Successfully assigned ruby-sample-build-1-build to ip-10-150-123-20
  Fri, 03 Jul 2015 13:49:04 +0800	Fri, 03 Jul 2015 13:49:04 +0800	1	{kubelet ip-10-150-123-20}	implicitly required container POD	pulled		Successfully pulled image "openshift/origin-pod:latest"
  Fri, 03 Jul 2015 13:49:05 +0800	Fri, 03 Jul 2015 13:49:05 +0800	1	{kubelet ip-10-150-123-20}	implicitly required container POD	created		Created with docker id 093ed00ab9b0f47b1863316820a22f49547b6420ad4ba928b4b09850ec02c251
  Fri, 03 Jul 2015 13:49:05 +0800	Fri, 03 Jul 2015 13:49:05 +0800	1	{kubelet ip-10-150-123-20}	implicitly required container POD	started		Started with docker id 093ed00ab9b0f47b1863316820a22f49547b6420ad4ba928b4b09850ec02c251
  Fri, 03 Jul 2015 13:49:06 +0800	Fri, 03 Jul 2015 13:49:06 +0800	1	{kubelet ip-10-150-123-20}	spec.containers{sti-build}		created		Created with docker id 7e8010c8a7feedc1742f0b4509b6f703a4d7f5490a111d6ae95b60aa1c1c781d
  Fri, 03 Jul 2015 13:49:06 +0800	Fri, 03 Jul 2015 13:49:06 +0800	1	{kubelet ip-10-150-123-20}	spec.containers{sti-build}		started		Started with docker id 7e8010c8a7feedc1742f0b4509b6f703a4d7f5490a111d6ae95b60aa1c1c781d

Comment 6 Josep 'Pep' Turro Mauri 2015-09-08 18:00:55 UTC
(In reply to openshift-github-bot from comment #4)
> Commit pushed to master at https://github.com/openshift/origin
> 
> https://github.com/openshift/origin/commit/
> 01afcf9008e319321e96a36a02f3e8334e11a154

This was part of v1.0.1, so closing this BZ.


Note You need to log in before you can comment on or make changes to this bug.