Bug 1795144

Summary:	Adding a default pull secret using the UI doesn't link it with the "default" service account [openshift-4.4]
Product:	OpenShift Container Platform	Reporter:	Sergio G. <sgarciam>
Component:	Management Console	Assignee:	Jakub Hadvig <jhadvig>
Status:	CLOSED ERRATA	QA Contact:	Yadan Pei <yapei>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.2.z	CC:	aos-bugs, bpeterse, jhadvig, jokerman, ltitov, xiaocwan, yapei
Target Milestone:	---
Target Release:	4.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1800320 (view as bug list)		Environment:
Last Closed:	2020-05-04 11:26:45 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1800320

Description Sergio G. 2020-01-27 09:22:08 UTC

Description of problem:
Adding a default pull secret for a namespace using the UI doesn't link it with the "default" service account for pulling images.

Version-Release number of selected component (if applicable):
4.2.2

How reproducible:
Always

Steps to Reproduce:
1. Create a namespace using the UI
2. Add to it a default pull secret using https://console-openshift-console.apps.<cluster>.<subdomain>/k8s/cluster/namespaces/<namespace>
3. Deploy an image which requires the pull secret

Actual results:
It fails

Expected results:
The pull secret should be used for pulling as the text in the UI writes clearly: "Specify default credentials to be used to authenticate and download containers within this namespace. These credentials will be the default unless a pod references a specific pull secret."

Comment 1 Sergio G. 2020-01-27 09:26:22 UTC

Version-Release in the description is wrong.

I've confirmed this bug in 4.2.14.

Comment 2 Jakub Hadvig 2020-01-28 12:53:08 UTC

Was trying to reproduce this issue on both 4.4. and 4.2 cluster but without any luck.
After I added the "Default Pull Secret"to the namespace I was able to find and also
deploy the private image without any issues.

Comment 3 Sergio G. 2020-01-28 13:45:20 UTC

Can you confirm if the secret is shown in "oc describe sa default" as used for pulling images?

It wasn't there for me until I linked it manually.

Comment 4 XiaochuanWang 2020-02-04 03:39:53 UTC

Tested on 4.4.0-0.nightly-2020-02-02-225006

After Project created, sa "default" is as below:
$ oc describe sa default
Name:                default
Namespace:           xiaocwan-s
Labels:              <none>
Annotations:         <none>
Image pull secrets:  default-dockercfg-4vsq7
Mountable secrets:   default-token-9lkgk
                     default-dockercfg-4vsq7
Tokens:              default-token-4jrd8
                     default-token-9lkgk
Events:              <none>

Image app using Deployment by "default-token-9lkgk" is failed.
Image app using DC by "default-dockercfg-4vsq7" could succeed.

As per my understanding, It needs a message such as the description "Expected results:" said.
Does image app must use the correct secret (which is "Image pull secrets") from SA "default"? Is that another bug?
Please notice the Target Release is still `--`

Comment 5 Jakub Hadvig 2020-02-04 15:32:16 UTC

Was able to reproduce the issue and about to sent fix.

Comment 6 Yadan Pei 2020-02-06 09:42:08 UTC

1. Add a default pull secret to namespace by following steps
Administration -> Namespaces -> yapei-1 -> click edit icon in Default Pull Secret -> add credentials used to pull private image from dockerhub, create

2. A secret 'yapei-dp' is created in namespace
$ oc get secret
NAME                            TYPE                                  DATA   AGE
yapei-dp                        kubernetes.io/dockerconfigjson        1      6m9s

3. Check sa/default, the added pull secret is not linked to sa/default
$ oc describe sa default 
Name:                default
Namespace:           yapei-1
Labels:              <none>
Annotations:         <none>
Image pull secrets:  default-dockercfg-wfs66
Mountable secrets:   default-token-p5kvs
                     default-dockercfg-wfs66
Tokens:              default-token-dpvbt
                     default-token-p5kvs
Events:              <none>

4. Deploy a DeploymentConfig from private image, the image can be searched in Deploy Image catalog and DC is created successfully, but pods can't be running
$ oc get pods
NAME                           READY   STATUS             RESTARTS   AGE
wordpress-1-deploy             1/1     Running            0          7m37s
wordpress-1-kh4m8              0/1     ImagePullBackOff   0          7m28s

$ oc describe pod wordpress-1-kh4m8
.....
Events:
  Type     Reason     Age                   From                                                Message
  ----     ------     ----                  ----                                                -------
  Normal   Scheduled  <unknown>             default-scheduler                                   Successfully assigned yapei-1/wordpress-1-kh4m8 to ip-10-0-134-11.us-east-2.compute.internal
  Warning  Failed     6m53s (x6 over 8m7s)  kubelet, ip-10-0-134-11.us-east-2.compute.internal  Error: ImagePullBackOff
  Normal   Pulling    6m40s (x4 over 8m8s)  kubelet, ip-10-0-134-11.us-east-2.compute.internal  Pulling image "yapei/wordpress@sha256:92f7a99d237efd588d8e4ddc11e938049ce38816084e35a31a42e8c3cd98a940"
  Warning  Failed     6m39s (x4 over 8m7s)  kubelet, ip-10-0-134-11.us-east-2.compute.internal  Failed to pull image "yapei/wordpress@sha256:92f7a99d237efd588d8e4ddc11e938049ce38816084e35a31a42e8c3cd98a940": rpc error: code = Unknown desc = Error reading manifest sha256:92f7a99d237efd588d8e4ddc11e938049ce38816084e35a31a42e8c3cd98a940 in docker.io/yapei/wordpress: errors:
denied: requested access to the resource is denied
unauthorized: authentication required
  Warning  Failed   6m39s (x4 over 8m7s)  kubelet, ip-10-0-134-11.us-east-2.compute.internal  Error: ErrImagePull
  Normal   BackOff  3m5s (x21 over 8m7s)  kubelet, ip-10-0-134-11.us-east-2.compute.internal  Back-off pulling image "yapei/wordpress@sha256:92f7a99d237efd588d8e4ddc11e938049ce38816084e35a31a42e8c3cd98a940"

This issue is reproduced on 4.4.0-0.nightly-2020-02-05-220946

Comment 8 Yadan Pei 2020-02-07 10:16:14 UTC

1. Before adding
$ oc describe sa/default
Name:                default
Namespace:           yapei1-1
Labels:              <none>
Annotations:         <none>
Image pull secrets:  default-dockercfg-rzn65
Mountable secrets:   default-dockercfg-rzn65
                     default-token-wxt8b
Tokens:              default-token-qjzs2
                     default-token-wxt8b
Events:              <none>

2. Adding default pull secret for namespace in UI, then check again
[yapei@localhost repos]$ oc describe sa/default
Name:                default
Namespace:           yapei1-1
Labels:              <none>
Annotations:         <none>
Image pull secrets:  default-dockercfg-rzn65
                     yapei-defaultps
Mountable secrets:   default-dockercfg-rzn65
                     default-token-wxt8b
Tokens:              default-token-qjzs2
                     default-token-wxt8b
Events:              <none>

3. Try to deploy from a private image 

$ oc describe pod wordpress-1-p9s2b -n yapei1-1     // image is successfully pulled
Events:
  Type     Reason     Age                From                                                 Message
  ----     ------     ----               ----                                                 -------
  Normal   Scheduled  <unknown>          default-scheduler                                    Successfully assigned yapei1-1/wordpress-1-p9s2b to ip-10-0-164-249.us-east-2.compute.internal
  Normal   Pulling    73s                kubelet, ip-10-0-164-249.us-east-2.compute.internal  Pulling image "yapei/wordpress@sha256:92f7a99d237efd588d8e4ddc11e938049ce38816084e35a31a42e8c3cd98a940"
  Normal   Pulled     56s                kubelet, ip-10-0-164-249.us-east-2.compute.internal  Successfully pulled image "yapei/wordpress@sha256:92f7a99d237efd588d8e4ddc11e938049ce38816084e35a31a42e8c3cd98a940"
  Normal   Created    16s (x4 over 56s)  kubelet, ip-10-0-164-249.us-east-2.compute.internal  Created container wordpress
  Normal   Started    16s (x4 over 56s)  kubelet, ip-10-0-164-249.us-east-2.compute.internal  Started container wordpress
  Normal   Pulled     16s (x3 over 55s)  kubelet, ip-10-0-164-249.us-east-2.compute.internal  Container image "yapei/wordpress@sha256:92f7a99d237efd588d8e4ddc11e938049ce38816084e35a31a42e8c3cd98a940" already present on machine
  Warning  BackOff    1s (x6 over 54s)   kubelet, ip-10-0-164-249.us-east-2.compute.internal  Back-off restarting failed container


Verified on 4.4.0-0.nightly-2020-02-07-001901

Comment 9 Sergio G. 2020-02-21 09:10:20 UTC

Any news on this issue? If it's verified, can we include it in any errata to get it fixed?
Will it be released for 4.4? 4.3?

Comment 10 Sergio G. 2020-02-27 08:46:55 UTC

Can we get an answer to the backport fot 4.3?

Comment 11 Sergio G. 2020-02-28 07:30:40 UTC

Can we get an answer to the backport for 4.3?

Comment 14 errata-xmlrpc 2020-05-04 11:26:45 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581

Comment 15 Red Hat Bugzilla 2024-01-06 04:27:45 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days