Description of problem: Using OC client 4.5.1 with a merged KUBECONFIG for multiple clusters, will result in error: You must be logged in to the server (Unauthorized). Version-Release number of selected component (if applicable): OC Client 4.5.1 How reproducible: Always Steps to Reproduce: 1) Download OC Client 4.5.1 2) Have a "merged" KUBECONFIG - pointing to two clusters, like: export KUBECONFIG="/path/to/kubeconfig_1:/path/to/kubeconfig_2" 3) Run: oc status Actual results: $ export KUBECONFIG="/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig" $ /mnt/skynet-data/ocp-install/GOBIN/oc version Client Version: 4.5.1 Server Version: 4.5.1 Kubernetes Version: v1.18.3+8b0a82f $ export KUBECONFIG="/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig" $ /mnt/skynet-data/ocp-install/GOBIN/oc version Client Version: 4.5.1 Server Version: 4.4.3 Kubernetes Version: v1.17.1 $ export KUBECONFIG="/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig:/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig" $ /mnt/skynet-data/ocp-install/GOBIN/oc config view apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://api.default-cl1.devcluster.openshift.com:6443 name: default-cl1 - cluster: certificate-authority-data: DATA+OMITTED server: https://api.nmanos-cluster-a.devcluster.openshift.com:6443 name: nmanos-cluster-a contexts: - context: cluster: nmanos-cluster-a namespace: test-submariner user: admin name: nmanos-cluster-a - context: cluster: default-cl1 namespace: test-submariner user: admin name: nmanos-cluster-b current-context: nmanos-cluster-a kind: Config preferences: {} users: - name: admin user: client-certificate-data: REDACTED client-key-data: REDACTED $ /mnt/skynet-data/ocp-install/GOBIN/oc version Client Version: 4.5.1 error: You must be logged in to the server (Unauthorized) $ /mnt/skynet-data/ocp-install/GOBIN/oc status error: You must be logged in to the server (Unauthorized) Expected results (in OC Client 4.2 for example): $ export KUBECONFIG="/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig:/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig" $ oc version Client Version: version.Info{Major:"", Minor:"", GitVersion:"v4.2.0-alpha.0-4-g38b0f09", GitCommit:"38b0f09", GitTreeState:"clean", BuildDate:"2019-08-12T19:05:43Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.3+8b0a82f", GitCommit:"8b0a82f", GitTreeState:"clean", BuildDate:"2020-07-10T05:34:00Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"} OpenShift Version: 4.5.1 $ oc status In project test-submariner on server https://api.nmanos-cluster-a.devcluster.openshift.com:6443 Additional info: -
I just verified this with an earlier version of oc 4.5.0-rc.6 and this works as expected. I'd suggest checking if each of the on their own are working as expected, first and then try merging them.
It is reproduced on my env. Please try to check it when one of the clusters is using older OCP version: On Cluster A: $ export KUBECONFIG="/mnt/skynet-data/nmanos-cluster-a/auth/kubeconfig" $ oc version Client Version: 4.5.2 Server Version: 4.5.2 Kubernetes Version: v1.18.3+b74c5ed On Cluster B: $ export KUBECONFIG="/mnt/skynet-data/nmanos-cluster-b/auth/kubeconfig" $ oc version Client Version: 4.5.2 Server Version: 4.4.3 Kubernetes Version: v1.17.1 $ export KUBECONFIG="/mnt/skynet-data/nmanos-cluster-a/auth/kubeconfig:/mnt/skynet-data/nmanos-cluster-b/auth/kubeconfig" $ oc get all -A error: You must be logged in to the server (Unauthorized)
The cluster version has nothing to do with your problem, I've verified oc binaries all the way back to 4.3 and all worked w/o any problems. Please ensure that you can invoke oc get all with each of the KUBECONFIGs and only then verify merging. From my tests all worked just fine.
### Using OC client 4.2 : $ oc version Client Version: version.Info{Major:"", Minor:"", GitVersion:"v4.2.0-alpha.0-4-g38b0f09", GitCommit:"38b0f09", GitTreeState:"clean", BuildDate:"2019-08-12T19:05:43Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0+d4cacc0", GitCommit:"d4cacc0", GitTreeState:"clean", BuildDate:"2020-06-08T20:03:02Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"} ### On 1st Cluster - OC 4.2 works as expected: $ export KUBECONFIG=/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig $ oc status In project test-submariner on server https://api.default-cl1.devcluster.openshift.com:6443 svc/nginx-cl-b - 100.96.22.141:8080 deployment/nginx-cl-b deploys nginxinc/nginx-unprivileged:stable-alpine deployment #1 running for 2 hours - 1 pod 1 info identified, use 'oc status --suggest' to see details. ### On 2nd Cluster - OC 4.2 works as expected: $ export KUBECONFIG=/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig $ oc status In project test-submariner on server https://api.nmanos-cluster-a.devcluster.openshift.com:6443 pod/netshoot-cl-a runs nicolaka/netshoot 2 infos identified, use 'oc status --suggest' to see details. ### With MERGED Kubeconfigs - OC 4.2 works: # Note that it retrieves 1st Cluster status only, so it's not the current bug issue (but might be related). $ export KUBECONFIG=/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig:/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig $ oc status In project test-submariner on server https://api.default-cl1.devcluster.openshift.com:6443 svc/nginx-cl-b - 100.96.22.141:8080 deployment/nginx-cl-b deploys nginxinc/nginx-unprivileged:stable-alpine deployment #1 running for 2 hours - 1 pod 1 info identified, use 'oc status --suggest' to see details. ### Now using OC client 4.5: $ OC45=/mnt/skynet-data/ocp-install/GOBIN/oc ### On 1st Cluster - OC 4.5 works as expected: $ export KUBECONFIG=/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig $ $OC45 version Client Version: 4.5.3 Server Version: 4.4.3 Kubernetes Version: v1.17.1 $ $OC45 status In project test-submariner on server https://api.default-cl1.devcluster.openshift.com:6443 svc/nginx-cl-b - 100.96.22.141:8080 deployment/nginx-cl-b deploys nginxinc/nginx-unprivileged:stable-alpine deployment #1 running for 3 hours - 1 pod 1 info identified, use 'oc status --suggest' to see details. ### On 2nd Cluster - OC 4.5 works as expected: $ export KUBECONFIG=/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig $ $OC45 version Client Version: 4.5.3 Server Version: 4.5.3 Kubernetes Version: v1.18.3+3107688 $ $OC45 status In project test-submariner on server https://api.nmanos-cluster-a.devcluster.openshift.com:6443 pod/netshoot-cl-a runs nicolaka/netshoot You have no services, deployment configs, or build configs. Run 'oc new-app' to create an application. ### But with MERGED Kubeconfigs - OC 4.5 fails on "Unauthorized" error: $ export KUBECONFIG=/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig:/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig $ $OC45 status error: You must be logged in to the server (Unauthorized) $ $OC45 version Client Version: 4.5.3 error: You must be logged in to the server (Unauthorized) ### Trying with verbose command - see how curl to the API returned error 401: bash-4.2$ $OC45 -v=9 version I0726 11:15:19.005285 54189 loader.go:375] Config loaded from file: /mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig I0726 11:15:19.007042 54189 loader.go:375] Config loaded from file: /mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig I0726 11:15:19.008893 54189 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/b66f2d3" 'https://api.nmanos-cluster-a.devcluster.openshift.com:6443/version?timeout=32s' I0726 11:15:19.087203 54189 round_trippers.go:443] GET https://api.nmanos-cluster-a.devcluster.openshift.com:6443/version?timeout=32s 401 Unauthorized in 78 milliseconds I0726 11:15:19.087236 54189 round_trippers.go:449] Response Headers: I0726 11:15:19.087242 54189 round_trippers.go:452] Audit-Id: 9fd55163-9229-427a-b892-8803efe7a6e3 I0726 11:15:19.087246 54189 round_trippers.go:452] Cache-Control: no-cache, private I0726 11:15:19.087250 54189 round_trippers.go:452] Content-Type: application/json I0726 11:15:19.087254 54189 round_trippers.go:452] Content-Length: 129 I0726 11:15:19.087258 54189 round_trippers.go:452] Date: Sun, 26 Jul 2020 08:15:34 GMT I0726 11:15:19.096685 54189 request.go:1068] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401} I0726 11:15:19.102904 54189 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: oc/openshift (linux/amd64) kubernetes/b66f2d3" 'https://api.nmanos-cluster-a.devcluster.openshift.com:6443/apis/config.openshift.io/v1/clusteroperators/openshift-apiserver' I0726 11:15:19.122818 54189 round_trippers.go:443] GET https://api.nmanos-cluster-a.devcluster.openshift.com:6443/apis/config.openshift.io/v1/clusteroperators/openshift-apiserver 401 Unauthorized in 19 milliseconds I0726 11:15:19.122863 54189 round_trippers.go:449] Response Headers: I0726 11:15:19.122871 54189 round_trippers.go:452] Content-Type: application/json I0726 11:15:19.122879 54189 round_trippers.go:452] Content-Length: 129 I0726 11:15:19.122885 54189 round_trippers.go:452] Date: Sun, 26 Jul 2020 08:15:34 GMT I0726 11:15:19.122893 54189 round_trippers.go:452] Audit-Id: a8d1d3bc-0c41-4a11-ad05-a9467542bd9c I0726 11:15:19.122900 54189 round_trippers.go:452] Cache-Control: no-cache, private I0726 11:15:19.122956 54189 request.go:1068] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401} Client Version: 4.5.3 I0726 11:15:19.124215 54189 helpers.go:216] server response object: [{ "metadata": {}, "status": "Failure", "message": "Unauthorized", "reason": "Unauthorized", "code": 401 }] F0726 11:15:19.124273 54189 helpers.go:115] error: You must be logged in to the server (Unauthorized)
I’m adding UpcomingSprint, because I was occupied by fixing bugs with higher priority/severity, developing new features with higher priority, or developing new features to improve stability at a macro level. I will revisit this bug next sprint.
This bug hasn't had any activity in the last 30 days. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're marking this bug as "LifecycleStale" and decreasing the severity/priority. If you have further information on the current state of the bug, please update it, otherwise this bug can be closed in about 7 days. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.
On my env this is still reproduced: export KUBECONFIG="/mnt/skynet-data/ocp-install/nmanos-cluster-a/auth/kubeconfig:/mnt/skynet-data/ocp-install/ocpup/.config/cl1/auth/kubeconfig" oc version Client Version: 4.5.7 error: You must be logged in to the server (Unauthorized) Let me know if you need more information.
The LifecycleStale keyword was removed because the bug got commented on recently. The bug assignee was notified.
This bug hasn't had any activity in the last 30 days. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're marking this bug as "LifecycleStale" and decreasing the severity/priority. If you have further information on the current state of the bug, please update it, otherwise this bug can be closed in about 7 days. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. Additionally, you can add LifecycleFrozen into Keywords if you think this bug should never be marked as stale. Please consult with bug assignee before you do that.
*** This bug has been marked as a duplicate of bug 1868384 ***