Bug 1731278

Summary: Bootstrap user login test breaks kubeadmin user on Azure
Product: OpenShift Container Platform Reporter: Mike Fiedler <mifiedle>
Component: apiserver-authAssignee: Mo <mkhan>
Status: CLOSED ERRATA QA Contact: Mike Fiedler <mifiedle>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.2.0CC: aos-bugs, mfojtik, mkhan, nagrawal
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-16 06:29:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Mike Fiedler 2019-07-18 21:27:20 UTC 
Description of problem:

Running the extended test "The bootstrap user should successfully login with password decoded from kubeadmin secret [Suite:openshift/conformance/parallel]"
on Azure breaks kubeadmin every time.

The test is supposed to restore the kube-system/kubeadmin secret back to its former state after the test but for some reason on Azure it does not.   The test works fine on AWS

Version-Release number of selected component (if applicable):  
cluster:  4.2.0-0.nightly-2019-07-17-083655
openshift-tests: from the same build


How reproducible: Always


Steps to Reproduce:
1.  Install an IPI standard cluster on Azure
2.  KUBECONFIG=/root/.kube/config 
3.  openshift-tests run-test  "The bootstrap user should successfully login with password decoded from kubeadmin secret [Suite:openshift/conformance/parallel]"

Actual results:

The test fails with a 401 Unauthorized after the attempt to restore the kubeadmin secret.   Login as kubeadmin after this is impossible

Expected results:

Test passes

Additional info:

kubeadmin secret before running the test:

apiVersion: v1
data:
  kubeadmin: JDJhJDEwJFZMOFVBbDFPbWxjcjNNVDgwc2NLSi44SkVhaDNuQ05zU0JOUlIzWElySG5aWXRjTU0ycG0u
kind: Secret
metadata:
  creationTimestamp: "2019-07-17T20:22:25Z"
  name: kubeadmin
  namespace: kube-system
  resourceVersion: "52"
  selfLink: /api/v1/namespaces/kube-system/secrets/kubeadmin
  uid: 9767c691-a8d0-11e9-9616-000d3a945d2f
type: Opaque

kubeadmin secret after running the test:

apiVersion: v1
data:
  kubeadmin: JDJhJDEwJERpdUJLdkxYcnFGdnByMnZzZTJtRk9sQUpFT0c4eFUwZGlXaEV2QS9RQ3RZdXBscXN6cDAu
kind: Secret
metadata:
  creationTimestamp: "2019-07-17T20:22:25Z"
  name: kubeadmin
  namespace: kube-system
  resourceVersion: "308166"
  selfLink: /api/v1/namespaces/kube-system/secrets/kubeadmin
  uid: 9767c691-a8d0-11e9-9616-000d3a945d2f
type: Opaque


When run on AWS the before/after are identical.
Comment 1 Mo 2019-07-19 15:52:15 UTC 
#23440 will make it so that the test always restores the cluster's state.

I will need logs from that e2e failure to actually determine _why_ it is failing on Azure.
Comment 3 Mike Fiedler 2019-07-25 13:35:29 UTC 
Verified on 4.2.0-0.nightly-2019-07-25-075148
Comment 4 errata-xmlrpc 2019-10-16 06:29:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922