Bug 1538261 - 3.5 to 3.6 upgrade does not reconcile roles properly
Summary: 3.5 to 3.6 upgrade does not reconcile roles properly
Keywords:
Status: CLOSED DUPLICATE of bug 1493213
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: apiserver-auth
Version: 3.6.1
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Simo Sorce
QA Contact: Chuan Yu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-24 17:56 UTC by Steven Walter
Modified: 2021-03-11 17:00 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-01-26 14:25:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3331671 0 None None None 2018-01-24 21:04:34 UTC

Description Steven Walter 2018-01-24 17:56:24 UTC
Description of problem:
Customer upgraded from 3.5 to 3.6. Now they cannot pull images from openshift project and some other roles seem unusual as well.

Version-Release number of selected component (if applicable):
v3.6.173.0.21

How reproducible:
Unconfirmed


Actual results:
- All nodes fail to pull images from the `openshift` project in the internal registry with:
~~~
Failed to pull image "172.30.217.14:5000/openshift/example@sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx": rpc error: code = 2 desc = unauthorized: authentication required
~~~

- I created a unique namespace for each project and can now grant a user access to a particular project BUT now that user cannot use or create a new project
~~~
Error
Access denied
The project testing does not exist or you are not authorized to view it.
~~~

- I moved each image from `/openshift/projectname` to `/projectname/projectname` and then assigned access to a new user:
~~~
# oadm policy add-role-to-user admin <username> -n <existingprojectname>
~~~
- But user still does not have access, even with "admin" role added


Expected results:
Pull from openshift normally; be admin in project; etc

Additional info:
When we run:

# oc adm policy reconcile-cluster-roles --additive-only=false -o name
clusterrole/sudoer
clusterrole/cluster-reader
clusterrole/system:build-strategy-docker
clusterrole/system:build-strategy-custom
clusterrole/system:build-strategy-source
clusterrole/system:build-strategy-jenkinspipeline
clusterrole/admin
clusterrole/edit
clusterrole/view
clusterrole/basic-user
clusterrole/self-access-reviewer
clusterrole/self-provisioner
clusterrole/cluster-status
clusterrole/system:image-auditor
clusterrole/system:image-puller
clusterrole/system:image-pusher
clusterrole/system:image-builder
clusterrole/system:image-pruner
clusterrole/system:image-signer
clusterrole/system:deployer
clusterrole/system:oauth-token-deleter
clusterrole/system:router
clusterrole/system:registry
clusterrole/system:node
clusterrole/system:sdn-reader
clusterrole/system:sdn-manager
clusterrole/system:webhook
clusterrole/system:discovery
clusterrole/registry-admin
clusterrole/registry-editor
clusterrole/registry-viewer
clusterrole/system:node-bootstrapper

We also notice that the shared-resource-viewer policy in the openshift project is missing some entries:

Customer's:

  roles:
  - name: shared-resource-viewer
    role:
      . . .
      rules:
      - apiGroups:
        - ""

Mine:

  roles:
  - name: shared-resource-viewer
    role:
      . . .
      rules:
      - apiGroups:
        - template.openshift.io
        - ""

It looks like oc adm policy reconcile-cluster-roles --additive=true as run in the upgrader script does not make the required changes to apiGroups

Comment 1 Simo Sorce 2018-01-24 18:23:35 UTC
Which version of the oc command are you using to reconcile ?
And di you use --confirm ?

Comment 2 Steven Walter 2018-01-24 18:56:30 UTC
oc version
oc v3.5.5.31
kubernetes v1.5.2+43a9be4
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://xxxxxxxx:443
openshift v3.6.173.0.21
kubernetes v1.6.1+5115d708d7


Customer has not tried --confirm yet but is expected to soon

Comment 3 Simo Sorce 2018-01-24 20:04:22 UTC
Please ask them to use a 3.6 client too, I think that's required.

Comment 4 Steven Walter 2018-01-25 14:37:11 UTC
# oc adm policy reconcile-cluster-roles --additive-only=false --confirm
clusterrole/system:replication-controller
clusterrole/system:endpoint-controller
clusterrole/system:replicaset-controller
clusterrole/system:job-controller
clusterrole/system:hpa-controller
clusterrole/system:daemonset-controller
clusterrole/system:disruption-controller
clusterrole/system:namespace-controller
clusterrole/system:gc-controller
clusterrole/system:certificate-signing-controller
clusterrole/system:statefulset-controller
clusterrole/system:build-controller
clusterrole/system:deploymentconfig-controller
clusterrole/system:deployment-controller
# oc version
oc v3.6.173.0.21
kubernetes v1.6.1+5115d708d7
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://example.com:443
openshift v3.6.173.0.21
kubernetes v1.6.1+5115d708d7


apiVersion: v1
items:
- apiVersion: v1
  kind: Policy
  lastModified: 2017-08-15T17:52:40Z
  metadata:
    creationTimestamp: 2017-08-15T17:52:40Z
    name: default
    namespace: openshift
    resourceVersion: "4135131"
    selfLink: /oapi/v1/namespaces/openshift/policies/default
    uid: 886e0c01-81e2-11e7-b5fd-001a4ac8872c
  roles:
  - name: shared-resource-viewer
    role:
      metadata:
        creationTimestamp: 2017-08-15T17:52:40Z
        name: shared-resource-viewer
        namespace: openshift
        resourceVersion: "95"
        uid: 886de1cc-81e2-11e7-b5fd-001a4ac8872c
      rules:
      - apiGroups:
        - ""
        attributeRestrictions: null
        resources:
        - templates
        verbs:
        - get
        - list
        - watch
      - apiGroups:
        - ""
        attributeRestrictions: null
        resources:
        - imagestreamimages
        - imagestreams
        - imagestreamtags
        verbs:
        - get
        - list
        - watch
      - apiGroups:
        - ""
        attributeRestrictions: null
        resources:
        - imagestreams/layers
        verbs:
        - get
kind: List
metadata: {}
resourceVersion: ""
selfLink: ""

Comment 5 Mo 2018-01-26 00:26:58 UTC
shared-resource-viewer does not get updated via that command.

This is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1493213 which was fixed in https://github.com/openshift/openshift-ansible/pull/5649

It can be fixed "manually" by running https://github.com/openshift/origin/pull/16465#issuecomment-331203378

Please mark duplicate and close.

Comment 6 Simo Sorce 2018-01-26 14:25:40 UTC

*** This bug has been marked as a duplicate of bug 1493213 ***


Note You need to log in before you can comment on or make changes to this bug.