Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1746687

Summary: Faied to execute Migration operation since user cannot get replicationcontrollers.
Product: OpenShift Container Platform Reporter: Xin jiang <xjiang>
Component: Migration ToolingAssignee: Jason Montleon <jmontleo>
Status: CLOSED ERRATA QA Contact: Xin jiang <xjiang>
Severity: medium Docs Contact:
Priority: high    
Version: 4.2.0CC: chezhang, sregidor, xjiang
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-16 06:38:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Xin jiang 2019-08-29 06:26:01 UTC
Description of problem:

Faied to execute Migration operation since user cannot get replicationcontrollers.

Version-Release number of selected component (if applicable):
OCP version: openshift v3.9.97
Severity:  medium 
priority:  medium

migration-operator:
    image: quay.io/ocpmigrate/mig-operator:stable
    imageID: docker-pullable://quay.io/ocpmigrate/mig-operator@sha256:e9f7132c9f87c243574274720eb8938896b949e5ab276bdd267ca249f7b2e604

velero:
    image: quay.io/ocpmigrate/migration-plugin:stable
    imageID: docker-pullable://quay.io/ocpmigrate/migration-plugin@sha256:b4493d826260eb1e3e02ba935aaedfd5310fefefb461ca7dcd9a5d55d4aa8f35        
    image: quay.io/ocpmigrate/velero:stable
    imageID: docker-pullable://quay.io/ocpmigrate/velero@sha256:957725dec5f0fb6a46dee78bd49de9ec4ab66903eabb4561b62ad8f4ad9e6f05

How reproducible:
always

Steps to Reproduce:
1. Install migration controller on ocp 3.9
oc create -f https://raw.githubusercontent.com/fusor/mig-operator/master/operator.yml
oc create -f https://raw.githubusercontent.com/fusor/mig-operator/master/controller-3.yml

2. Install migration controller on ocp 4.2
oc create -f https://raw.githubusercontent.com/fusor/mig-operator/master/operator.yml
oc create -f https://raw.githubusercontent.com/fusor/mig-operator/master/controller-4.yml

3. Create the mssql app
oc create -f https://raw.githubusercontent.com/XinRedhat/appmig-test/master/mssql-pv.yml

4. Add a migration plan for the mssql app 

5. Execute Stage for the migration plan

6. Execute Migrate for the migration plan 

Actual results:
Migration action failed and found error

# oc logs registry-mssql-fd4k6-1-deploy
error: couldn't get deployment registry-mssql-fd4k6-1: replicationcontrollers "registry-mssql-fd4k6-1" is forbidden: User "system:serviceaccount:openshift-migration:deployer" cannot get replicationcontrollers in the namespace "openshift-migration": User "system:serviceaccount:openshift-migration:deployer" cannot get replicationcontrollers in project "openshift-migration"


Expected results:
Migration should be successful and the mssql app is migrated to ocp 4

Additional info:

I tried to install migration controller on ocp 3.11 with the same commands and found difference against rolebindings.
ocp3.9
# oc get rolebindings -n openshift-migration
NAME                 ROLE                                     USERS     GROUPS    SERVICE ACCOUNTS     SUBJECTS
migration-operator   openshift-migration/migration-operator

3.11
# oc get rolebindings -n openshift-migration
NAME                    ROLE                                     USERS     GROUPS                                       SERVICE ACCOUNTS     SUBJECTS
migration-operator      openshift-migration/migration-operator                                                          migration-operator
system:deployers        /system:deployer                                                                                deployer
system:image-builders   /system:image-builder                                                                           builder
system:image-pullers    /system:image-puller

Comment 1 Zihan Tang 2019-08-29 09:09:01 UTC
In 3.7 also hit this issue:

oc logs -f registry-sets-k4k55-1-deploy -n openshift-migration
error: couldn't get deployment registry-sets-k4k55-1: User "system:serviceaccount:openshift-migration:deployer" cannot get replicationcontrollers in the namespace "openshift-migration": User "system:serviceaccount:openshift-migration:deployer" cannot get replicationcontrollers in project "openshift-migration" (get replicationcontrollers registry-sets-k4k55-1)

Comment 2 John Matthews 2019-08-30 13:52:07 UTC
Related
https://access.redhat.com/solutions/3419991

Environment
OpenShift Container Platform
3.7
Issue
Receieving a message similar to the following when attempting a deployment.

Raw
error: couldn't get deployment <deployment>: User "system:serviceaccount:<service account>:deployer" cannot get replicationcontrollers in the namespace "<namespace>": User "system:serviceaccount:<service account>:deployer" cannot get replicationcontrollers in project <project> (get replicationcontrollers <deployment>)
Resolution
Projects should be created using the ProjectRequest API. If it was not, you'll need to create the following role bindings within the project:

Raw
# Add `deployer` serviceaccount to `system:deployer` role
$ oc -n <project> adm policy add-role-to-user -z deployer system:deployer

# Add `builder` serviceaccount to `system:image-builder` role
$ oc -n <project> adm policy add-role-to-user -z builder system:image-builder

# Add `system:serviceaccounts:<project>` group  to `system:image-puller` role
$ oc -n <project> adm policy add-role-to-group system:image-puller system:serviceaccounts:<project>
Root Cause
Role bindings missing for deployer Service Account.

Comment 3 John Matthews 2019-08-30 14:13:05 UTC
Would like to see if the solution to this can be as simple as change creation of 'Namespace' to 'Project' in mig-operator:
  https://github.com/fusor/mig-operator/blob/master/roles/migrationcontroller/templates/mig_namespace.yml.j2

Comment 4 Jason Montleon 2019-08-30 15:41:13 UTC
This article describes the problem:
https://access.redhat.com/solutions/3419991

Basically if the namespace is not created with the ProjectRequest API in earlier versions of Openshift the default rolebindings do not get created.

I tried creating a project instead of a namespace in operator.yml by altering it to look like:
---
apiVersion: v1
kind: Project
metadata:
  labels:
    control-plane: controller-manager
    controller-tools.k8s.io: "1.0"
  name: "openshift-migration"

This did not change the behavior and still does not appear to use the ProjectRequest API and also does not create the rolebindings. I think we have a question out to someone who would know better as to whether there is a better way to do this with yaml using oc create -f ...

If no better solution turns up we can:
A.) Add the rolebinding creation to the operator.yml, which would probably just result in a couple/few AlreadyExists messages on 3.10/3.11.

B.) Remove the namespace creation from the yaml and add a small script that does something like:
#!/bin/bash
oc new-project openshift-migration
oc create -f operator.yml

Right now I think a couple folks have said they would prefer option A.

Comment 5 Jason Montleon 2019-08-30 16:21:24 UTC
Can you try after creating the operator with this yml? If it works we'll go ahead and merge it.

https://github.com/fusor/mig-operator/blob/a0b90dd352a450ec2deafd65ecec60237c3fddd1/operator.yml


https://github.com/fusor/mig-operator/pull/61

Comment 6 Jason Montleon 2019-08-30 16:54:19 UTC
Looks like we can literally just do a ProjectRequest in the yaml:
https://github.com/fusor/mig-operator/blob/b6c411630c5e6a29c6cc52dd988d4cdd1b4e4468/operator.yml

Comment 7 Jason Montleon 2019-09-04 14:02:42 UTC
Using https://github.com/fusor/mig-operator/blob/master/operator-3.yml to install the operator on OpenShift 3 should prevent the issue.

Comment 8 Xin jiang 2019-09-06 04:58:27 UTC
this issue is gone in following version.
  - containerID: cri-o://f2a992396101cdd8a6f68b946894dfa8694688491990093d12f88b130bb0e2d1
    image: quay.io/ocpmigrate/mig-operator:stable
    imageID: quay.io/ocpmigrate/mig-operator@sha256:4fc8089f6d0199191d65ae002825df3ff8aaedf3b52069f7c6e5376c20291f31

Comment 9 errata-xmlrpc 2019-10-16 06:38:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922