Bug 1882440

Summary: When imagestreams and pvcs are excluded, the miganalytic resource and the UI still show them in the main counters
Product: Migration Toolkit for Containers Reporter: Sergio <sregidor>
Component: GeneralAssignee: Jason Montleon <jmontleo>
Status: CLOSED ERRATA QA Contact: Xin jiang <xjiang>
Severity: low Docs Contact: Avital Pinnick <apinnick>
Priority: unspecified    
Version: 1.3.0CC: chezhang, ernelson, rjohnson, rpattath, whu, xjiang
Target Milestone: ---   
Target Release: 1.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-11 12:54:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sergio 2020-09-24 14:40:11 UTC 
Description of problem:
When we configure  disable_image_migration: "true" and disable_pv_migration: true, the miganalytic resource shows a value != 0 in the pvc and images counters.


Version-Release number of selected component (if applicable):
MTC 1.3
TARGET CLUSTER: OCP 4.5
SOURCE CLUSTER: OCP 4.2

How reproducible:
Always

Steps to Reproduce:
1. Disable the images and pv migrations
$ oc patch migrationcontroller migration-controller -p '{"spec":{"disable_image_migration": "true" } }' --type='merge' -n openshift-migration
$ oc patch migrationcontroller migration-controller -p '{"spec":{"disable_pv_migration": true } }' --type='merge' -n openshift-migration

Wait until migrationcontroller pod is restarted

2. In source cluster, create a new application with pvcs and imagestreams

oc new-project bztest
oc new-app cakephp-mysql-persistent


3. Create a migration plan with name "bztest" for this namespace

4. Get the miganalytic resource for this migrationplan



Actual results:
The counters show the images and the pvcs even if they are being ignored in the migration


$ oc get miganalytics bztest
NAME     READY   PLAN     PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY
bztest   True    bztest   100        98          1        221Mi       1     1Gi


In the UI we can see that the analytic shows the number of PVCs and Imagestreams too.

Expected results:
Since the pvcs and imagestreams are not going to be migrated, they should not be shown in the migration plan analytic counters.


Additional info:
Showing 0 value is not right neither, because even if the pvcs/imagestreams are not going to be migrated, they exist and they are there. The UI should display that those PVCs/Imagestreams will be ignored.
Comment 1 whu 2020-09-25 10:24:36 UTC 
I have some similar confusion about the number of `kubenetes resources` in console

I created 2 namespacs, at first both namespace have below resource 
-----------------
ocp-34818-analytics1     services 3  imagestreams 1  pvc  3
ocp-34818-analytics1     services 2  imagestreams 1  pvc  1  
-----------------

after creating migplan and click `refrese` in the console, I got below numbers of `kubenetes resources` 

-----------------
ocp-34818-analytics1  55
ocp-34818-analytics2  46
Plan total 101
-----------------

[setcion 1]

Then I run below steps

-->oc get miganalytic mig-plan-ocp-34818-analytics -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        101         2        442Mi       4     2148Mi       91s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="services")].count}'
3

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="services")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="services")].count}'
2

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="services")].count}'


--> oc patch migrationcontroller migration-controller -p '{"spec":{"excluded_resources": ["clusterserviceversions", "subscriptions", "templateinstances", "imagetags", "services"] }}' --type='merge' -n openshift-migration
migrationcontroller.migration.openshift.io/migration-controller patched

--> oc get migrationcontroller migration-controller -o yaml -n  openshift-migration
  excluded_resources:
  - clusterserviceversions
  - subscriptions
  - templateinstances
  - imagetags
  - services

The original controller pod is migration-controller-7bcc9db575-7vvrd
wait for controllor pod restart
..........
The new controller pod is migration-controller-6d7c968fc-crrfv

--> click refresh in UI page

-->oc get miganalytic mig-plan-ocp-34818-analytics -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        201         2        442Mi       4     2148Mi       36s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="services")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="services")].count}'
3

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="services")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="services")].count}'
2


Now I got below report of  `kubenetes resources`  from console
-----------------
ocp-34818-analytics1  112
ocp-34818-analytics2  89
Plan total 201
-----------------



[setcion 2]

Then I run below steps 
-->oc get miganalytic mig-plan-ocp-34818-analytics  -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        201         2        442Mi       4     2148Mi       37s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="imagestreams")].count}'
1

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="imagestreams")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="imagestreams")].count}'
1

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="imagestreams")].count}'


--> oc patch migrationcontroller migration-controller -p '{"spec":{"disable_image_migration": "true" } }' --type='merge' -n openshift-migration
migrationcontroller.migration.openshift.io/migration-controller patched

--> oc get migrationcontroller migration-controller -o yaml -n  openshift-migration
  disable_image_migration: "true"

The original controller pod is migration-controller-6d7c968fc-crrfv
wait for controllor pod restart
........
The new controller pod is migration-controller-c5957f959-v8zlj


--> Please click refresh in UI page

-->oc get miganalytic mig-plan-ocp-34818-analytics -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        200         2        442Mi       4     2148Mi       42s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="imagestreams")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="imagestreams")].count}'
1

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="imagestreams")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="imagestreams")].count}'
1


Now I got below report of  `kubenetes resources`  from console
-----------------
ocp-34818-analytics1  111
ocp-34818-analytics2  89
Plan total 200
-----------------

[section 3]

Then I run below steps
-->oc get miganalytic mig-plan-ocp-34818-analytics -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        200         2        442Mi       4     2148Mi       43s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'
3

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'
1

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'


--> oc patch migrationcontroller migration-controller -p '{"spec":{"disable_pv_migration": true } }' --type='merge' -n openshift-migration
migrationcontroller.migration.openshift.io/migration-controller patched

--> oc get migrationcontroller migration-controller -o yaml -n  openshift-migration
  disable_pv_migration: true

The original controller pod is migration-controller-c5957f959-v8zlj
wait for controllor pod restart
.......
The new controller pod is migration-controller-57795c768f-kdvqr


--> Please click refresh in UI page

-->oc get miganalytic mig-plan-ocp-34818-analytics -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        196         2        442Mi       4     2148Mi       34s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'
3

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'
1

Now I got below report of  `kubenetes resources`  from console
-----------------
ocp-34818-analytics1  108
ocp-34818-analytics2  88
Plan total 196
-----------------


[section 4]
Then I run below steps

-->oc get miganalytic mig-plan-ocp-34818-analytics  -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        196         2        442Mi       4     2148Mi       35s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="services")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="services")].count}'
3

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="imagestreams")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="imagestreams")].count}'
1

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'
3

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="services")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="services")].count}'
2

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="imagestreams")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="imagestreams")].count}'
1

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'


-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'
1

--> oc patch migrationcontroller migration-controller -p '{"spec":{"excluded_resources": ["clusterserviceversions", "subscriptions", "templateinstances", "imagetags"] }}' --type='merge' -n openshift-migration
migrationcontroller.migration.openshift.io/migration-controller patched


--> oc patch migrationcontroller migration-controller -p '{"spec":{"disable_image_migration": "false" } }' --type='merge' -n openshift-migration
migrationcontroller.migration.openshift.io/migration-controller patched


--> oc patch migrationcontroller migration-controller -p '{"spec":{"disable_pv_migration": false } }' --type='merge' -n openshift-migration
migrationcontroller.migration.openshift.io/migration-controller patched

--> oc get migrationcontroller migration-controller -o yaml -n  openshift-migration
spec:
  azure_resource_group: ""
  cluster_name: host
  disable_image_migration: "false"
  disable_pv_migration: false
  excluded_resources:
  - clusterserviceversions
  - subscriptions
  - templateinstances
  - imagetags
  mig_failure_rollback: false
  mig_namespace_limit: "5"
  mig_pod_limit: "10"
  mig_pv_limit: "6"
  migration_controller: true
  migration_ui: true
  migration_velero: true
  olm_managed: true
  restic_timeout: 1h
  version: 1.0 (OLM)
status:
  phase: Reconciling

The original controller pod is migration-controller-57795c768f-kdvqr
wait for controllor pod restart
.......
The new controller pod is migration-controller-699f48f897-bvjj9


--> Please click refresh in UI page

-->oc get miganalytic mig-plan-ocp-34818-analytics -n openshift-migration
NAME                           READY   PLAN                           PROGRESS   RESOURCES   IMAGES   IMAGESIZE   PVS   PVCAPACITY   AGE
mig-plan-ocp-34818-analytics   True    mig-plan-ocp-34818-analytics   100        207         2        442Mi       4     2148Mi       35s

--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="services")].count}'
3

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="services")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="imagestreams")].count}'
1

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="imagestreams")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'
3

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics1")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="services")].count}'
2

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="services")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="imagestreams")].count}'
1

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="imagestreams")].count}'


--> oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].k8sResources[?(@.kind=="persistentvolumeclaims")].count}'
1

-->oc get miganalytic mig-plan-ocp-34818-analytics -o 'jsonpath={.status.analytics.namespaces[?(@.namespace=="ocp-34818-analytics2")].excludedK8SResources[?(@.kind=="persistentvolumeclaims")].count}'

Now I got below report of  `kubenetes resources`  from console
-----------------
ocp-34818-analytics1  115
ocp-34818-analytics2  92
Plan total 207
-----------------



Only the result of setion 3 and 4 make sense for me.  I feel confused about the result of section 1 and 2
Comment 3 Jason Montleon 2020-09-25 12:44:26 UTC 
There are probably two pieces to this. 

1. We can not spend time counting resources that won't be migrated.
https://github.com/konveyor/mig-controller/pull/676

2. A UI component to replace the resulting 0's or make it clearer that these resources are being skipped because of settings.
Comment 4 Jason Montleon 2020-12-11 17:20:25 UTC 
I think this was fixed with https://github.com/konveyor/mig-controller/pull/676
Comment 8 Sergio 2021-01-11 12:47:51 UTC 
Verified using MTC 1.4.0

openshift-migration-rhel7-operator@sha256:60a0bdc7fca0d3d597efae0c242f7dc25da45c072b33198c3eb7fa425a604472
    - name: MIG_CONTROLLER_REPO
      value: openshift-migration-controller-rhel8@sha256
    - name: MIG_CONTROLLER_TAG
      value: 6f53fa6c8ea2648736ced2d38ebb2ead46d3975f71d7efe4bd24e6fec223aaee

When the image or PV migrations are disabled, the analytic does not count those resources any more.

Moved to VERIFIED status.
Comment 10 errata-xmlrpc 2021-02-11 12:54:46 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Migration Toolkit for Containers (MTC) tool image release advisory 1.4.0), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5329