Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1886125 - Migrations from OCP3.11 to OCP4.3 are not working
Summary: Migrations from OCP3.11 to OCP4.3 are not working
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Migration Toolkit for Containers
Classification: Red Hat
Component: General
Version: 1.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 1.3.z
Assignee: Scott Seago
QA Contact: Xin jiang
Avital Pinnick
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-07 17:05 UTC by Sergio
Modified: 2020-10-22 14:29 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-22 14:28:58 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:4232 0 None None None 2020-10-22 14:29:00 UTC

Description Sergio 2020-10-07 17:05:21 UTC
Description of problem:
Migrations from OCP3.11 to OCP4.3 are not failing, but the result is that no resource is migrated to the target 4.3 cluster


Version-Release number of selected component (if applicable):
MTC 1.3
SOURCE CLUSTER: OCP3.11 AWS
TARGET CLUSTER: OCP4.3 AWS
REPLICATION REPOSITORY: AWS S3


How reproducible:
Always


Steps to Reproduce:
1. Install MTC in an OCP4.3 cluster

2. Add the replication repository

3. Add an OCP 3.11 cluster as source cluster

4. Migrate any application

# Django for instance, but could be any app
oc new-project bztest
oc new-app --template django-psql-persistent


Actual results:
The resources are not migrated, and in the velero restore logs we can read that those resources are being discarded


Expected results:
The resources should be migrated.


Additional info:

Comment 1 Erik Nelson 2020-10-07 17:08:23 UTC
Targeting 1.3.1 for this fix

Comment 2 Scott Seago 2020-10-07 17:55:32 UTC
Fix is here: https://github.com/konveyor/openshift-velero-plugin/pull/60

Comment 8 whu 2020-10-12 10:28:56 UTC
Has been verified in MTC 1.3.1   

source cluster: AWS + ocp 3.11
target cluster: AWS + ocp 4.3
MTC version : 1.3.1
Images:
    - name: HOOK_RUNNER_REPO
      value: openshift-migration-hook-runner-rhel7@sha256
    - name: HOOK_RUNNER_TAG
      value: 5654481182b818e1d64bf45fdc77b26d6a50685bb216ee44ac98fc9349aad4d9
    - name: MIG_CONTROLLER_REPO
      value: openshift-migration-controller-rhel8@sha256
    - name: MIG_CONTROLLER_TAG
      value: 75429a47cf566cb150ff3282e005a01eb6a1e5925d3508ac85404784d5c2c011
    - name: MIG_UI_REPO
      value: openshift-migration-ui-rhel8@sha256
    - name: MIG_UI_TAG
      value: dbdae78e8c6c0ca502e26d948730c987266a12dd98a0205b857c3be49c2efc60
    - name: MIGRATION_REGISTRY_REPO
      value: openshift-migration-registry-rhel8@sha256
    - name: MIGRATION_REGISTRY_TAG
      value: d3ddd2e5f25a72c19dc68259fc5917a7ddc3e6ddfc4e087ab79d12bf4da7c3a0
    - name: VELERO_REPO
      value: openshift-migration-velero-rhel8@sha256
    - name: VELERO_TAG
      value: facc52585b8754d8d0781fa6630a32de99275db049add2068e396b7dd5a4b455
    - name: VELERO_PLUGIN_REPO
      value: openshift-velero-plugin-rhel8@sha256
    - name: VELERO_PLUGIN_TAG
      value: 25a2c3b8d4e59d60a1ea115da2d073ae839f3cdfc61bbe2868d11e74716d5e60
    - name: VELERO_AWS_PLUGIN_REPO
      value: openshift-migration-velero-plugin-for-aws-rhel8@sha256
    - name: VELERO_AWS_PLUGIN_TAG
      value: d45a5b58e0cbc6285e5e6fc881498482cd00f6ff7187ca4ee66a1aab55df345e
    - name: VELERO_GCP_PLUGIN_REPO
      value: openshift-migration-velero-plugin-for-gcp-rhel8@sha256
    - name: VELERO_GCP_PLUGIN_TAG
      value: 8a3bcf50dc79061ed46847fd8a01df30d6b9d81d202cc507685dad503a60e618
    - name: VELERO_AZURE_PLUGIN_REPO
      value: openshift-migration-velero-plugin-for-microsoft-azure-rhel8@sha256
    - name: VELERO_AZURE_PLUGIN_TAG
      value: 7bad9f19b6af44345b8e8b0f09b495972012b890ffd73bb4e97b57048f974ca2
    - name: VELERO_RESTIC_RESTORE_HELPER_REPO
      value: openshift-migration-velero-restic-restore-helper-rhel8@sha256
    - name: VELERO_RESTIC_RESTORE_HELPER_TAG
      value: 14647a69e66de66571d61898f491f6f5fa1f0b1804f87dadf3514b2030e6a4bc
    image: quay-enterprise-quay-enterprise.apps.cam-tgt-9240.qe.devcluster.openshift.com/admin/openshift-migration-rhel7-operator@sha256:7b13a60e87854e11ee4e5ff4857597f63bce042ccddfd5009584ce369c2b733f



Test result:

[In source cluster]

-->To create new namespace ocp-11111-nginx-default
namespace/ocp-11111-nginx-default created

-->To create nginx test data under ocp-11111-nginx-default
persistentvolumeclaim/nginx-logs created
persistentvolumeclaim/nginx-html created
deployment.apps/nginx-deployment created
service/my-nginx created
route.route.openshift.io/my-nginx created
Error from server (AlreadyExists): namespaces "ocp-11111-nginx-default" already exists

No resources found.
-->pod is not start, please wait......
-->pod is not start, please wait......
-->pod is not start, please wait......
-->pod is not start, please wait......
-->pod is not start, please wait......
-->pod is not start, please wait......
nginx-deployment-557dd97bf8-dwljd   1/1       Running   0          31s

-->oc get all -n ocp-11111-nginx-default
NAME                                    READY     STATUS    RESTARTS   AGE
pod/nginx-deployment-557dd97bf8-dwljd   1/1       Running   0          32s

NAME               TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/my-nginx   ClusterIP   172.30.91.61   <none>        8081/TCP   33s

NAME                               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   1         1         1            1           33s

NAME                                          DESIRED   CURRENT   READY     AGE
replicaset.apps/nginx-deployment-557dd97bf8   1         1         1         33s

NAME                                HOST/PORT                                                       PATH      SERVICES   PORT      TERMINATION   WILDCARD
route.route.openshift.io/my-nginx   my-nginx-ocp-11111-nginx-default.apps.1012-w0s.qe.rhcloud.com             my-nginx   8081                    None

-->oc get pvc -n ocp-11111-nginx-default
NAME         STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nginx-html   Bound     pvc-d17efab2-0c73-11eb-803c-0ee22dd9114b   1Gi        RWO            gp2            34s
nginx-logs   Bound     pvc-d176dbf7-0c73-11eb-803c-0ee22dd9114b   1Gi        RWO            gp2            35s

-->To get nginx web page first time
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.15.5</center>
</body>
</html>

-->To create hello world page

-->To get nginx web page second time
<h1>HELLO WORLD</h1>

-->Verify mounted volumes in source (this example is for nfs)
Filesystem            1K-blocks    Used Available Use% Mounted on
overlay                31754240 6911500  24842740  22% /
tmpfs                   3745672       0   3745672   0% /dev
tmpfs                   3745672       0   3745672   0% /sys/fs/cgroup
/dev/mapper/rhel-root  31754240 6911500  24842740  22% /etc/hosts
shm                       65536       0     65536   0% /dev/shm
/dev/xvdbu               999320    2572    980364   1% /var/log/nginx
/dev/xvdbb               999320    2568    980368   1% /usr/share/nginx/html
tmpfs                   3745672      16   3745656   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs                   3745672       0   3745672   0% /proc/acpi
tmpfs                   3745672       0   3745672   0% /proc/scsi
tmpfs                   3745672       0   3745672   0% /sys/firmware

-->The error.log file should contain the error we forced
2020/10/12 10:15:48 [error] 6#6: *1 directory index of "/usr/share/nginx/html/" is forbidden, client: 10.131.0.1, server: localhost, request: "GET / HTTP/1.1", host: "my-nginx-ocp-11111-nginx-default.apps.1012-w0s.qe.rhcloud.com"

-->the access.log file should contain the traces for our requests:
10.131.0.1 - - [12/Oct/2020:10:15:48 +0000] "GET / HTTP/1.1" 403 153 "-" "curl/7.61.1" "66.187.232.129"
10.131.0.1 - - [12/Oct/2020:10:15:55 +0000] "GET / HTTP/1.1" 200 21 "-" "curl/7.61.1" "66.187.232.129"



[In target cluster]
After migration

-->oc get all -n ocp-11111-nginx-default
NAME                                    READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-6578f8dd49-t2dqr   1/1     Running   0          85s

NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/my-nginx   ClusterIP   172.30.196.123   <none>        8081/TCP   84s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   1/1     1            1           85s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-deployment-6578f8dd49   1         1         1       85s

NAME                                HOST/PORT                                                                        PATH   SERVICES   PORT   TERMINATION   WILDCARD
route.route.openshift.io/my-nginx   my-nginx-ocp-11111-nginx-default.apps.cam-tgt-9240.qe.devcluster.openshift.com          my-nginx   8081                 None

-->oc get pvc -n ocp-11111-nginx-default
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nginx-html   Bound    pvc-955ba14f-c4fe-42a5-8b47-58058c5a29e6   1Gi        RWO            gp2            3m6s
nginx-logs   Bound    pvc-f8cde5f3-f88c-4f42-817e-21f5f7941975   1Gi        RWO            gp2            3m6s

-->To get nginx web page first time
<h1>HELLO WORLD</h1>

-->Verify mounted volumes in source (this example is for nfs)
Filesystem                           1K-blocks     Used Available Use% Mounted on
overlay                              125277164 10944188 114332976   9% /
tmpfs                                    65536        0     65536   0% /dev
tmpfs                                  4079704        0   4079704   0% /sys/fs/cgroup
shm                                      65536        0     65536   0% /dev/shm
tmpfs                                  4079704     5544   4074160   1% /etc/passwd
/dev/mapper/coreos-luks-root-nocrypt 125277164 10944188 114332976   9% /etc/hosts
/dev/xvdcf                              999320     2576    980360   1% /var/log/nginx
/dev/xvdbm                              999320     2572    980364   1% /usr/share/nginx/html
tmpfs                                  4079704       24   4079680   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs                                  4079704        0   4079704   0% /proc/acpi
tmpfs                                  4079704        0   4079704   0% /proc/scsi
tmpfs                                  4079704        0   4079704   0% /sys/firmware

-->The error.log file should contain the error we forced
2020/10/12 10:15:48 [error] 6#6: *1 directory index of "/usr/share/nginx/html/" is forbidden, client: 10.131.0.1, server: localhost, request: "GET / HTTP/1.1", host: "my-nginx-ocp-11111-nginx-default.apps.1012-w0s.qe.rhcloud.com"

-->the access.log file should contain the traces for our requests:
10.131.0.1 - - [12/Oct/2020:10:15:48 +0000] "GET / HTTP/1.1" 403 153 "-" "curl/7.61.1" "66.187.232.129"
10.131.0.1 - - [12/Oct/2020:10:15:55 +0000] "GET / HTTP/1.1" 200 21 "-" "curl/7.61.1" "66.187.232.129"
10.129.2.6 - - [12/Oct/2020:10:23:30 +0000] "GET / HTTP/1.1" 200 21 "-" "curl/7.61.1" "66.187.232.129"

Comment 12 errata-xmlrpc 2020-10-22 14:28:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Migration Toolkit for Containers (MTC) tool image release advisory 1.3.1), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4232


Note You need to log in before you can comment on or make changes to this bug.