Description of problem: It's missing rsync-transfer image definition in OCP 3.x operator.yml, it causes ImagePullBackOff issue. - name: RSYNC_TRANSFER_REPO value: openshift-migration-rsync-transfer-rhel8@sha256 - name: RSYNC_TRANSFER_TAG value: 715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a Version-Release number of selected component (if applicable): MTC 1.4.0 openshift-migration-rhel7-operator@sha256:236d5f964061938f4e845e6d9e62926da68435c0f0304329a0473e060cdb53f1 How reproducible: Always Steps to Reproduce: 1. Download operator.yml $ podman cp $(podman create brew.registry.redhat.io/rhmtc/openshift-migration-rhel7-operator@sha256:236d5f964061938f4e845e6d9e62926da68435c0f0304329a0473e060cdb53f1):/operator.yml ./ 2. check operator.yml Actual results: It's missing rsync-transfer image - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: REGISTRY value: registry.redhat.io - name: PROJECT value: rhmtc - name: HOOK_RUNNER_REPO value: openshift-migration-hook-runner-rhel7@sha256 - name: HOOK_RUNNER_TAG value: 69b322784c2fc4227062daed570010add395734b8b2358b1dcb41fb97a209d5a - name: MIG_CONTROLLER_REPO value: openshift-migration-controller-rhel8@sha256 - name: MIG_UI_REPO value: openshift-migration-ui-rhel8@sha256 - name: MIG_LOG_READER_REPO value: openshift-migration-log-reader-rhel8@sha256 - name: MIGRATION_REGISTRY_REPO value: openshift-migration-registry-rhel8@sha256 - name: MIGRATION_REGISTRY_TAG value: ce78f94658169f04e67d2385998c42aad61d53fd12c45aa6911ee28f709fe2e9 - name: VELERO_REPO value: openshift-migration-velero-rhel8@sha256 - name: VELERO_PLUGIN_REPO value: openshift-velero-plugin-rhel8@sha256 - name: VELERO_RESTIC_RESTORE_HELPER_REPO value: openshift-migration-velero-restic-restore-helper-rhel8@sha256 - name: VELERO_AWS_PLUGIN_REPO value: openshift-migration-velero-plugin-for-aws-rhel8@sha256 - name: VELERO_GCP_PLUGIN_REPO value: openshift-migration-velero-plugin-for-gcp-rhel8@sha256 - name: VELERO_AZURE_PLUGIN_REPO value: openshift-migration-velero-plugin-for-microsoft-azure-rhel8@sha256 - name: VELERO_TAG value: 9c065c17a02f7f7cd67eac37dabe6af0845002a77839823f23fef2b03ff6bbed - name: VELERO_RESTIC_RESTORE_HELPER_TAG value: 42188e1b4b1901096e5af2dbb98c9eb01f8eb5f5501bc78033167dd3a2229131 - name: VELERO_PLUGIN_TAG value: 84d17dba4c0e46246e24255287b1e16e124a8eeccb4d14c5d1c0eda24c1c0410 - name: VELERO_AWS_PLUGIN_TAG value: b88f43762a26aa53c621cecf7611fb6be13fd73f31bf6423ad5e456e22fa0854 - name: VELERO_GCP_PLUGIN_TAG value: 74c6295ddad4daf584f45aa41dd0411ebaac806a35713f9b5a5c607cc81c4ff6 - name: VELERO_AZURE_PLUGIN_TAG value: 9a83890dcc3836c34f45c927fe7a887f58af8c86aef5708829936527627b44cf - name: MIG_UI_TAG value: 6aabdf05a9b795cde3a224be3ad917d4246e17db66fba9910080eeb460aed7d8 - name: MIG_CONTROLLER_TAG value: 5671667a75e12cd538a34c208bfb076534296cdf91f7e97ef48fc59f8ce3c172 - name: MIG_LOG_READER_TAG value: a1cfa2f7f96e5deaf86c07c784d20debb13fa95b006bb735ffd2e08043d1cce5 volumes: - name: runner emptyDir: {} Expected results: rsync-transfer image should be defined in operator.yml - name: RSYNC_TRANSFER_REPO value: openshift-migration-rsync-transfer-rhel8@sha256 - name: RSYNC_TRANSFER_TAG value: 715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a Additional info:
Jason, Even I manually modified operator.yaml by adding rsync-transfer image and pointed image registry to "brew.registry.redhat.io". it still pull down image from "registry.redhat.io" (This is OCP 3.11 cluster) . operator.yml: ...... spec: serviceAccountName: migration-operator containers: - name: operator image: brew.registry.redhat.io/rhmtc/openshift-migration-rhel7-operator@sha256:236d5f964061938f4e845e6d9e62926da68435c0f0304329a0473e060cdb53f1 imagePullPolicy: Always ...... - name: REGISTRY value: brew.registry.redhat.io - name: PROJECT value: rhmtc - name: RSYNC_TRANSFER_REPO value: openshift-migration-rsync-transfer-rhel8@sha256 - name: RSYNC_TRANSFER_TAG value: 715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a $ oc get pod -n openshift-migration migration-operator-58fbc99b59-k4gzt -o yaml apiVersion: v1 kind: Pod metadata: annotations: openshift.io/scc: anyuid creationTimestamp: 2021-01-04T07:59:33Z generateName: migration-operator-58fbc99b59- labels: app: migration pod-template-hash: "1496755615" name: migration-operator-58fbc99b59-k4gzt namespace: openshift-migration ownerReferences: - apiVersion: apps/v1 blockOwnerDeletion: true controller: true kind: ReplicaSet name: migration-operator-58fbc99b59 uid: c822f5a0-4e62-11eb-9616-0e5a72214cc1 resourceVersion: "63899" selfLink: /api/v1/namespaces/openshift-migration/pods/migration-operator-58fbc99b59-k4gzt uid: c829fe52-4e62-11eb-9616-0e5a72214cc1 spec: containers: - env: - name: OPERATOR_NAME value: migration-operator - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: WATCH_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: REGISTRY value: brew.registry.redhat.io - name: PROJECT value: rhmtc - name: RSYNC_TRANSFER_REPO value: openshift-migration-rsync-transfer-rhel8@sha256 - name: RSYNC_TRANSFER_TAG value: 715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a $ oc -n ocp-24769-cakephp describe pod directvolumemigration-stunnel-transfer ..... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 4m default-scheduler Successfully assigned ocp-24769-cakephp/directvolumemigration-stunnel-transfer to ip-172-18-3-10.ec2.internal Normal Pulling 3m (x2 over 4m) kubelet, ip-172-18-3-10.ec2.internal pulling image "registry.redhat.io/rhmtc/openshift-migration-rsync-transfer-rhel8@sha256:715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a" Warning Failed 3m (x2 over 3m) kubelet, ip-172-18-3-10.ec2.internal Failed to pull image "registry.redhat.io/rhmtc/openshift-migration-rsync-transfer-rhel8@sha256:715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a": rpc error: code = Unknown desc = unknown: Not Found Warning Failed 3m (x2 over 3m) kubelet, ip-172-18-3-10.ec2.internal Error: ErrImagePull Normal SandboxChanged 3m (x7 over 3m) kubelet, ip-172-18-3-10.ec2.internal Pod sandbox changed, it will be killed and re-created. Normal BackOff 3m (x6 over 3m) kubelet, ip-172-18-3-10.ec2.internal Back-off pulling image "registry.redhat.io/rhmtc/openshift-migration-rsync-transfer-rhel8@sha256:715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a" Warning Failed 3m (x6 over 3m) kubelet, ip-172-18-3-10.ec2.internal Error: ImagePullBackOff
Please post the contents of the mig-cluster-config configmap.
$ oc get configmap -n openshift-migration migration-cluster-config -o yaml apiVersion: v1 data: REGISTRY_IMAGE: brew.registry.redhat.io/rhmtc/openshift-migration-registry-rhel8@sha256:ce78f94658169f04e67d2385998c42aad61d53fd12c45aa6911ee28f709fe2e9 RSYNC_TRANSFER_IMAGE: brew.registry.redhat.io/rhmtc/openshift-migration-rsync-transfer-rhel8@sha256:715d6869c6d23d572808d64c511b94c6af07a6bce08b4f62a78542b61f3af02a STAGE_IMAGE: brew.registry.redhat.io/rhmtc/openshift-migration-velero-restic-restore-helper-rhel8@sha256:42188e1b4b1901096e5af2dbb98c9eb01f8eb5f5501bc78033167dd3a2229131 kind: ConfigMap metadata: creationTimestamp: 2021-01-04T08:01:39Z labels: control-plane: controller-manager controller-tools.k8s.io: "1.0" name: migration-cluster-config namespace: openshift-migration ownerReferences: - apiVersion: migration.openshift.io/v1alpha1 kind: MigrationController name: migration-controller uid: 09bf136f-4e63-11eb-9616-0e5a72214cc1 resourceVersion: "64133" selfLink: /api/v1/namespaces/openshift-migration/configmaps/migration-cluster-config uid: 12f174b7-4e63-11eb-9616-0e5a72214cc1
controller PR: https://github.com/konveyor/mig-controller/pull/871
verified that it can pull down rsync-transfer image from expected registry on 3.x. directvolumemigration-stunnel-transfer 0/1 Pending 0 0s directvolumemigration-stunnel-transfer 0/1 Pending 0 0s directvolumemigration-stunnel-transfer 0/1 ContainerCreating 0 0s ^@directvolumemigration-stunnel-transfer 1/1 Running 0 10s directvolumemigration-rsync-transfer-mysql 0/1 Pending 0 0s directvolumemigration-rsync-transfer-mysql 0/1 Pending 0 0s directvolumemigration-rsync-transfer-mysql 0/1 ContainerCreating 0 0s directvolumemigration-rsync-transfer-mysql 1/1 Running 0 19s
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Migration Toolkit for Containers (MTC) tool image release advisory 1.4.0), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5329