Bug 1745906
| Summary: | InitialBackupFailed when migrate app with imagestream from 3.9 to 4.2 | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Zihan Tang <zitang> | ||||||
| Component: | Migration Tooling | Assignee: | Dylan Murray <dymurray> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Xin jiang <xjiang> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 4.2.0 | CC: | chezhang, jmatthew, sregidor, xjiang | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | 4.3.0 | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2020-02-06 20:20:44 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 1608404 [details]
logs
Created attachment 1608405 [details]
restic log
In 3.7 also hit this issue. This bug looks like issue https://github.com/fusor/mig-controller/issues/285 I am moving this back to ON_QA. Please retest as issue #285 has been closed and it seems like this was around the registry pods that do image migration. We have done a lot of work around that front since August I would like to verify that this bug is still valid as I have migrated imagestreams without issue. This issue is gone in the 1.1 images.
- containerID: cri-o://ebe56ed55233c1b30979c43bed0c9952fbffe790a2063ca56c0b6c5c4661048f
image: image-registry.openshift-image-registry.svc:5000/rhcam-1-1/openshift-migration-controller-rhel8@sha256:b55c0c36333656a46e1d0500cf11cc8aa06e093d312e7c54f8e1075d4ab4c6c1
imageID: image-registry.openshift-image-registry.svc:5000/rhcam-1-1/openshift-migration-controller-rhel8@sha256:b55c0c36333656a46e1d0500cf11cc8aa06e093d312e7c54f8e1075d4ab4c6c1
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2020:0440 |
Description of problem: When migrate application with imagestream, it failed with error: InitialBackupFailed storage: EBS to EBS Version-Release number of selected component (if applicable): OCP 3.X: v3.9.97 ocp4.2: 4.2.0-0.nightly-2019-08-26-235330 velero : quay.io/ocpmigrate/velero:stable quay.io/ocpmigrate/velero@sha256:957725dec5f0fb6a46dee78bd49de9ec4ab66903eabb4561b62ad8f4ad9e6f05 controller: quay.io/ocpmigrate/mig-controller:stable quay.io/ocpmigrate/mig-controller@sha256:dda76bd4de888f9bc3731a71e6ff70f83a4551386a29f02391bad5feb9cb336d quay.io/ocpmigrate/migration-plugin:stable How reproducible: always Steps to Reproduce: 1. install django by template: # oc new-project django # oc new-app --template django-psql-persistent install successfully 2. create a migplan to migrate in UI. 3. migrate directly Actual results: migrate failed: $ oc get migmigration -n openshift-migration ac7007a0-c89b-11e9-a0ed-4dd37a8e6868 -o yaml apiVersion: migration.openshift.io/v1alpha1 kind: MigMigration metadata: annotations: touch: 468147a2-be15-412c-b214-86bf9db92588 creationTimestamp: "2019-08-27T07:24:15Z" generation: 7 name: ac7007a0-c89b-11e9-a0ed-4dd37a8e6868 namespace: openshift-migration resourceVersion: "38742" selfLink: /apis/migration.openshift.io/v1alpha1/namespaces/openshift-migration/migmigrations/ac7007a0-c89b-11e9-a0ed-4dd37a8e6868 uid: ac71e49c-c89b-11e9-bb38-060e68b47d1a spec: migPlanRef: name: zitang-1 namespace: openshift-migration quiescePods: true stage: false status: conditions: - category: Advisory durable: true lastTransitionTime: "2019-08-27T07:24:51Z" message: 'The migration has failed. See: Errors.' reason: InitialBackupFailed status: "True" type: Failed errors: - 'Backup: openshift-migration-operator/ac7007a0-c89b-11e9-a0ed-4dd37a8e6868-cbjt9 partially failed.' phase: Completed startTimestamp: "2019-08-27T07:24:14Z" erros in ocp3 verelo pod: time="2019-08-27T07:24:47Z" level=info msg="1 errors encountered backup up item" backup=openshift-migration-operator/ac7007a0-c89b-11e9-a0ed-4dd37a8e6868-cbjt9 group=image.openshift.io/v1 logSource="pkg/backup/resource_backupper.go:260" name=django-psql-persistent namespace=django resource=imagestreams time="2019-08-27T07:24:47Z" level=error msg="Error backing up item" backup=openshift-migration-operator/ac7007a0-c89b-11e9-a0ed-4dd37a8e6868-cbjt9 error="error executing custom action (groupResource=imagestreams.image.openshift.io, namespace=django, name=django-psql-persistent): rpc error: code = Unknown desc = Error trying to reuse blob sha256:18f0f6de460133d70b9f6a0d1c7f2f682c324471861334b6b329a5c04653d834 at destination: pinging docker registry returned: Get http://172.30.27.60:5000/v2/: dial tcp 172.30.27.60:5000: connect: no route to host" error.file="/go/src/github.com/heptio/velero/pkg/backup/item_backupper.go:314" error.function="github.com/heptio/velero/pkg/backup.(*defaultItemBackupper).executeActions" group=image.openshift.io/v1 logSource="pkg/backup/resource_backupper.go:264" name=django-psql-persistent namespace=django resource=imagestreams Expected results: migrate successfully Additional info: $ oc get migplan -n openshift-migration zitang-1 -o yaml apiVersion: migration.openshift.io/v1alpha1 kind: MigPlan metadata: annotations: touch: 2b41e5db-d75e-40c8-a70b-0887e10ca2a5 creationTimestamp: "2019-08-27T07:20:22Z" finalizers: - openshift.io/migration generation: 14 name: zitang-1 namespace: openshift-migration resourceVersion: "42623" selfLink: /apis/migration.openshift.io/v1alpha1/namespaces/openshift-migration/migplans/zitang-1 uid: 2203fef9-c89b-11e9-bb38-060e68b47d1a spec: destMigClusterRef: name: host namespace: openshift-migration migStorageRef: name: zitang-mig namespace: openshift-migration namespaces: - django persistentVolumes: - capacity: 1Gi name: pvc-c743240f-c89a-11e9-921e-0ed5916ed6ba pvc: accessModes: - ReadWriteOnce name: postgresql namespace: django selection: action: copy storageClass: gp2 storageClass: gp2 supported: actions: - copy - move srcMigClusterRef: name: ocp39 namespace: openshift-migration status: conditions: - category: Required lastTransitionTime: "2019-08-27T07:21:06Z" message: The `persistentVolumes` list has been updated with discovered PVs. reason: Done status: "True" type: PvsDiscovered - category: Required lastTransitionTime: "2019-08-27T07:21:07Z" message: The storage resources have been created. status: "True" type: StorageEnsured - category: Required lastTransitionTime: "2019-08-27T07:21:07Z" message: The migration registry resources have been created. status: "True" type: RegistriesEnsured - category: Required lastTransitionTime: "2019-08-27T07:21:07Z" message: The migration plan is ready. status: "True"