Description of problem: Sometimes a migration may fail as the stage pods are created and stay in Pending as they are unable to mount a PVC due to it being stuck in a Terminating state. Version-Release number of selected component (if applicable): MTC 1.4.0 SOURCE CLUSTER: AZURE 4.3 TARGET CLUSTER: AZURE 4.6 REPLICATION REPOSITORY: AZURE How reproducible: Always Steps to Reproduce: 1) Create a Pod that mounts a PVC 2) Create a MigPlan that references the Pod and PVC. Indirect migration. 3) Delete the PVC while it is mounted to the Pod, PVC will be in terminating 4) Run a migration with the MigPlan Actual results: Stage pod is on "ContainerCreating" status forever and the migration run is stuck. $ oc get pods NAME READY STATUS RESTARTS AGE nginx-deployment-b99766f9c-2zwfd 1/1 Running 0 36m stage-nginx-deployment-b99766f9c-2zwfd-4jc75 0/1 ContainerCreating 0 32m $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE nginx-html Terminating pvc-377f520b-4595-4c35-be3b-cc2c69853ead 1Gi RWO managed-premium 36m nginx-logs Terminating pvc-e25278b3-1cc6-45c0-a8c5-2dd066ce70b4 1Gi RWO managed-premium 36m MigMigration status: status: conditions: - category: Advisory lastTransitionTime: "2020-12-15T09:32:17Z" message: 'Step: 23/47' reason: StagePodsCreated status: "True" type: Running - category: Required lastTransitionTime: "2020-12-15T09:30:56Z" message: The migration is ready. status: "True" type: Ready - category: Required durable: true lastTransitionTime: "2020-12-15T09:31:48Z" message: The migration registries are healthy. status: "True" type: RegistriesHealthy - category: Advisory durable: true lastTransitionTime: "2020-12-15T09:32:15Z" message: '[1] Stage pods created.' status: "True" type: StagePodsCreated itinerary: Final observedDigest: a02f199bd5de68b777effef8d73e22948d8990bd9f46678bbef11823f36899fa phase: StagePodsCreated pipeline: - completed: "2020-12-15T09:31:51Z" message: Completed name: Prepare started: "2020-12-15T09:30:55Z" - completed: "2020-12-15T09:32:14Z" message: Completed name: Backup progress: - 'Backup openshift-migration/ocp-32834-pvc-terminating-mig-1608024651-s2s7w: 40 out of estimated total of 40 objects backed up (18s)' started: "2020-12-15T09:31:51Z" - message: Waiting for all Stage Pods to start. name: StageBackup phase: StagePodsCreated progress: - 'Pod ocp-32834-pvc-terminating/stage-nginx-deployment-b99766f9c-2zwfd-4jc75: Container sleep-0 ' started: "2020-12-15T09:32:14Z" - message: Not started name: StageRestore - message: Not started name: DirectImage - message: Not started name: DirectVolume - message: Not started name: Restore - message: Not started name: Cleanup startTimestamp: "2020-12-15T09:30:55Z" Expected results: The migration should fail Additional info: This is a regression of BZ https://bugzilla.redhat.com/show_bug.cgi?id=1854914
@dymurray what's the expected beahvior here? I'm not sure I understand what would be considered a "Bug" here. Dylan, can you actually confirm this is a bug here? I'm not sure what we should actually expect to happen under these circumstances.
Discussing with Dylan, it's not clear this is entirely unexpected behavior given the PVC was actually deleted. Furthermore, we'll be moving away from a stage pod approach as direct migrations become the default mode of copy transfer. We'll keep this bz around for archive sake, but will be descoping for the near future.
This link is a BZ openend by John long ago: https://bugzilla.redhat.com/show_bug.cgi?id=1854914 As a result the bug was fixed, and since then the migration fails when it finds PVCs in "Terminating" status. In 1.4.0 the fix is not there any more.
Okay it sounds like there is potentially a regression here that has to be looked into.
verified. it will report a error in the UI Danger alert:This migration has following error conditions: PVC: ocp-cakephpaaaa/mysql, deleted.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Migration Toolkit for Containers (MTC) tool image release advisory 1.4.0), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5329