Bug 2299628

Summary: Migration stuck as DirectVolumeMigration fails with "InvalidPVCs" error
Product: Migration Toolkit for Containers Reporter: ssingla
Component: GeneralAssignee: John Matthews <jmatthew>
Status: CLOSED ERRATA QA Contact: ssingla
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 1.8.4CC: rjohnson
Target Milestone: ---Keywords: Regression
Target Release: 1.8.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-09-26 03:47:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description ssingla 2024-07-24 04:15:08 UTC
Description of problem:
When we perform a migration, it is getting stuck at the DVM step, and finally fails. 

Version-Release number of selected component (if applicable):
MTC 1.8.4

How reproducible:
Always

Steps to Reproduce:
1.Deploy an application without PVCs
2.Execute cutover migration

Actual results:

Migration stuck at the DVM phase, DVM is having a critical condition -> InvalidPVCs

Expected results:

Migration should be completed successfully.

Additional info:

$ oc get dvm -n openshift-migration ocp-24686-project-mig-1721792717-4vfzv -o yaml
apiVersion: migration.openshift.io/v1alpha1
kind: DirectVolumeMigration
metadata:
  creationTimestamp: "2024-07-24T03:45:53Z"
  generateName: ocp-24686-project-mig-1721792717-
  generation: 2
  labels:
    app.kubernetes.io/part-of: openshift-migration
    migmigration: 108654a4-6601-4980-8d03-492a43170afe
    migration-direct-volume: 108654a4-6601-4980-8d03-492a43170afe
  name: ocp-24686-project-mig-1721792717-4vfzv
  namespace: openshift-migration
  ownerReferences:
  - apiVersion: migration.openshift.io/v1alpha1
    controller: true
    kind: MigMigration
    name: ocp-24686-project-mig-1721792717
    uid: 108654a4-6601-4980-8d03-492a43170afe
  resourceVersion: "50435"
  uid: 4bb9015c-ab9e-42fa-bbcd-1faa7cfa45a2
spec:
  createDestinationNamespaces: true
  destMigClusterRef:
    name: host
    namespace: openshift-migration
  srcMigClusterRef:
    name: source-cluster
    namespace: openshift-migration
status:
  conditions:
  - category: Critical
    lastTransitionTime: "2024-07-24T03:45:53Z"
    message: The set of persistent volume claims is invalid
    reason: NotSet
    status: "True"
    type: InvalidPVCs
  observedDigest: 81765c55821b2e9bf40284e8babc73a22f5469087c91247cec45fa0f3d7a5692
  phaseDescription: ""


-------------------------------------------------

$ oc get migmigration -n openshift-migration ocp-24686-project-mig-1721792717 -o yaml
apiVersion: migration.openshift.io/v1alpha1
kind: MigMigration
metadata:
  annotations:
    openshift.io/touch: e3bbcc5d-4972-11ef-9c40-0a580a820214
  creationTimestamp: "2024-07-24T03:45:41Z"
  generation: 40
  labels:
    controller-tools.k8s.io: "1.0"
    migration.openshift.io/migplan-name: ocp-24686-project-migplan-1721792717
    migration.openshift.io/migration-uid: 108654a4-6601-4980-8d03-492a43170afe
  name: ocp-24686-project-mig-1721792717
  namespace: openshift-migration
  ownerReferences:
  - apiVersion: migration.openshift.io/v1alpha1
    kind: MigPlan
    name: ocp-24686-project-migplan-1721792717
    uid: fe8056b4-21e6-47f7-8609-a7d2dc5acc23
  resourceVersion: "62072"
  uid: 108654a4-6601-4980-8d03-492a43170afe
spec:
  migPlanRef:
    name: ocp-24686-project-migplan-1721792717
    namespace: openshift-migration
  stage: false
status:
  conditions:
  - category: Advisory
    lastTransitionTime: "2024-07-24T03:45:53Z"
    message: 'Step: 38/49'
    reason: WaitForDirectVolumeMigrationToComplete
    status: "True"
    type: Running
  - category: Required
    lastTransitionTime: "2024-07-24T03:45:41Z"
    message: The migration is ready.
    status: "True"
    type: Ready
  itinerary: Final
  observedDigest: 56082d54a76ee13d45e1a3981ceb3f88b0beba45d0c447f7dec4aba8a7f99260
  phase: WaitForDirectVolumeMigrationToComplete
  pipeline:
  - completed: "2024-07-24T03:45:46Z"
    message: Completed
    name: Prepare
    started: "2024-07-24T03:45:41Z"
  - completed: "2024-07-24T03:45:52Z"
    message: Completed
    name: Backup
    progress:
    - 'Backup openshift-migration/ocp-24686-project-mig-1721792717-initial-wxdkl:
      15 out of estimated total of 15 objects backed up (4s)'
    started: "2024-07-24T03:45:46Z"
  - completed: "2024-07-24T03:45:53Z"
    message: Completed
    name: StageBackup
    started: "2024-07-24T03:45:52Z"
  - completed: "2024-07-24T03:45:53Z"
    message: Waiting for Direct Image Migration to complete.
    name: DirectImage
    phase: WaitForDirectImageMigrationToComplete
    progress:
    - 0 total ImageStreams; 0 running; 0 successful; 0 failed
    started: "2024-07-24T03:45:53Z"
  - name: DirectVolume
    phase: WaitForDirectVolumeMigrationToComplete
    progress:
    - 0 total volumes; 0 successful; 0 running; 0 failed
    started: "2024-07-24T03:45:53Z"
  - message: Not started
    name: Restore
  - message: Not started
    name: Cleanup
  startTimestamp: "2024-07-24T03:45:41Z"

Comment 10 errata-xmlrpc 2024-09-26 03:47:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Migration Toolkit for Containers (MTC) 1.8.4 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:7164