Bug 2001786 - Migration fails during "Stage Backup" step because volume path on host not found
Summary: Migration fails during "Stage Backup" step because volume path on host not found
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Migration Toolkit for Containers
Classification: Red Hat
Component: Controller
Version: 1.6.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 1.6.0
Assignee: Pranav Gaikwad
QA Contact: Xin jiang
Avital Pinnick
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-09-07 07:53 UTC by Prasad Joshi
Modified: 2021-09-29 14:36 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-29 14:35:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github konveyor mig-controller pull 1203 0 None None None 2021-09-10 17:48:05 UTC
Github konveyor mig-controller pull 1204 0 None None None 2021-09-10 20:41:18 UTC
Red Hat Product Errata RHSA-2021:3694 0 None None None 2021-09-29 14:36:09 UTC

Description Prasad Joshi 2021-09-07 07:53:30 UTC
Description of problem: When we execute migration it is failing at the StageBackup step. PodVolumeBackup is failed due to 'error getting volume path on host: expected one matching path, got 0'


Version-Release number of selected component (if applicable):
SOURCE CLUSTER: Azure OCP 4.6 (MTC 1.6.0)
TARGET CLUSTER: Azure OCP 4.9 (MTC 1.6.0)
REPLICATION REPOSITORY: Azure Blob Storage


How reproducible:
Always


Steps to Reproduce:
1. In source cluster create django application

$ oc new-project ocp-40299-cancelrollback
$ oc new-app django-psql-persistent

2. Go to the MTC UI and create a migration plan for this namespace.
3. Execute stage migration  
4. Execute Cutover migration



Actual results: Migration is completed with warning. Backup is partiallyFailed and PodVolumeBackup failed due to 'error getting volume path on host: expected one matching path, got 0'


Expected results: StageBackup step should be completed without any failure.


Additional info: 

oc get backup migration-614c4-stage-z2shw -n openshift-migration -o yaml

status:
  completionTimestamp: "2021-09-06T10:45:38Z"
  errors: 1
  expiration: "2021-10-06T10:45:23Z"
  formatVersion: 1.1.0
  phase: PartiallyFailed
  progress:
    itemsBackedUp: 6
    totalItems: 6
  startTimestamp: "2021-09-06T10:45:23Z"
  version: 1

oc get podvolumebackup migration-614c4-stage-z2shw-qstbx -n openshift-migration -o yaml

status:
  completionTimestamp: "2021-09-06T10:45:37Z"
  message: 'error getting volume path on host: expected one matching path, got 0'
  phase: Failed
  progress: {}
  startTimestamp: "2021-09-06T10:45:37Z"


oc get migmigration migration-614c4 -n openshift-migration -o yaml

status:
  conditions:
  - category: Warn
    durable: true
    lastTransitionTime: "2021-09-06T10:45:39Z"
    message: 'Stage Backup openshift-migration/migration-614c4-stage-z2shw: partially failed on source cluster'
    status: "True"
    type: VeleroStageBackupPartiallyFailed
  - category: Advisory
    durable: true
    lastTransitionTime: "2021-09-06T10:46:36Z"
    message: The migration has completed with warnings, please look at `Warn` conditions.
    reason: Completed
    status: "True"
    type: SucceededWithWarnings
  itinerary: Final
  observedDigest: b6379d5e70f2e9f71791cd922a3b0e6e305a78498919ce72207017d20e61278b
  phase: Completed
  pipeline:
  - completed: "2021-09-06T10:44:42Z"
    message: Completed
    name: Prepare
    started: "2021-09-06T10:44:15Z"
  - completed: "2021-09-06T10:45:05Z"
    message: Completed
    name: Backup
    progress:
    - 'Backup openshift-migration/migration-614c4-initial-ddsvp: 44 out of estimated total of 44 objects backed up (20s)'
    started: "2021-09-06T10:44:42Z"
  - completed: "2021-09-06T10:45:46Z"
    message: Completed
    name: StageBackup
    progress:
    - 'Backup openshift-migration/migration-614c4-stage-z2shw: partially failed. 6 out of estimated total of 6 objects backed up (15s)'
    - 'PodVolumeBackup openshift-migration/migration-614c4-stage-z2shw-qstbx: Failed. 0 bytes out of 0 bytes backed up (0s)'
    started: "2021-09-06T10:45:05Z"
  - completed: "2021-09-06T10:46:24Z"
    message: Completed
    name: StageRestore
    progress:
    - 'Restore openshift-migration/migration-614c4-stage-6h5dn: 4 out of estimated total of 4 objects restored (2s)'
    - stage-postgresql-fd4km is missing. Migration might fail
    started: "2021-09-06T10:45:46Z"

Comment 1 Prasad Joshi 2021-09-07 07:59:27 UTC
It failed in IDVM, IDIM

Comment 2 Xin jiang 2021-09-10 03:35:05 UTC
Indirect migration often failed due to this bug.

Comment 6 Xin jiang 2021-09-15 02:58:53 UTC
verified with mtc 1.6.0

registry.redhat.io/rhmtc/openshift-migration-controller-rhel8@sha256:3b5efa9c8197fe0313a2ab7eb184d135ba9749c9a4f0d15a6abb11c0d18b9194

Comment 8 errata-xmlrpc 2021-09-29 14:35:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Migration Toolkit for Containers (MTC) 1.6.0 security & bugfix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3694


Note You need to log in before you can comment on or make changes to this bug.