Bug 2230141

Summary: [RFE] Support migration of EFS volumes
Product: Migration Toolkit for Containers Reporter: Luqman Barry <lbarry>
Component: pvc-migrateAssignee: Pranav Gaikwad <pgaikwad>
Status: NEW --- QA Contact: ssingla
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.7.10CC: dymurray, lbarry, pgaikwad, rjohnson
Target Milestone: ---   
Target Release: 1.8.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
StageBackup screen capture none

Description Luqman Barry 2023-08-08 18:59:58 UTC 
Created attachment 1982419 [details]
StageBackup screen capture

Description of problem:

During MTC Cutover migration with IndirectVolumeMigration, IndirectImageMigration. Pods on the source cluster are not quiesced. 

The StageBackup,  always fails with this error in the logs when PV/PVC is being backed up. 

```
Lchown: lchow /host_pods/some-uuid/volumes/kubernetes.io~csi/pvc-some-uid/mount/mongodb/data/db/index-xxx operation not permitted ignoring error for /mongodb/data/db/index-xxx
```

This error is repeated for many (maybe all) files.

Version-Release number of selected component (if applicable):

The source (OCP4.8) and target (OCP4.12+) clusters use AWS EFS for storage. MTC v1.7.10 is running on both clusters.

How reproducible:

It is reproducible. I just cleanup whatever was migrated (namespace, pvc,..), recreate the MigPlan and run a Cutover migration.

Steps to Reproduce:
1. Ensure source and target clusters are using AWS EFS with dynamic volume provisioning
2. Ensure MTC v1.7.10 is installed and configured on source (OCP4.8) and target (OCP4.12+) clusters
3. Create MigPlan with PVCs selected
4. Run Cutover/Stage migration with source pods in Running state

Actual results:

StageBackup sub-steps (PodVolumeBackup) fail with the error log shared above for every file MTC fails to backup.

Expected results:

PVC migration should succeed using MTC, the customer wants to be able use this for 
 batched migrations of their apps. The alternative will require static volume provisioning with AWS DataSync for data copy.

Additional info:

Here's an article briefly discussing the need for MTC to support chown.

- https://github.com/konveyor/eng-kbase/blob/master/content/posts/mtc-pv-uid-gid-prereqs.md

- https://github.com/kubernetes-sigs/aws-efs-csi-driver/issues/300