Bug 2184386

Summary: EPERM during tar extraction while cloning a PVC
Product: Container Native Virtualization (CNV) Reporter: nijin ashok <nashok>
Component: StorageAssignee: Michael Henriksen <mhenriks>
Status: CLOSED ERRATA QA Contact: Kevin Alon Goldblatt <kgoldbla>
Severity: high Docs Contact:
Priority: high    
Version: 4.11.3CC: jpeimer, mhenriks, ngavrilo, yadu
Target Milestone: ---   
Target Release: 4.11.5   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: CNV v4.11.4-115 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-07-25 15:00:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description nijin ashok 2023-04-04 13:50:59 UTC
Description of problem:

In 4.11, the CDI source pod will send the entire directory using tar (tar -c . ). This also sends the current directory '.'. Then the cdi-upload-pod extracts it using --preserve-permissions and when it extracts the current directory '.', it will try to set the permission on the parent directory. 

A simple example of the above:

~~~
[root@dell-r530-40 new]# tar -cvf /tmp/test.tar .
./
./test
./test1

[root@dell-r530-40 new]# chown test.test /root/new/

[root@dell-r530-40 new]# ls -ld /root/new/
drwxr-xr-x. 2 test test 31 Apr  1 08:35 /root/new/

[root@dell-r530-40 new]# tar -xv --preserve-permissions  -f /tmp/test.tar
./
./test
./test1

[root@dell-r530-40 new]# ls -ld /root/new/
drwxr-xr-x. 2 root root 31 Apr  1 08:35 /root/new/
~~~


Some storage doesn't seem to like this and it is failing. We don't have tar STDOUT logged in 4.11, but I took the strace of tar extraction and can see this:

~~~
1488141 11:14:58.442184 fchownat(3, ".", 0, 0, 0) = -1 EPERM (Operation not permitted)
1488141 11:14:58.442584 write(2, "/usr/bin/tar: ", 14) = 14
1488141 11:14:58.442637 write(2, ".: Cannot change ownership to ui"..., 42) = 42
1488141 11:14:58.442676 write(2, ": Operation not permitted", 25) = 25
1488141 11:14:58.442720 write(2, "\n", 1) = 1
~~~

The customer is using the Trident storage class.

Note that this is changed in 4.12, where it is reading and listing the individual files in the /data/ and sending it using tar.


https://github.com/kubevirt/containerized-data-importer/pull/2410/files#diff-ee284ba07c3d9a93a120236d65096dd843a2f979681004f33a84072e176a15f7L154

The request is to backport the whole fix or part of it so that tar extraction will not fail.
  

Version-Release number of selected component (if applicable):

OpenShift Virtualization   4.11.3

How reproducible:

100% in the customer's environment

Steps to Reproduce:

1. Clone the PVC by creating a DV with the source as PVC.
2. The cloning fails with below error:

~~~
2023-03-21T12:44:26.290384642Z I0321 12:44:26.290318       1 util.go:197] begin untar to /data...
2023-03-21T12:46:58.613756375Z E0321 12:46:58.613691       1 util.go:217] exit status 2
~~~

Tar strace shows EPERM error while it tries to set ownership on the current '.' directory.

Actual results:

EPERM during tar extraction while cloning a PVC.

Expected results:

Cloning should work.

Additional info:

Comment 1 Yan Du 2023-04-12 12:18:59 UTC
Michael, could we backport the changes to 4.11 regarding the tar extraction?

Comment 13 errata-xmlrpc 2023-07-25 15:00:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Virtualization 4.11.5 Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4271