Bug 2237678

Summary: VM snapshot is failing during fsfreeze with permission denied error
Product: Container Native Virtualization (CNV) Reporter: nijin ashok <nashok>
Component: StorageAssignee: Álvaro Romero <alromero>
Status: CLOSED MIGRATED QA Contact: Natalie Gavrielov <ngavrilo>
Severity: medium Docs Contact:
Priority: high    
Version: 4.13.3CC: akalenyu, dafrank
Target Milestone: ---   
Target Release: 4.15.0   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-12-14 16:12:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description nijin ashok 2023-09-06 10:38:43 UTC
Description of problem:

If we mount a disk, by default it will have `unlabeled_t` SELinux context.

~~~
# mount /dev/sda /moun_point/

# ls -lZd /moun_point/
drwxr-xr-x. 2 root root system_u:object_r:unlabeled_t:s0 6 Sep  6 06:12 /moun_point/
~~~

Taking a snapshot of these VMs will fail during fsfreeze phase:

~~~
{"component":"virt-handler","kind":"","level":"error","msg":"Failed to freeze VMI","name":"rhel8-5px23yqqlrnm0s1w","namespace":"new-nijin-cnv","pos":"lifecycle.go:124","reason":"server error. command Freeze failed: \"LibvirtError(Code=1, Domain=10, Message='internal error: unable to execute QEMU agent command 'guest-fsfreeze-freeze': failed to open /moun_point: Permission denied')\"","timestamp":"2023-09-06T10:16:22.703074Z","uid":"b5bbd812-18a7-495a-93fa-9172dd9375f5"}
~~~


The qemu-ga won't be able to work on unlabeled_t directories:

~~~
Sep 06 06:16:33 rhel8-5px23yqqlrnm0s1w setroubleshoot[5401]: SELinux is preventing /usr/bin/qemu-ga from read access on the directory /moun_point. For >

                                                             Then you can run restorecon. The access attempt may have been stopped due to insufficient >
                                                             Do
                                                             # /sbin/restorecon -v /moun_point

                                                             If you want to allow virt to qemu ga read nonsecurity files
                                                             Then you must tell SELinux about this by enabling the 'virt_qemu_ga_read_nonsecurity_files>

                                                             Do
                                                             setsebool -P virt_qemu_ga_read_nonsecurity_files 1
~~~

As mentioned in the message, we need to either fix the context or set boolean virt_qemu_ga_read_nonsecurity_files to 1 to fix the problem.

 
Version-Release number of selected component (if applicable):

OpenShift Virtualization   4.13.3

How reproducible:

100%

Steps to Reproduce:

1. Create a RHEL 8 or 9 VM from a template.
2. Add a new disk, format it, and mount it on a directory.
3. Try to take a snapshot of the VM.
4. It will be in "progress" state for some time and will finally fail.

Actual results:

VM snapshot is failing during fsfreeze with permission denied error.

Expected results:

I am not sure where it should be fixed. The boolean virt_qemu_ga_read_nonsecurity_files was implemented in bug 1747960 and it is false by default. In the case of RHV, the snapshot does not fail even if fsfreeze fails, and the snapshot operation completes with a warning. In the case of OpenShift Virtualization, the user is clueless about the failure and has to look at the logs to understand the problem. I think it would be helpful if we could pick up the error and show the recommendation to the user to correct the context.      


Additional info:

Comment 1 Kedar Bidarkar 2023-09-06 12:40:20 UTC
It appears, this bug is related to the storage component. Please revert back to Virt component if you feel otherwise.

Comment 2 Alex Kalenyuk 2023-11-16 15:03:27 UTC
Isn't this a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2223411?
@dafrank

Comment 3 Red Hat Bugzilla 2024-05-17 04:25:04 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days