Bug 810241 - Save the guest to pre-created file on root_squashing export nfs with dynamic_ownership=1 Permission denied
Save the guest to pre-created file on root_squashing export nfs with dynamic_...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.3
Unspecified Unspecified
high Severity high
: rc
: ---
Assigned To: Michal Privoznik
: Regression
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-04-05 08:57 EDT by EricLee
Modified: 2012-06-20 02:52 EDT (History)
10 users (show)

See Also:
Fixed In Version: libvirt-0.9.10-16.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-20 02:52:47 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
libvirtd.log (21.95 KB, text/x-log)
2012-04-05 08:57 EDT, EricLee
no flags Details

  None (edit)
Description EricLee 2012-04-05 08:57:14 EDT
Created attachment 575401 [details]
libvirtd.log

Description of problem:
Save the guest to pre-created file on root_squashing export nfs with dynamic_ownership=1 Permission denied

Version-Release number of selected component (if applicable):
qemu-kvm-0.12.1.2-2.269.el6.x86_64
libvirt-0.9.10-10.el6.x86_64
kernel-2.6.32-251.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. set the dynamic_ownership = 1 in /etc/libvirt/qemu.conf
dynamic_ownership = 1
also edit the user and group as qemu
user = "qemu"
group = "qemu"

2. add user vdsm:
# useradd vdsm -g qemu -u 36

3. setup a nfs server
# cat /etc/exports
/var/lib/libvirt/images      *(rw,root_squash)

# service nfs restart

4. created a file with ownership 36:36 in nfs export dir
# touch /var/lib/libvirt/images/save
# chown 36:36 /var/lib/libvirt/images/save
# chmod 664 /var/lib/libvirt/images/save

# ll /var/lib/libvirt/
drwxr-xr-x. 3 root root 4096 Apr  5 17:02 images

5. mount the nfs export on the local host
# mount -o vers=3 $nfs_server_ip:/var/lib/libvirt/images /mnt/dir

6. # ll /mnt/dir/save 
-rw-rw-r--. 1 vdsm kvm 0 Apr  5 19:50 /mnt/dir/save

7. # ll /mnt
drwxr-xr-x.  3 root root  4096 Apr  5 17:02 /mnt/dir

8. save a running domain to the pre-created file
(conduct some operations in guest, such as: ls, ps aux...)
# virsh save qcow2_old /mnt/dir/save

Actual results:
Failed with error:
error: Failed to save domain raw to /mnt/dir/save
error: Error from child process creating '/mnt/dir/save': Permission denied

Expected results:
Save successfully

Additional info:
I set the Keywords to "Regression" cause using libvirt-0.9.4-23.el6.x86_64.rpm and qemu-kvm-0.12.1.2-2.209.el6.x86_64.rpm save the guest successfully.
And
when I save the guest to local dir using the newer versions:
# chown qemu:qemu /data/images/qcow2.img

# virsh start qcow2_old
Domain qcow2_old started

# ll /data/images/
-rw-rw-rw-. 1 qemu qemu  3086876672 Apr  5 20:21 qcow2.img

# virsh save qcow2_old /mnt/tmp/qcow2_old.save 
Domain qcow2_old saved to /mnt/tmp/qcow2_old.save

# ll /data/images/ -Z
-rw-rw-rw-. root root system_u:object_r:default_t:s0   qcow2.img
that means save has change the img's own to root:root

# ll /mnt/tmp/ -Z
-rw-------. root root system_u:object_r:virt_content_t:s0 qcow2_old.save
Comment 3 Dave Allan 2012-04-09 11:17:34 EDT
Michal, what's going on with this BZ?
Comment 4 Michal Privoznik 2012-04-10 13:45:08 EDT
I don't think I see what's broken here. I mean, if you set dynamic_ownership=1, you tell libvirt to dynamically change ownership of touched files for you. However, there are some cases where one simply can't change ownership, so libvirt throws an error. You've set user and group to 'qemu'. Therefore libvirt is trying to chown /mnt/dir/save to qemu:qemu. However, this is not possible on NFS. Neither as root:root neither as any user (from machines other than the one actually exporting NFS):

# sudo -g qemu -u qemu chown root:root /mnt/dir/save 
chown: changing ownership of `/mnt/dir/save': Operation not permitted

# chown root:root /mnt/dir/save 
chown: changing ownership of `/mnt/dir/save': Operation not permitted

where /mnt/dir is a NFS:

# mount | grep /mnt/dir
bart:/nfs_share on /mnt/dir type nfs (rw,vers=3,addr=10.34.27.62)

However, if we would not use root_squash the problem would go away because client (=libvirt) would be allowed to perform operations as root on the server instead of remapping to nobody:nobody.

Hence, Eric is it okay with you to close this as NOTABUG?
Comment 16 EricLee 2012-05-03 09:54:24 EDT
with new package: libvirt-0.9.10-16.el6

1.
dynamic_ownership = 0
#cat /ext/exports
/data/images/	*(rw,no_root_squash)

# ll /mnt/755nfs/save 
-rwxrwxr--. 1 vdsm kvm 0 Apr  1 16:54 /mnt/755nfs/save

# virsh save qcow2 /mnt/755nfs/save 

Domain qcow2 saved to /mnt/755nfs/save

# ll /mnt/755nfs/save 
-rwxrwxr--. 1 vdsm kvm 392542212 May  3 17:46 /mnt/755nfs/save

# virsh restore /mnt/755nfs/save 
Domain restored from /mnt/755nfs/save

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 4     qcow2                          running

And for that, there is no qcow2.save file in /mnt/755nfs:

# virsh save qcow2 /mnt/755nfs/qcow2.save

Domain qcow2 saved to /mnt/755nfs/qcow2.save

# virsh restore /mnt/755nfs/qcow2.save 
Domain restored from /mnt/755nfs/qcow2.save

2.
dynamic_ownership = 1(and user=root group=root as default in qemu.conf)
#cat /ext/exports
/data/images/	*(rw,no_root_squash)

# ll /mnt/755nfs/save 
-rwxrwxr--. 1 vdsm kvm 0 May  3 17:54 /mnt/755nfs/save

# virsh save qcow2 /mnt/755nfs/save 

Domain qcow2 saved to /mnt/755nfs/save

# ll /mnt/755nfs/save 
-rwxrwxr--. 1 root root 396600981 May  3 17:58 /mnt/755nfs/save

# virsh restore /mnt/755nfs/save 
Domain restored from /mnt/755nfs/save

And for there is no qcow2.save file in /mnt/755nfs:

# virsh save qcow2 /mnt/755nfs/qcow2.save

Domain qcow2 saved to /mnt/755nfs/qcow2.save

# virsh restore /mnt/755nfs/qcow2.save
Domain restored from /mnt/755nfs/qcow2.save

3.
dynamic_ownership = 1
#cat /ext/exports
/data/images/	*(rw,root_squash)

ll /mnt/755nfs/save
-rwxrwxr--. 1 vdsm kvm            0 May  3 18:12 save

# virsh save qcow2 /mnt/755nfs/save 
error: Failed to save domain qcow2 to /mnt/755nfs/save
error: Error from child process creating '/mnt/755nfs/save': Permission denied

And for no qcow2.save file in /mnt/755nfs:

# virsh save qcow2 /mnt/755nfs/qcow2.save 
error: Failed to save domain qcow2 to /mnt/755nfs/qcow2.save
error: Error from child process creating '/mnt/755nfs/qcow2.save': Permission denied

and save to local dir:
# ll /data/images/save 
-rwxrwxr--. 1 vdsm kvm 0 May  3 18:22 /data/images/save

# virsh save qcow2 /data/images/save

Domain qcow2 saved to /data/images/save

# ll /data/images/save 
-rwxrwxr--. 1 root root 396697701 May  3 18:23 /data/images/save

And for no qcow2.save file in /data/images:
# virsh save qcow2 /data/images/qcow2.save

Domain qcow2 saved to /data/images/qcow2.save

# ll /data/images/qcow2.save 
-rw-------. 1 root root 396681541 May  3 18:21 /data/images/qcow2.save

4.
on nfs server:
dynamic_ownership = 0
#cat /ext/exports
/data/images/	*(rw,root_squash)
# chown qemu:qemu /data/images/
# ll /data/images/ -d
drwxr-xr-x. 2 qemu qemu 4096 May  3 19:33 /data/images/
# touch /data/images/save
# chown 36:36 /data/images/bbc
# chmod 664 /data/images/bbc

on test machine:
# mount -o vers=3 10.66.4.232:/data/images /mnt/755nfs
# useradd vdsm -g kvm -u 36

# ll /mnt/755nfs/bbc
-rw-rw-r--. 1 vdsm kvm 0 May  3 19:46 /mnt/755nfs/bbc

# virsh save qcow2 /mnt/755nfs/bbc

Domain qcow2 saved to /mnt/755nfs/bbc

# virsh restore /mnt/755nfs/bbc
Domain restored from /mnt/755nfs/bbc

# ll /mnt/755nfs/bbc
-rw-rw-r--. 1 vdsm kvm 570963273 May  3 19:47 /mnt/755nfs/bbc

libvirt do not try to chown of bbc, then save successfully, it is the expected result.

Is this enough to verify this bug?
Comment 17 Michal Privoznik 2012-05-07 03:46:09 EDT
Yes. We can see how dynamic_ownership forces or not libvirt to chown() a file on a NFS. Thanks for verifying.
Comment 18 EricLee 2012-05-07 22:08:07 EDT
According to Comment 16 & 17, set the status to VERIFIED.
Comment 23 errata-xmlrpc 2012-06-20 02:52:47 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0748.html

Note You need to log in before you can comment on or make changes to this bug.