Red Hat Bugzilla – Bug 810241
Save the guest to pre-created file on root_squashing export nfs with dynamic_ownership=1 Permission denied
Last modified: 2012-06-20 02:52:47 EDT
Created attachment 575401 [details] libvirtd.log Description of problem: Save the guest to pre-created file on root_squashing export nfs with dynamic_ownership=1 Permission denied Version-Release number of selected component (if applicable): qemu-kvm-0.12.1.2-2.269.el6.x86_64 libvirt-0.9.10-10.el6.x86_64 kernel-2.6.32-251.el6.x86_64 How reproducible: 100% Steps to Reproduce: 1. set the dynamic_ownership = 1 in /etc/libvirt/qemu.conf dynamic_ownership = 1 also edit the user and group as qemu user = "qemu" group = "qemu" 2. add user vdsm: # useradd vdsm -g qemu -u 36 3. setup a nfs server # cat /etc/exports /var/lib/libvirt/images *(rw,root_squash) # service nfs restart 4. created a file with ownership 36:36 in nfs export dir # touch /var/lib/libvirt/images/save # chown 36:36 /var/lib/libvirt/images/save # chmod 664 /var/lib/libvirt/images/save # ll /var/lib/libvirt/ drwxr-xr-x. 3 root root 4096 Apr 5 17:02 images 5. mount the nfs export on the local host # mount -o vers=3 $nfs_server_ip:/var/lib/libvirt/images /mnt/dir 6. # ll /mnt/dir/save -rw-rw-r--. 1 vdsm kvm 0 Apr 5 19:50 /mnt/dir/save 7. # ll /mnt drwxr-xr-x. 3 root root 4096 Apr 5 17:02 /mnt/dir 8. save a running domain to the pre-created file (conduct some operations in guest, such as: ls, ps aux...) # virsh save qcow2_old /mnt/dir/save Actual results: Failed with error: error: Failed to save domain raw to /mnt/dir/save error: Error from child process creating '/mnt/dir/save': Permission denied Expected results: Save successfully Additional info: I set the Keywords to "Regression" cause using libvirt-0.9.4-23.el6.x86_64.rpm and qemu-kvm-0.12.1.2-2.209.el6.x86_64.rpm save the guest successfully. And when I save the guest to local dir using the newer versions: # chown qemu:qemu /data/images/qcow2.img # virsh start qcow2_old Domain qcow2_old started # ll /data/images/ -rw-rw-rw-. 1 qemu qemu 3086876672 Apr 5 20:21 qcow2.img # virsh save qcow2_old /mnt/tmp/qcow2_old.save Domain qcow2_old saved to /mnt/tmp/qcow2_old.save # ll /data/images/ -Z -rw-rw-rw-. root root system_u:object_r:default_t:s0 qcow2.img that means save has change the img's own to root:root # ll /mnt/tmp/ -Z -rw-------. root root system_u:object_r:virt_content_t:s0 qcow2_old.save
Michal, what's going on with this BZ?
I don't think I see what's broken here. I mean, if you set dynamic_ownership=1, you tell libvirt to dynamically change ownership of touched files for you. However, there are some cases where one simply can't change ownership, so libvirt throws an error. You've set user and group to 'qemu'. Therefore libvirt is trying to chown /mnt/dir/save to qemu:qemu. However, this is not possible on NFS. Neither as root:root neither as any user (from machines other than the one actually exporting NFS): # sudo -g qemu -u qemu chown root:root /mnt/dir/save chown: changing ownership of `/mnt/dir/save': Operation not permitted # chown root:root /mnt/dir/save chown: changing ownership of `/mnt/dir/save': Operation not permitted where /mnt/dir is a NFS: # mount | grep /mnt/dir bart:/nfs_share on /mnt/dir type nfs (rw,vers=3,addr=10.34.27.62) However, if we would not use root_squash the problem would go away because client (=libvirt) would be allowed to perform operations as root on the server instead of remapping to nobody:nobody. Hence, Eric is it okay with you to close this as NOTABUG?
Moving to POST: http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-April/msg00841.html
with new package: libvirt-0.9.10-16.el6 1. dynamic_ownership = 0 #cat /ext/exports /data/images/ *(rw,no_root_squash) # ll /mnt/755nfs/save -rwxrwxr--. 1 vdsm kvm 0 Apr 1 16:54 /mnt/755nfs/save # virsh save qcow2 /mnt/755nfs/save Domain qcow2 saved to /mnt/755nfs/save # ll /mnt/755nfs/save -rwxrwxr--. 1 vdsm kvm 392542212 May 3 17:46 /mnt/755nfs/save # virsh restore /mnt/755nfs/save Domain restored from /mnt/755nfs/save # virsh list --all Id Name State ---------------------------------------------------- 4 qcow2 running And for that, there is no qcow2.save file in /mnt/755nfs: # virsh save qcow2 /mnt/755nfs/qcow2.save Domain qcow2 saved to /mnt/755nfs/qcow2.save # virsh restore /mnt/755nfs/qcow2.save Domain restored from /mnt/755nfs/qcow2.save 2. dynamic_ownership = 1(and user=root group=root as default in qemu.conf) #cat /ext/exports /data/images/ *(rw,no_root_squash) # ll /mnt/755nfs/save -rwxrwxr--. 1 vdsm kvm 0 May 3 17:54 /mnt/755nfs/save # virsh save qcow2 /mnt/755nfs/save Domain qcow2 saved to /mnt/755nfs/save # ll /mnt/755nfs/save -rwxrwxr--. 1 root root 396600981 May 3 17:58 /mnt/755nfs/save # virsh restore /mnt/755nfs/save Domain restored from /mnt/755nfs/save And for there is no qcow2.save file in /mnt/755nfs: # virsh save qcow2 /mnt/755nfs/qcow2.save Domain qcow2 saved to /mnt/755nfs/qcow2.save # virsh restore /mnt/755nfs/qcow2.save Domain restored from /mnt/755nfs/qcow2.save 3. dynamic_ownership = 1 #cat /ext/exports /data/images/ *(rw,root_squash) ll /mnt/755nfs/save -rwxrwxr--. 1 vdsm kvm 0 May 3 18:12 save # virsh save qcow2 /mnt/755nfs/save error: Failed to save domain qcow2 to /mnt/755nfs/save error: Error from child process creating '/mnt/755nfs/save': Permission denied And for no qcow2.save file in /mnt/755nfs: # virsh save qcow2 /mnt/755nfs/qcow2.save error: Failed to save domain qcow2 to /mnt/755nfs/qcow2.save error: Error from child process creating '/mnt/755nfs/qcow2.save': Permission denied and save to local dir: # ll /data/images/save -rwxrwxr--. 1 vdsm kvm 0 May 3 18:22 /data/images/save # virsh save qcow2 /data/images/save Domain qcow2 saved to /data/images/save # ll /data/images/save -rwxrwxr--. 1 root root 396697701 May 3 18:23 /data/images/save And for no qcow2.save file in /data/images: # virsh save qcow2 /data/images/qcow2.save Domain qcow2 saved to /data/images/qcow2.save # ll /data/images/qcow2.save -rw-------. 1 root root 396681541 May 3 18:21 /data/images/qcow2.save 4. on nfs server: dynamic_ownership = 0 #cat /ext/exports /data/images/ *(rw,root_squash) # chown qemu:qemu /data/images/ # ll /data/images/ -d drwxr-xr-x. 2 qemu qemu 4096 May 3 19:33 /data/images/ # touch /data/images/save # chown 36:36 /data/images/bbc # chmod 664 /data/images/bbc on test machine: # mount -o vers=3 10.66.4.232:/data/images /mnt/755nfs # useradd vdsm -g kvm -u 36 # ll /mnt/755nfs/bbc -rw-rw-r--. 1 vdsm kvm 0 May 3 19:46 /mnt/755nfs/bbc # virsh save qcow2 /mnt/755nfs/bbc Domain qcow2 saved to /mnt/755nfs/bbc # virsh restore /mnt/755nfs/bbc Domain restored from /mnt/755nfs/bbc # ll /mnt/755nfs/bbc -rw-rw-r--. 1 vdsm kvm 570963273 May 3 19:47 /mnt/755nfs/bbc libvirt do not try to chown of bbc, then save successfully, it is the expected result. Is this enough to verify this bug?
Yes. We can see how dynamic_ownership forces or not libvirt to chown() a file on a NFS. Thanks for verifying.
According to Comment 16 & 17, set the status to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-0748.html