Bug 924853

Well, the SMB share has to be configured and mounted in a way that the user qemu-kvm process is running as will have read/write access to the files there. In case you use svirt, you also need to set virt_use_samba boolean using setsebool -P virt_use_samba 1, see virtd_selinux man page.

Moreover, I believe you should use path to a filename rather than just the mount point, i.e., /path/to/cifs/mount/image.qcow or something similar.

Jiri, the SMB share is indeed mounted with global read/write permissions for everyone. All other users and applications can write to it except that of libvirt. I am not using svirt.

I did ran it with full path to the destination image before filing the bug report but it still fails.

Below shows the unsuccessful attempts :-P 

# history| grep blockcopy | grep .img
  303  2013-03-14 16:21:55 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --pivot
  307  2013-03-14 16:28:16 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --finish
  315  2013-03-14 16:32:51 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --finish
  317  2013-03-14 16:33:45 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --finish
  326  2013-03-14 16:56:03 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose
  337  2013-03-14 17:02:54 virsh blockcopy --domain co6 vda /home/backup.img --wait --finish --verbose
  460  2013-03-22 02:53:59 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  470  2013-03-22 02:56:53 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  473  2013-03-22 02:59:25 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  479  2013-03-22 03:04:04 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  481  2013-03-22 03:04:47 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  484  2013-03-22 03:09:13 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  494  2013-03-22 06:22:21 virsh blockcopy co6 vda /home/co6.img --wait --verbose --finish
  503  2013-03-22 06:46:09 virsh blockcopy co6 vda /home/co6.img --wait --verbose --finish
  504  2013-03-22 06:46:27 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  511  2013-03-22 06:48:29 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  512  2013-03-22 06:50:05 virsh blockcopy co6 vda /rbackup/co6.img --wait --verbose --finish
  520  2013-03-22 06:56:06 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  528  2013-03-22 06:58:15 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  533  2013-03-22 07:15:55 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  538  2013-03-22 07:18:26 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  548  2013-03-22 07:31:13 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish

Did these steps below, tried to reproduce this issue:

Version-Release number of selected component (if applicable):
libvirt-0.10.2-18.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.355.el6_4.2.x86_64

Setup:
1. Prepare a samba server and mount to it with cifs.
#mount.cifs //10.66.6.110/home /home -o username=root,password=redhat,rw,nounix

2. On the samba client, create a new file with some contents on the mounted directory: /home successfully.

3. Set virt_use_samba sebool
#setsebool -P virt_use_samba 1
#setenfore 1

Actions:
1. Create a guest.(guest image file is not on the cifs)
# virsh create rh7-qcow2.xml 
Domain rh7-qcow2 created from rh7-qcow2.xml

2. virsh blockcopy --verbose --wait --finish
# virsh blockcopy rh7-qcow2 /var/lib/libvirt/images/rh7-qcow2.img /home/blockcopy --verbose --wait --finish
error: internal error unable to execute QEMU command '__com.redhat_drive-mirror': Could not open '/home/blockcopy': Permission denied

Hi, Bassu

I think the error I met is different to yours, I wonder if you can show me your samba server configuration settings, and you mount.cifs command line ?

This bug was not selected to be addressed in Red Hat Enterprise Linux 6. We will look at it again within the Red Hat Enterprise Linux 7 product.

I can reproduce this on rhel6 with libvirt-0.10.2-40.el6.x86_64

# rpm -q libvirt
libvirt-0.10.2-40.el6.x86_64

1. Prepare samba server, and mount it

# mount.cifs //10.66.X.XXX/test /home/shyu -o username=root,rw,password=redhat,rw,nounix,dir_mode=0777,file_mode=0777

# mount|grep cifs
//10.66.X.XXX/test on /home/shyu type cifs (rw)

2. Prepare an transient running guest 

# virsh list --transient 
 Id    Name                           State
----------------------------------------------------
 14    rhel6                          running
# virsh domblklist rhel6 
Target     Source
------------------------------------------------
vda        /var/lib/libvirt/images/kvm-rhel6.5-x86_64-qcow2.img

3. Do blockcopy to /home/shyu/

virsh blockcopy rhel6 vda /home/shyu/copy.img   --verbose --wait 
error: internal error unable to execute QEMU command '__com.redhat_drive-mirror': Could not open '/home/shyu/copy.img': Invalid argument

I will upload the libvirtd log and guest log 

More info:

# getsebool virt_use_samba
virt_use_samba --> on
# getenforce 
Enforcing

Created attachment 918142 [details]
libvirtd log

Created attachment 918143 [details]
guest log

Hi Peter,

I can also hit this on rhel7 
# rpm -q libvirt qemu-kvm-rhev
libvirt-1.1.1-29.el7.x86_64
qemu-kvm-rhev-1.5.3-60.el7ev_0.2.x86_64

0. Prepare env as below:

# mount.cifs  //10.66.5.xxx/test /home/shyu -o username=root,rw,password=redhat,rw,nounix,dir_mode=0777,file_mode=0777

# mount|grep cifs
//10.66.5.xxx/test on /home/shyu type cifs (rw,relatime,vers=1.0,cache=strict,username=root,domain=SHYU_TEST_PC,uid=0,noforceuid,gid=0,noforcegid,addr=10.66.5.xxx,file_mode=0777,dir_mode=0777,nounix,serverino,rsize=61440,wsize=65536,actimeo=1)

# getsebool -a|grep samba
samba_create_home_dirs --> on
samba_domain_controller --> on
samba_enable_home_dirs --> on
samba_export_all_ro --> on
samba_export_all_rw --> on
samba_portmapper --> on
samba_run_unconfined --> on
samba_share_fusefs --> on
samba_share_nfs --> on
sanlock_use_samba --> on
use_samba_home_dirs --> on
virt_sandbox_use_samba --> on
virt_use_samba --> on

# getenforce 
Enforcing

# iptables -F

1. Prepare transient guest
 
# virsh list --transient
 Id    Name                           State
----------------------------------------------------
 4     test                           running

2. Do blockcopy 

# virsh blockcopy test vda /home/shyu/copy.img --verbose --wait
error: internal error: unable to execute QEMU command 'drive-mirror': Could not open file: Invalid argument

Created attachment 918284 [details]
libvirtd log

Created attachment 918285 [details]
guest log

Could you please re-try the scenario and:

1) post the audit log
2) try mounting the CIFS filesystem under a different directory (/mnt)
3) disable enforcing mode in selinux

Thanks, I'm not able to reproduce it in my setup.

(In reply to Peter Krempa from comment #17)
> Could you please re-try the scenario and:
> 
> 1) post the audit log
> 2) try mounting the CIFS filesystem under a different directory (/mnt)
> 3) disable enforcing mode in selinux
> 
> Thanks, I'm not able to reproduce it in my setup.

I also can't reproduce it after try many times. But I found an werid problem about the "disk size" of the destination file. See below:


1. Do blockcopy to an transisent geust to mount point of samba

# mount|grep cifs
//10.66.5.xxx/test on /mnt/Samba type cifs (rw,relatime,vers=1.0,cache=strict,username=root,domain=SHYU_TEST_PC,uid=0,noforceuid,gid=0,noforcegid,addr=10.66.5.xxx,file_mode=0777,dir_mode=0777,nounix,serverino,rsize=61440,wsize=65536,actimeo=1)

# time virsh blockcopy rhel6 vda /mnt/Samba/copy.img  --wait --verbose 
Block Copy: [100 %]
Now in mirroring phase

real	0m8.670s
user	0m0.026s
sys	0m0.017s

2. Check destination file size by "qemu-img" and "ls"

# ls -lash /mnt/Samba/copy.img
1.0M -rwxrwxrwx. 1 root root 1.2G Jul 21 13:20 /mnt/Samba/copy.img

# qemu-img info /mnt/Samba/copy.img
image: /mnt/Samba/copy.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 1.0M                                   ..........
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false

3. Abort blockcopy job by --abort, then recheck destination file size

# virsh blockjob rhel6 vda --abort 

# ls -lash /mnt/Samba/copy.img
2.0G -rwxrwxrwx. 1 root root 1.2G Jul 21 13:22 /mnt/Samba/copy.img

# qemu-img info /mnt/Samba/copy.img
image: /mnt/Samba/copy.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 2.0G                                  ...........
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false

Disk size change after abort blockjob. And this can't reproduce with local dir.

(In reply to Shanzhi Yu from comment #18)
> (In reply to Peter Krempa from comment #17)
> > Could you please re-try the scenario and:
> > 
> > 1) post the audit log
> > 2) try mounting the CIFS filesystem under a different directory (/mnt)
> > 3) disable enforcing mode in selinux
> > 
> > Thanks, I'm not able to reproduce it in my setup.
> 
> I also can't reproduce it after try many times. But I found an werid problem
> about the "disk size" of the destination file. See below:
> 

The main problem here is apparently that mounting the cifs share under /home is a problem as /home has some special selinux attributes and apparently doesn't work in such config. As it works when it's mounted in /mnt this should be fine.


> 
> 1. Do blockcopy to an transisent geust to mount point of samba
> 
> # mount|grep cifs
> //10.66.5.xxx/test on /mnt/Samba type cifs
> (rw,relatime,vers=1.0,cache=strict,username=root,domain=SHYU_TEST_PC,uid=0,
> noforceuid,gid=0,noforcegid,addr=10.66.5.xxx,file_mode=0777,dir_mode=0777,
> nounix,serverino,rsize=61440,wsize=65536,actimeo=1)
> 
> # time virsh blockcopy rhel6 vda /mnt/Samba/copy.img  --wait --verbose 
> Block Copy: [100 %]
> Now in mirroring phase
> 
> real	0m8.670s
> user	0m0.026s
> sys	0m0.017s
> 
> 2. Check destination file size by "qemu-img" and "ls"
> 
> # ls -lash /mnt/Samba/copy.img
> 1.0M -rwxrwxrwx. 1 root root 1.2G Jul 21 13:20 /mnt/Samba/copy.img
> 
> # qemu-img info /mnt/Samba/copy.img
> image: /mnt/Samba/copy.img
> file format: qcow2
> virtual size: 5.0G (5368709120 bytes)
> disk size: 1.0M                                   ..........
> cluster_size: 65536
> Format specific information:
>     compat: 1.1
>     lazy refcounts: false
> 
> 3. Abort blockcopy job by --abort, then recheck destination file size
> 
> # virsh blockjob rhel6 vda --abort 
> 
> # ls -lash /mnt/Samba/copy.img
> 2.0G -rwxrwxrwx. 1 root root 1.2G Jul 21 13:22 /mnt/Samba/copy.img
> 
> # qemu-img info /mnt/Samba/copy.img
> image: /mnt/Samba/copy.img
> file format: qcow2
> virtual size: 5.0G (5368709120 bytes)
> disk size: 2.0G                                  ...........
> cluster_size: 65536
> Format specific information:
>     compat: 1.1
>     lazy refcounts: false
> 
> Disk size change after abort blockjob. And this can't reproduce with local
> dir.

At this point all the work is done by qemu and thus you should file a bug against qemu.

(In reply to Peter Krempa from comment #19)
> (In reply to Shanzhi Yu from comment #18)
> > (In reply to Peter Krempa from comment #17)
> > > Could you please re-try the scenario and:

Thanks, file one bug 1121858 to qemu-kvm-rhev

Move this bug to VERIFIED according comment 18 and comment 19

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html