Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 924853 - blockcopy to cifs fails
blockcopy to cifs fails
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt (Show other bugs)
7.0
x86_64 Linux
unspecified Severity high
: rc
: ---
Assigned To: Peter Krempa
Virtualization Bugs
: TestOnly
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-03-22 12:10 EDT by Bassu
Modified: 2016-04-26 10:36 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-03-05 02:20:08 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
libvirtd log (152.19 KB, text/plain)
2014-07-15 07:05 EDT, Shanzhi Yu
no flags Details
guest log (9.14 KB, text/plain)
2014-07-15 07:06 EDT, Shanzhi Yu
no flags Details
libvirtd log (162.49 KB, text/plain)
2014-07-15 22:22 EDT, Shanzhi Yu
no flags Details
guest log (4.80 KB, text/plain)
2014-07-15 22:22 EDT, Shanzhi Yu
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0323 normal SHIPPED_LIVE Low: libvirt security, bug fix, and enhancement update 2015-03-05 07:10:54 EST

  None (edit)
Description Bassu 2013-03-22 12:10:16 EDT
Description of problem:
Blockcopy through virsh on an SMB mounted destination fails

Version-Release number of selected component (if applicable):
libvirt-0.10.2-18.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.355.el6.2.x86_6

How reproducible:
- mount an smb share through mount.cifs (nounix,dir_mode,file_mode required)
- issue "virsh blockcopy dom vda /path/to/cifs/mount --verbose --wait --finish"

Actual results:
error: internal error unable to execute QEMU command '__com.redhat_drive-mirror': Could not open '/path/to/cifs/mount': Invalid argument

Expected results:
Successfully copied 

Additional info:
As virsh writes the image as qemu user so mount options nounix,dir_mode=777,file_mode=777 might be required.
Writing to the share works fine from other apps/users but not through virsh.
Comment 1 Jiri Denemark 2013-04-09 07:05:35 EDT
Well, the SMB share has to be configured and mounted in a way that the user qemu-kvm process is running as will have read/write access to the files there. In case you use svirt, you also need to set virt_use_samba boolean using setsebool -P virt_use_samba 1, see virtd_selinux man page.

Moreover, I believe you should use path to a filename rather than just the mount point, i.e., /path/to/cifs/mount/image.qcow or something similar.
Comment 2 Bassu 2013-04-09 08:13:52 EDT
Jiri, the SMB share is indeed mounted with global read/write permissions for everyone. All other users and applications can write to it except that of libvirt. I am not using svirt.

I did ran it with full path to the destination image before filing the bug report but it still fails.

Below shows the unsuccessful attempts :-P 

# history| grep blockcopy | grep .img
  303  2013-03-14 16:21:55 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --pivot
  307  2013-03-14 16:28:16 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --finish
  315  2013-03-14 16:32:51 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --finish
  317  2013-03-14 16:33:45 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose --finish
  326  2013-03-14 16:56:03 virsh blockcopy co6 /home/vms/co6.img /home/backup.img --wait --verbose
  337  2013-03-14 17:02:54 virsh blockcopy --domain co6 vda /home/backup.img --wait --finish --verbose
  460  2013-03-22 02:53:59 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  470  2013-03-22 02:56:53 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  473  2013-03-22 02:59:25 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  479  2013-03-22 03:04:04 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  481  2013-03-22 03:04:47 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  484  2013-03-22 03:09:13 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  494  2013-03-22 06:22:21 virsh blockcopy co6 vda /home/co6.img --wait --verbose --finish
  503  2013-03-22 06:46:09 virsh blockcopy co6 vda /home/co6.img --wait --verbose --finish
  504  2013-03-22 06:46:27 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  511  2013-03-22 06:48:29 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  512  2013-03-22 06:50:05 virsh blockcopy co6 vda /rbackup/co6.img --wait --verbose --finish
  520  2013-03-22 06:56:06 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  528  2013-03-22 06:58:15 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  533  2013-03-22 07:15:55 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  538  2013-03-22 07:18:26 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
  548  2013-03-22 07:31:13 virsh blockcopy co6 vda /rbackup/newton/co6.img --wait --verbose --finish
Comment 4 chhu 2013-05-16 08:53:31 EDT
Did these steps below, tried to reproduce this issue:

Version-Release number of selected component (if applicable):
libvirt-0.10.2-18.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.355.el6_4.2.x86_64

Setup:
1. Prepare a samba server and mount to it with cifs.
#mount.cifs //10.66.6.110/home /home -o username=root,password=redhat,rw,nounix

2. On the samba client, create a new file with some contents on the mounted directory: /home successfully.

3. Set virt_use_samba sebool
#setsebool -P virt_use_samba 1
#setenfore 1

Actions:
1. Create a guest.(guest image file is not on the cifs)
# virsh create rh7-qcow2.xml 
Domain rh7-qcow2 created from rh7-qcow2.xml

2. virsh blockcopy --verbose --wait --finish
# virsh blockcopy rh7-qcow2 /var/lib/libvirt/images/rh7-qcow2.img /home/blockcopy --verbose --wait --finish
error: internal error unable to execute QEMU command '__com.redhat_drive-mirror': Could not open '/home/blockcopy': Permission denied
Comment 5 chhu 2013-06-05 04:31:05 EDT
Hi, Bassu

I think the error I met is different to yours, I wonder if you can show me your samba server configuration settings, and you mount.cifs command line ?
Comment 8 Jiri Denemark 2014-04-04 17:37:42 EDT
This bug was not selected to be addressed in Red Hat Enterprise Linux 6. We will look at it again within the Red Hat Enterprise Linux 7 product.
Comment 10 Shanzhi Yu 2014-07-15 07:02:01 EDT
I can reproduce this on rhel6 with libvirt-0.10.2-40.el6.x86_64

# rpm -q libvirt
libvirt-0.10.2-40.el6.x86_64

1. Prepare samba server, and mount it

# mount.cifs //10.66.X.XXX/test /home/shyu -o username=root,rw,password=redhat,rw,nounix,dir_mode=0777,file_mode=0777

# mount|grep cifs
//10.66.X.XXX/test on /home/shyu type cifs (rw)

2. Prepare an transient running guest 

# virsh list --transient 
 Id    Name                           State
----------------------------------------------------
 14    rhel6                          running
# virsh domblklist rhel6 
Target     Source
------------------------------------------------
vda        /var/lib/libvirt/images/kvm-rhel6.5-x86_64-qcow2.img

3. Do blockcopy to /home/shyu/

virsh blockcopy rhel6 vda /home/shyu/copy.img   --verbose --wait 
error: internal error unable to execute QEMU command '__com.redhat_drive-mirror': Could not open '/home/shyu/copy.img': Invalid argument

I will upload the libvirtd log and guest log 

More info:

# getsebool virt_use_samba
virt_use_samba --> on
# getenforce 
Enforcing
Comment 11 Shanzhi Yu 2014-07-15 07:05:52 EDT
Created attachment 918142 [details]
libvirtd log
Comment 12 Shanzhi Yu 2014-07-15 07:06:24 EDT
Created attachment 918143 [details]
guest log
Comment 14 Shanzhi Yu 2014-07-15 22:19:04 EDT
Hi Peter,

I can also hit this on rhel7 
# rpm -q libvirt qemu-kvm-rhev
libvirt-1.1.1-29.el7.x86_64
qemu-kvm-rhev-1.5.3-60.el7ev_0.2.x86_64

0. Prepare env as below:

# mount.cifs  //10.66.5.xxx/test /home/shyu -o username=root,rw,password=redhat,rw,nounix,dir_mode=0777,file_mode=0777

# mount|grep cifs
//10.66.5.xxx/test on /home/shyu type cifs (rw,relatime,vers=1.0,cache=strict,username=root,domain=SHYU_TEST_PC,uid=0,noforceuid,gid=0,noforcegid,addr=10.66.5.xxx,file_mode=0777,dir_mode=0777,nounix,serverino,rsize=61440,wsize=65536,actimeo=1)

# getsebool -a|grep samba
samba_create_home_dirs --> on
samba_domain_controller --> on
samba_enable_home_dirs --> on
samba_export_all_ro --> on
samba_export_all_rw --> on
samba_portmapper --> on
samba_run_unconfined --> on
samba_share_fusefs --> on
samba_share_nfs --> on
sanlock_use_samba --> on
use_samba_home_dirs --> on
virt_sandbox_use_samba --> on
virt_use_samba --> on

# getenforce 
Enforcing

# iptables -F

1. Prepare transient guest
 
# virsh list --transient
 Id    Name                           State
----------------------------------------------------
 4     test                           running

2. Do blockcopy 

# virsh blockcopy test vda /home/shyu/copy.img --verbose --wait
error: internal error: unable to execute QEMU command 'drive-mirror': Could not open file: Invalid argument
Comment 15 Shanzhi Yu 2014-07-15 22:22:13 EDT
Created attachment 918284 [details]
libvirtd log
Comment 16 Shanzhi Yu 2014-07-15 22:22:44 EDT
Created attachment 918285 [details]
guest log
Comment 17 Peter Krempa 2014-07-17 11:01:46 EDT
Could you please re-try the scenario and:

1) post the audit log
2) try mounting the CIFS filesystem under a different directory (/mnt)
3) disable enforcing mode in selinux

Thanks, I'm not able to reproduce it in my setup.
Comment 18 Shanzhi Yu 2014-07-21 01:28:58 EDT
(In reply to Peter Krempa from comment #17)
> Could you please re-try the scenario and:
> 
> 1) post the audit log
> 2) try mounting the CIFS filesystem under a different directory (/mnt)
> 3) disable enforcing mode in selinux
> 
> Thanks, I'm not able to reproduce it in my setup.

I also can't reproduce it after try many times. But I found an werid problem about the "disk size" of the destination file. See below:


1. Do blockcopy to an transisent geust to mount point of samba

# mount|grep cifs
//10.66.5.xxx/test on /mnt/Samba type cifs (rw,relatime,vers=1.0,cache=strict,username=root,domain=SHYU_TEST_PC,uid=0,noforceuid,gid=0,noforcegid,addr=10.66.5.xxx,file_mode=0777,dir_mode=0777,nounix,serverino,rsize=61440,wsize=65536,actimeo=1)

# time virsh blockcopy rhel6 vda /mnt/Samba/copy.img  --wait --verbose 
Block Copy: [100 %]
Now in mirroring phase

real	0m8.670s
user	0m0.026s
sys	0m0.017s

2. Check destination file size by "qemu-img" and "ls"

# ls -lash /mnt/Samba/copy.img
1.0M -rwxrwxrwx. 1 root root 1.2G Jul 21 13:20 /mnt/Samba/copy.img

# qemu-img info /mnt/Samba/copy.img
image: /mnt/Samba/copy.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 1.0M                                   ..........
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false

3. Abort blockcopy job by --abort, then recheck destination file size

# virsh blockjob rhel6 vda --abort 

# ls -lash /mnt/Samba/copy.img
2.0G -rwxrwxrwx. 1 root root 1.2G Jul 21 13:22 /mnt/Samba/copy.img

# qemu-img info /mnt/Samba/copy.img
image: /mnt/Samba/copy.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 2.0G                                  ...........
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false

Disk size change after abort blockjob. And this can't reproduce with local dir.
Comment 19 Peter Krempa 2014-07-21 10:25:19 EDT
(In reply to Shanzhi Yu from comment #18)
> (In reply to Peter Krempa from comment #17)
> > Could you please re-try the scenario and:
> > 
> > 1) post the audit log
> > 2) try mounting the CIFS filesystem under a different directory (/mnt)
> > 3) disable enforcing mode in selinux
> > 
> > Thanks, I'm not able to reproduce it in my setup.
> 
> I also can't reproduce it after try many times. But I found an werid problem
> about the "disk size" of the destination file. See below:
> 

The main problem here is apparently that mounting the cifs share under /home is a problem as /home has some special selinux attributes and apparently doesn't work in such config. As it works when it's mounted in /mnt this should be fine.


> 
> 1. Do blockcopy to an transisent geust to mount point of samba
> 
> # mount|grep cifs
> //10.66.5.xxx/test on /mnt/Samba type cifs
> (rw,relatime,vers=1.0,cache=strict,username=root,domain=SHYU_TEST_PC,uid=0,
> noforceuid,gid=0,noforcegid,addr=10.66.5.xxx,file_mode=0777,dir_mode=0777,
> nounix,serverino,rsize=61440,wsize=65536,actimeo=1)
> 
> # time virsh blockcopy rhel6 vda /mnt/Samba/copy.img  --wait --verbose 
> Block Copy: [100 %]
> Now in mirroring phase
> 
> real	0m8.670s
> user	0m0.026s
> sys	0m0.017s
> 
> 2. Check destination file size by "qemu-img" and "ls"
> 
> # ls -lash /mnt/Samba/copy.img
> 1.0M -rwxrwxrwx. 1 root root 1.2G Jul 21 13:20 /mnt/Samba/copy.img
> 
> # qemu-img info /mnt/Samba/copy.img
> image: /mnt/Samba/copy.img
> file format: qcow2
> virtual size: 5.0G (5368709120 bytes)
> disk size: 1.0M                                   ..........
> cluster_size: 65536
> Format specific information:
>     compat: 1.1
>     lazy refcounts: false
> 
> 3. Abort blockcopy job by --abort, then recheck destination file size
> 
> # virsh blockjob rhel6 vda --abort 
> 
> # ls -lash /mnt/Samba/copy.img
> 2.0G -rwxrwxrwx. 1 root root 1.2G Jul 21 13:22 /mnt/Samba/copy.img
> 
> # qemu-img info /mnt/Samba/copy.img
> image: /mnt/Samba/copy.img
> file format: qcow2
> virtual size: 5.0G (5368709120 bytes)
> disk size: 2.0G                                  ...........
> cluster_size: 65536
> Format specific information:
>     compat: 1.1
>     lazy refcounts: false
> 
> Disk size change after abort blockjob. And this can't reproduce with local
> dir.

At this point all the work is done by qemu and thus you should file a bug against qemu.
Comment 20 Shanzhi Yu 2014-07-22 01:14:58 EDT
(In reply to Peter Krempa from comment #19)
> (In reply to Shanzhi Yu from comment #18)
> > (In reply to Peter Krempa from comment #17)
> > > Could you please re-try the scenario and:

Thanks, file one bug 1121858 to qemu-kvm-rhev
Comment 21 Shanzhi Yu 2014-12-09 03:14:37 EST
Move this bug to VERIFIED according comment 18 and comment 19
Comment 23 errata-xmlrpc 2015-03-05 02:20:08 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html

Note You need to log in before you can comment on or make changes to this bug.