Red Hat Bugzilla – Bug 1103845
glusterfs backend does not support discard (libvirt)
Last modified: 2016-11-03 14:09:24 EDT
This works with libvirt with the following configuration: <disk type='network' device='disk'> <driver name='qemu' type='qcow2' discard='unmap'/> <-- important <source protocol='gluster' name='gv0/img7'> <host name='gluster-node-1'/> </source> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> In guest: # cat /sys/bus/scsi/devices/target2:0:0/2:0:0:0/scsi_disk/2:0:0:0/provisioning_mode unmap Setting test-only keyword.
Verified on libvirt-1.2.8-5.el7 qemu-kvm-rhev-2.1.2-4.el7.x86_64 qemu-kvm-1.5.3-76.el7 test matrix: qemu: qemu-kvm-rhev, qemu-kvm host fs type: ext4, xfs disk format: raw, qcow2 disk interface: scsi, ide, sata, virtio disk fs type: ext4, xfs Steps: mount the gluster server to your host mount -t glusterfs $ip:gluster-vol1 -o direct-io-mode=1 /tmp/gluster 1. Boot guest with the following xml <disk type='network' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source protocol='gluster' name='gluster-vol1/rhel7-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='qcow2' discard='unmap'/> <source protocol='gluster' name='gluster-vol1/t-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='sda' bus='scsi'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='qcow2' discard='ignore'/> <source protocol='gluster' name='gluster-vol1/y-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='sdb' bus='scsi'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='qcow2'/> <source protocol='gluster' name='gluster-vol1/z-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='sdc' bus='scsi'/> </disk> 2. check qemu line # ps -ef|grep qemu | grep discard -drive file=gluster://$ip/gluster-vol1/rhel7-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=gluster://$ip/gluster-vol1/t-qcow2.img,if=none,id=drive-scsi0-0-0-0,format=qcow2,discard=unmap -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -drive file=gluster://$ip/gluster-vol1/y-qcow2.img,if=none,id=drive-scsi0-0-0-1,format=qcow2,discard=ignore -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -drive file=gluster://$ip/gluster-vol1/z-qcow2.img,if=none,id=drive-scsi0-0-0-2,format=qcow2 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2 -netdev tap,fd=179,id=hostnet0,vhost=on,vhostfd=180 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:8f:13:eb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 3. guest ]# lsscsi [2:0:0:0] disk QEMU QEMU HARDDISK 2.1. /dev/sda [2:0:0:1] disk QEMU QEMU HARDDISK 2.1. /dev/sdb [2:0:0:2] disk QEMU QEMU HARDDISK 2.1. /dev/sdc guest ]# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:0/scsi_disk/2\:0\:0\:0/provisioning_mode unmap 4. In guest, make ext4 filesystem on sda, sdb #mkfs.ext4 /dev/sda #mkfs.ext4 /dev/sdb #mkfs.ext4 /dev/sdc 5. In host, check the disk block # stat t-qcow2.img File: ‘t-qcow2.img’ Size: 35061760 Blocks: 68360 IO Block: 131072 regular file # stat y-qcow2.img File: ‘y-qcow2.img’ Size: 35061760 Blocks: 68368 IO Block: 131072 regular file # stat z-qcow2.img File: ‘z-qcow2.img’ Size: 35061760 Blocks: 68368 IO Block: 131072 regular file 6. In guest, mount sda and sdb #mount /dev/sda /sda #mount /dev/sdb /sdb #mount /dev/sdc /sdc 7. In guest, dd a big file dd if=/dev/urandom of=/sda/file bs=1M count=300 && sync dd if=/dev/urandom of=/sdb/file bs=1M count=300 && sync dd if=/dev/urandom of=/sdc/file bs=1M count=300 && sync 8. In host, check the disk block # stat t-qcow2.img File: ‘t-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat y-qcow2.img File: ‘y-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat z-qcow2.img File: ‘z-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file 9. In guest, remove the file #rm /sda/file -f && sync #rm /sdb/file -f && sync #rm /sdc/file -f && sync #fstrim /sda #fstrim /sdb #fstrim /sdc 10. In host, check the disk block # stat t-qcow2.img File: ‘t-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat y-qcow2.img File: ‘y-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat z-qcow2.img File: ‘z-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file
Per comment #5 in qemu bug 1136534, mark it as failedQA. Will verify it after qemu bug 1136534 fixed.
bz 1055487 and bz 1136534 have been moved to rhel7.3.0, could you pls move this one to 7.3.0? thx
Verified with pkgs: libvirt-2.0.0-4.el7.x86_64 qemu-kvm-rhev-2.6.0-17.el7.x86_64 glusterfs-server-3.7.9-11.el7rhgs.x86_64 HOST# virsh dumpxml virtlab_test ... <disk type='network' device='disk'> <driver name='qemu' type='qcow2' discard='unmap'/> <source protocol='gluster' name='gluster-vol1/qcow2.img'> <host name='$IP'/> </source> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none' discard='unmap'/> <source protocol='gluster' name='gluster-vol1/raw.img'> <host name='$IP'/> </source> <target dev='sdb' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> ... HOST# virsh start virtlab_test Domain virtlab_test started HOST# ps -ef | grep qemu | grep unmap qemu 7090 1 4 19:26 ? 00:00:08 /usr/libexec/qemu-kvm -name guest=virtlab_test, .... -drive file=gluster://10.66.5.82/gluster-vol1/qcow2.img,format=qcow2,if=none,id=drive-scsi0-0-0-0,discard=unmap -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -drive file=gluster://10.66.5.82/gluster-vol1/raw.img,format=raw,if=none,id=drive-scsi0-0-0-1,cache=none,discard=unmap .... GLUSTER# stat raw.img File: ‘raw.img’ Size: 1073741824 Blocks: 8 IO Block: 4096 regular file GLUSTER# stat qcow2.img File: ‘qcow2.img’ Size: 197120 Blocks: 400 IO Block: 4096 regular file GUEST# lsscsi [2:0:0:0] disk QEMU QEMU HARDDISK 2.5+ /dev/sda [2:0:0:1] disk QEMU QEMU HARDDISK 2.5+ /dev/sdb GUEST# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:0/scsi_disk/2\:0\:0\:0/provisioning_mode unmap GUEST# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:1/scsi_disk/2\:0\:0\:1/provisioning_mode unmap GUEST# mkfs.ext4 /dev/sda GUEST# mkfs.ext4 /dev/sdb GUEST# mkdir /sda GUEST# mkdir /sdb GUEST# mount /dev/sda /sda GUEST# mount /dev/sdb /sdb GLUSTER# stat raw.img File: ‘raw.img’ Size: 1073741824 Blocks: 66352 IO Block: 4096 regular file GLUSTER# stat qcow2.img File: ‘qcow2.img’ Size: 559546368 Blocks: 67736 IO Block: 4096 regular file GUEST# dd if=/dev/zero of=/sda/file bs=1M count=500 500+0 records in 500+0 records out 524288000 bytes (524 MB) copied, 4.76586 s, 110 MB/s GUEST# dd if=/dev/zero of=/sdb/file bs=1M count=500 500+0 records in 500+0 records out 524288000 bytes (524 MB) copied, 6.36393 s, 82.4 MB/s GLUSTER# stat raw.img File: ‘raw.img’ Size: 1073741824 Blocks: 1123136 IO Block: 4096 regular file GLUSTER# stat qcow2.img File: ‘qcow2.img’ Size: 559546368 Blocks: 1164176 IO Block: 4096 regular file GUEST# rm /sda/file -f GUEST# fstrim /sda GUEST# rm /sdb/file -f GUEST# fstrim /sdb GLUSTER# stat raw.img File: ‘raw.img’ Size: 1073741824 Blocks: 344784 IO Block: 4096 regular file GLUSTER# stat qcow2.img File: ‘qcow2.img’ Size: 559546368 Blocks: 68624 IO Block: 4096 regular file
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2577.html