Bug 1103845
| Summary: | glusterfs backend does not support discard (libvirt) | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Hai Huang <hhuang> |
| Component: | libvirt | Assignee: | Peter Krempa <pkrempa> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.1 | CC: | areis, dyuan, jcody, juzhang, knoel, michen, mzhan, pbonzini, pkrempa, rbalakri, virt-maint, xuzhang, yanyang, yisun |
| Target Milestone: | rc | Keywords: | TestOnly |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1037503 | Environment: | |
| Last Closed: | 2016-11-03 18:09:24 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1055487, 1136534 | ||
| Bug Blocks: | 1059976 | ||
Verified on libvirt-1.2.8-5.el7 qemu-kvm-rhev-2.1.2-4.el7.x86_64 qemu-kvm-1.5.3-76.el7 test matrix: qemu: qemu-kvm-rhev, qemu-kvm host fs type: ext4, xfs disk format: raw, qcow2 disk interface: scsi, ide, sata, virtio disk fs type: ext4, xfs Steps: mount the gluster server to your host mount -t glusterfs $ip:gluster-vol1 -o direct-io-mode=1 /tmp/gluster 1. Boot guest with the following xml <disk type='network' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source protocol='gluster' name='gluster-vol1/rhel7-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='qcow2' discard='unmap'/> <source protocol='gluster' name='gluster-vol1/t-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='sda' bus='scsi'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='qcow2' discard='ignore'/> <source protocol='gluster' name='gluster-vol1/y-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='sdb' bus='scsi'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='qcow2'/> <source protocol='gluster' name='gluster-vol1/z-qcow2.img'> <host name='$ip'/> </source> <backingStore/> <target dev='sdc' bus='scsi'/> </disk> 2. check qemu line # ps -ef|grep qemu | grep discard -drive file=gluster://$ip/gluster-vol1/rhel7-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=gluster://$ip/gluster-vol1/t-qcow2.img,if=none,id=drive-scsi0-0-0-0,format=qcow2,discard=unmap -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -drive file=gluster://$ip/gluster-vol1/y-qcow2.img,if=none,id=drive-scsi0-0-0-1,format=qcow2,discard=ignore -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -drive file=gluster://$ip/gluster-vol1/z-qcow2.img,if=none,id=drive-scsi0-0-0-2,format=qcow2 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2 -netdev tap,fd=179,id=hostnet0,vhost=on,vhostfd=180 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:8f:13:eb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 3. guest ]# lsscsi [2:0:0:0] disk QEMU QEMU HARDDISK 2.1. /dev/sda [2:0:0:1] disk QEMU QEMU HARDDISK 2.1. /dev/sdb [2:0:0:2] disk QEMU QEMU HARDDISK 2.1. /dev/sdc guest ]# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:0/scsi_disk/2\:0\:0\:0/provisioning_mode unmap 4. In guest, make ext4 filesystem on sda, sdb #mkfs.ext4 /dev/sda #mkfs.ext4 /dev/sdb #mkfs.ext4 /dev/sdc 5. In host, check the disk block # stat t-qcow2.img File: ‘t-qcow2.img’ Size: 35061760 Blocks: 68360 IO Block: 131072 regular file # stat y-qcow2.img File: ‘y-qcow2.img’ Size: 35061760 Blocks: 68368 IO Block: 131072 regular file # stat z-qcow2.img File: ‘z-qcow2.img’ Size: 35061760 Blocks: 68368 IO Block: 131072 regular file 6. In guest, mount sda and sdb #mount /dev/sda /sda #mount /dev/sdb /sdb #mount /dev/sdc /sdc 7. In guest, dd a big file dd if=/dev/urandom of=/sda/file bs=1M count=300 && sync dd if=/dev/urandom of=/sdb/file bs=1M count=300 && sync dd if=/dev/urandom of=/sdc/file bs=1M count=300 && sync 8. In host, check the disk block # stat t-qcow2.img File: ‘t-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat y-qcow2.img File: ‘y-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat z-qcow2.img File: ‘z-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file 9. In guest, remove the file #rm /sda/file -f && sync #rm /sdb/file -f && sync #rm /sdc/file -f && sync #fstrim /sda #fstrim /sdb #fstrim /sdc 10. In host, check the disk block # stat t-qcow2.img File: ‘t-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat y-qcow2.img File: ‘y-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file # stat z-qcow2.img File: ‘z-qcow2.img’ Size: 350093312 Blocks: 683664 IO Block: 131072 regular file Per comment #5 in qemu bug 1136534, mark it as failedQA. Will verify it after qemu bug 1136534 fixed. bz 1055487 and bz 1136534 have been moved to rhel7.3.0, could you pls move this one to 7.3.0? thx Verified with pkgs:
libvirt-2.0.0-4.el7.x86_64
qemu-kvm-rhev-2.6.0-17.el7.x86_64
glusterfs-server-3.7.9-11.el7rhgs.x86_64
HOST# virsh dumpxml virtlab_test
...
<disk type='network' device='disk'>
<driver name='qemu' type='qcow2' discard='unmap'/>
<source protocol='gluster' name='gluster-vol1/qcow2.img'>
<host name='$IP'/>
</source>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none' discard='unmap'/>
<source protocol='gluster' name='gluster-vol1/raw.img'>
<host name='$IP'/>
</source>
<target dev='sdb' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
...
HOST# virsh start virtlab_test
Domain virtlab_test started
HOST# ps -ef | grep qemu | grep unmap
qemu 7090 1 4 19:26 ? 00:00:08 /usr/libexec/qemu-kvm -name guest=virtlab_test, .... -drive file=gluster://10.66.5.82/gluster-vol1/qcow2.img,format=qcow2,if=none,id=drive-scsi0-0-0-0,discard=unmap -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -drive file=gluster://10.66.5.82/gluster-vol1/raw.img,format=raw,if=none,id=drive-scsi0-0-0-1,cache=none,discard=unmap ....
GLUSTER# stat raw.img
File: ‘raw.img’
Size: 1073741824 Blocks: 8 IO Block: 4096 regular file
GLUSTER# stat qcow2.img
File: ‘qcow2.img’
Size: 197120 Blocks: 400 IO Block: 4096 regular file
GUEST# lsscsi
[2:0:0:0] disk QEMU QEMU HARDDISK 2.5+ /dev/sda
[2:0:0:1] disk QEMU QEMU HARDDISK 2.5+ /dev/sdb
GUEST# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:0/scsi_disk/2\:0\:0\:0/provisioning_mode
unmap
GUEST# cat /sys/bus/scsi/devices/target2\:0\:0/2\:0\:0\:1/scsi_disk/2\:0\:0\:1/provisioning_mode
unmap
GUEST# mkfs.ext4 /dev/sda
GUEST# mkfs.ext4 /dev/sdb
GUEST# mkdir /sda
GUEST# mkdir /sdb
GUEST# mount /dev/sda /sda
GUEST# mount /dev/sdb /sdb
GLUSTER# stat raw.img
File: ‘raw.img’
Size: 1073741824 Blocks: 66352 IO Block: 4096 regular file
GLUSTER# stat qcow2.img
File: ‘qcow2.img’
Size: 559546368 Blocks: 67736 IO Block: 4096 regular file
GUEST# dd if=/dev/zero of=/sda/file bs=1M count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 4.76586 s, 110 MB/s
GUEST# dd if=/dev/zero of=/sdb/file bs=1M count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 6.36393 s, 82.4 MB/s
GLUSTER# stat raw.img
File: ‘raw.img’
Size: 1073741824 Blocks: 1123136 IO Block: 4096 regular file
GLUSTER# stat qcow2.img
File: ‘qcow2.img’
Size: 559546368 Blocks: 1164176 IO Block: 4096 regular file
GUEST# rm /sda/file -f
GUEST# fstrim /sda
GUEST# rm /sdb/file -f
GUEST# fstrim /sdb
GLUSTER# stat raw.img
File: ‘raw.img’
Size: 1073741824 Blocks: 344784 IO Block: 4096 regular file
GLUSTER# stat qcow2.img
File: ‘qcow2.img’
Size: 559546368 Blocks: 68624 IO Block: 4096 regular file
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2577.html |
This works with libvirt with the following configuration: <disk type='network' device='disk'> <driver name='qemu' type='qcow2' discard='unmap'/> <-- important <source protocol='gluster' name='gv0/img7'> <host name='gluster-node-1'/> </source> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> In guest: # cat /sys/bus/scsi/devices/target2:0:0/2:0:0:0/scsi_disk/2:0:0:0/provisioning_mode unmap Setting test-only keyword.