Bug 1211973

Summary: 'guest-fstrim' failed for guest with os on spapr-vscsi disk
Product: Red Hat Enterprise Linux 7 Reporter: Gu Nini <ngu>
Component: qemu-guest-agentAssignee: Thomas Huth <thuth>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: hhuang, huding, juzhang, knoel, michen, mrezanin, qzhang, thuth, xuhan, ypu, zhengtli
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-guest-agent-2.3.0-2.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1213236 (view as bug list) Environment:
Last Closed: 2015-11-19 07:11:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1213236    

Description Gu Nini 2015-04-15 10:33:43 UTC
Description of problem:
For guest started with os on spapr-vscsi disk, it's a failure to execute the guest agent cmd 'guest-fstrim'


Version-Release number of selected component (if applicable):
Host kernel: 3.10.0-234.el7.ppc64
Qemu-kvm: qemu-kvm-rhev-2.2.0-8.el7.ppc64
Guest kernel: 2.6.32-548.el6.ppc64/3.10.0-229.el7.ppc64
Guest agent: 0.12.1.2-2.465.el6.ppc64/2.1.0-4.el7.ppc64

How reproducible:
100%

Steps to Reproduce:
1. Create a qcow2 img disk and a scsi debug disk in the host:
# qemu-img create -f qcow2 hd-0414.qcow2 2G
# modprobe scsi_debug lbpu=1 lbpws=1 
# lsscsi 
2. Start a guest(rhel6.7 or rhel7.2) with spapr-vscsi disk os and guest agent, and above 2 disks with 'discard=on' and as virtio-scsi hd/spapr-vscsi block on the backend: 

/usr/libexec/qemu-kvm -name spaprraw-0415 -machine pseries-rhel7.1.0,accel=kvm,usb=off -m 32768 -realtime mlock=off -smp 8,sockets=1,cores=16,threads=8 -uuid 95346a10-1828-403a-a610-ac5a52a29490 -no-user-config -nodefaults -monitor stdio -rtc base=utc,clock=host -no-shutdown -boot strict=on -device pci-ohci,id=usb1,bus=pci.0,addr=0x1 -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x8  -device spapr-vscsi,id=scsi0,reg=0x1000 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/media/ngu/spaprraw-0415,if=none,id=drive-scsi0-0-0-0,format=raw -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/media/ngu/iso/RHEL-7.1-20150219.1-Server-ppc64-dvd1.iso,if=none,id=drive-scsi0-0-1-0,readonly=on,format=raw,cache=none -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0,bootindex=2 -drive file=/media/ngu/hd-0414.qcow2,if=none,id=drive-scsi1-0-2-0,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=2,lun=0,drive=drive-scsi1-0-2-0,id=scsi1-0-2-0 -drive file=/dev/sdm,if=none,id=drive-scsi0-0-3-0,format=raw,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-block,bus=scsi0.0,channel=0,scsi-id=3,lun=0,drive=drive-scsi0-0-3-0,id=scsi0-0-3-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device spapr-vlan,netdev=hostnet0,id=net0,mac=52:54:00:c4:e7:90,reg=0x2000 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,reg=0x30000000 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/spaprraw-0415.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-kbd,id=input0 -device usb-mouse,id=input1 -device usb-tablet,id=input2 -vnc 0:1 -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -object rng-random,id=objrng0,filename=/dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=1234,period=2000,bus=pci.0,addr=0x6 -msg timestamp=on

3. In the guest, install and start guest agent; 'mkfs.ext4' and 'mount' the 2 data disks; write/remove some data on them
4. In the host, connect to the guest agent channel with cmd 'nc -U /var/lib/libvirt/qemu/channel/target/spaprraw-0415.qemu.guest_agent.0'
5. On above channel, execute guest agent cmd 'guest-fstrim' for 5 times:
{"execute":"guest-fstrim"}            
{"return": {}}
{"execute":"guest-fstrim"}
{"return": {}}
{"execute":"guest-fstrim"}
{"error": {"class": "GenericError", "desc": "failed to trim /media: Invalid argument"}}

{"execute":"guest-fstrim"}
{"error": {"class": "GenericError", "desc": "failed to trim /media: Invalid argument"}}

{"execute":"guest-fstrim"}
{"error": {"class": "GenericError", "desc": "failed to trim /media: Invalid argument"}}

Actual results:
The fstrim cmd execution failed since the 3rd time, and it's not a certain value for the cmd exection times to let fstrim fail, I have met it the 2nd, 3rd and 4th time

Expected results:
It should be always a success to execute 'guest-fstrim'

Additional info:
For guest with os on 'virtio blk' and 'virtio scsi' disk, there is no the problem

Comment 2 Gu Nini 2015-04-15 11:51:13 UTC
(In reply to Gu Nini from comment #0)

Please note after the bug, cmd 'fstrim' inside the guest could still be used normally.

Comment 3 Thomas Huth 2015-04-28 19:53:56 UTC
Seems like the problem can also be reproduced without real scsi disk. I can also get the error when I run qemu with this slightly stripped command line and by using disk images instead:

/usr/libexec/qemu-kvm -name bz1211973 -machine pseries-rhel7.1.0,accel=kvm,usb=off -m 32768 -realtime mlock=off -smp 8,sockets=1,cores=16,threads=8 -no-user-config -nodefaults -serial stdio -device pci-ohci,id=usb1,bus=pci.0,addr=0x1 -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x8  -device spapr-vscsi,id=scsi0,reg=0x1000 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/home/thuth/rhel-7.1-le.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/home/thuth/hd-0414.qcow2,if=none,id=drive-scsi1-0-2-0,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=2,lun=0,drive=drive-scsi1-0-2-0,id=scsi1-0-2-0 -drive file=/home/thuth/qemudisk.raw,if=none,id=drive-scsi0-0-3-0,format=raw,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=3,lun=0,drive=drive-scsi0-0-3-0,id=scsi0-0-3-0 -chardev socket,id=charchannel0,path=/tmp/qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -nographic

Comment 4 Thomas Huth 2015-04-29 10:47:18 UTC
This problem is likely the same as one that has been recently reported to the qemu mailing list:

https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg00109.html

(the values in struct fstrim_range have to be refreshed before each ioctl in qmp_guest_fstrim() in qga/commands-posix.c)

I'll try to revive that discussion to see whether we can get a fix in upstream...

Comment 6 Thomas Huth 2015-07-09 10:19:55 UTC
Fix is now available upstream: http://git.qemu-project.org/?p=qemu.git;a=commitdiff;h=73a652a1b08445e8d91e50cdbb2da50e571c61b3

Comment 7 Miroslav Rezanina 2015-07-13 08:16:32 UTC
Fix included in qemu-guest-agent-2.3.0-2.el7

Comment 9 huiqingding 2015-07-21 06:38:05 UTC
Reproduce this bug using the following version:
Host:
kernel-3.10.0-292.el7.x86_64
qemu-kvm-rhev-2.3.0-12.el7.x86_64
Guest:
kernel-3.10.0-229.el7.x86_64
qemu-guest-agent-2.1.0-4.el7.x86_64

Steps to Reproduce:
1. Start a guest rhel7.2 with scsi hd system disk and data disk:
# /usr/libexec/qemu-kvm -name bz1211973 -machine pc,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 8,sockets=1,cores=16,threads=8  -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x8 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/home/RHEL-Server-7.2-64-virtio-scsi.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/home/hd-0414.qcow2,if=none,id=drive-scsi1-0-2-0,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=2,lun=0,drive=drive-scsi1-0-2-0,id=scsi1-0-2-0 -drive file=/home/qemudisk.raw,if=none,id=drive-scsi0-0-3-0,format=raw,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=3,lun=0,drive=drive-scsi0-0-3-0,id=scsi0-0-3-0 -chardev socket,id=charchannel0,path=/tmp/qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0  -qmp tcp:0:6666,server,nowait -nodefaults -nodefconfig -spice port=5910,seamless-migration=on,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -monitor stdio -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,mac=52:54:00:c6:4b:15,id=test

2. In the guest, install and start guest agent; 'mkfs.ext4' and 'mount' the 2 data disks; write/remove some data on them

3. In the host, connect to the guest agent channel with cmd 'nc -U /tmp/qemu.guest_agent.0'

4. On above channel, execute guest agent cmd 'guest-fstrim' for 5 times:
{"execute":"guest-fstrim"}            
{"return": {}}
{"execute":"guest-fstrim"}
{"error": {"class": "GenericError", "desc": "failed to trim /media: Invalid argument"}}
{"execute":"guest-fstrim"}
{"error": {"class": "GenericError", "desc": "failed to trim /media: Invalid argument"}}

{"execute":"guest-fstrim"}
{"error": {"class": "GenericError", "desc": "failed to trim /media: Invalid argument"}}

{"execute":"guest-fstrim"}
{"error": {"class": "GenericError", "desc": "failed to trim /media: Invalid argument"}}

Actual results:
The fstrim cmd execution failed since the 2nd time

Comment 10 huiqingding 2015-07-21 06:40:56 UTC
Test this bug using the following version:
Host:
kernel-3.10.0-292.el7.x86_64
qemu-kvm-rhev-2.3.0-12.el7.x86_64
Guest:
kernel-3.10.0-229.el7.x86_64
qemu-guest-agent-2.3.0-2.el7.x86_64

Steps to Reproduce:
1. Start a guest rhel7.2 with scsi hd system disk and data disk:
# /usr/libexec/qemu-kvm -name bz1211973 -machine pc,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 8,sockets=1,cores=16,threads=8  -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x8 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/home/RHEL-Server-7.2-64-virtio-scsi.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/home/hd-0414.qcow2,if=none,id=drive-scsi1-0-2-0,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=2,lun=0,drive=drive-scsi1-0-2-0,id=scsi1-0-2-0 -drive file=/home/qemudisk.raw,if=none,id=drive-scsi0-0-3-0,format=raw,cache=none,aio=native,werror=stop,rerror=stop,discard=on -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=3,lun=0,drive=drive-scsi0-0-3-0,id=scsi0-0-3-0 -chardev socket,id=charchannel0,path=/tmp/qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0  -qmp tcp:0:6666,server,nowait -nodefaults -nodefconfig -spice port=5910,seamless-migration=on,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -monitor stdio -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,mac=52:54:00:c6:4b:15,id=test

2. In the guest, install and start guest agent; 'mkfs.ext4' and 'mount' the 2 data disks; write/remove some data on them

3. In the host, connect to the guest agent channel with cmd 'nc -U /tmp/qemu.guest_agent.0'

4. On above channel, execute guest agent cmd 'guest-fstrim' for 5 times:
{"execute":"guest-fstrim"}            
{"return": {}}
{"execute":"guest-fstrim"}            
{"return": {}}
{"execute":"guest-fstrim"}            
{"return": {}}
{"execute":"guest-fstrim"}            
{"return": {}}
{"execute":"guest-fstrim"}            
{"return": {}}

Actual results:
The fstrim cmd execution always success.

Comment 11 juzhang 2015-07-27 04:44:43 UTC
According to comment9 and comment10, set this issue as verified.

Comment 13 errata-xmlrpc 2015-11-19 07:11:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2217.html