Bug 1753992

Summary: core dump when testing persistent reservation in guest
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: Paolo Bonzini <pbonzini>
Component: qemu-kvmAssignee: Virtualization Maintenance <virt-maint>
qemu-kvm sub component: General QA Contact: Virtualization Bugs <virt-bugs>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: high CC: armbru, coli, ddepaula, dyuan, jinzhao, juzhang, knoel, lmen, mlevitsk, ngu, pbonzini, qzhang, virt-maint, xuwei, yanqzhan, yisun, ymankad
Version: 8.1Keywords: Regression
Target Milestone: rc   
Target Release: 8.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-4.1.0-11.module+el8.1.0+4250+4f5fbfdc Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1720047 Environment:
Last Closed: 2019-11-06 07:19:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1688183, 1720047    

Comment 3 Paolo Bonzini 2019-09-20 13:21:35 UTC
> 1. What is the scope of harm if this BZ is not resolved in this release?  Reviewers want to know which RHEL features or customers are affected and if it will impact any Layered Product or Hardware partner plans.

Persistent reservation on multipath disks are completely broken; this bug is blocking both the feature and its testing.

> 2. What are the risks associated with resolving this BZ?  Reviewers want to know the scope of retesting, potential regressions

The patch is trivial; fixing this bug may reveal other breakage in the same area, since the feature is currently untestable.

> 3. Provide any other details that meet blocker criteria or should be weighed in making a decision (Other releases affected, upstream status, business impacts, etc).

Fixed upstream (commit 6b9d62c2a9e83bbad73fb61406f0ff69b46ff6f3).

Comment 7 Xueqiang Wei 2019-09-24 08:14:55 UTC
Tested on qemu-kvm-4.1.0-11.module+el8.1.0+4250+4f5fbfdc, not hit this issue. So set status to VERIFIED.


Versions:
Host:
kernel-4.18.0-144.el8.x86_64
qemu-kvm-4.1.0-11.module+el8.1.0+4250+4f5fbfdc

Guest:
kernel-4.18.0-138.el8.x86_64

# multipath -ll
mpathb (360050763008084e6e0000000000001a8) dm-4 IBM,2145
size=100G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 2:0:1:0 sde 8:64 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  `- 2:0:0:0 sdd 8:48 active ready running
mpatha (360050763008084e6e0000000000001a4) dm-3 IBM,2145
size=100G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 1:0:1:0 sdc 8:32 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  `- 1:0:0:0 sdb 8:16 active ready running


Steps:
1. pass-through /dev/sdb

/usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1 \
    -device pcie-root-port,id=pcie_root_port_0,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
    -device pcie-root-port,id=pcie_root_port_1,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device pcie-root-port,id=pcie_root_port_2,slot=4,chassis=4,addr=0x4,bus=pcie.0  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_w2u90exl/monitor-qmpmonitor1-20181127-024837-wdAVx2FL,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_w2u90exl/monitor-catch_monitor-20181127-024837-wdAVx2FL,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idulvcka  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_w2u90exl/serial-serial0-20181127-024837-wdAVx2FL,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20181127-024837-wdAVx2FL,path=/var/tmp/avocado_w2u90exl/seabios-20181127-024837-wdAVx2FL,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20181127-024837-wdAVx2FL,iobase=0x402 \
    -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
    -device qemu-xhci,id=usb1,bus=pcie.0-root-port-5,addr=0x0 \
    -object iothread,id=iothread0 \
    -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
    -device virtio-scsi-pci,iothread=iothread0,id=virtio_scsi_pci0,bus=pcie.0-root-port-6,addr=0x0 \
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/rhel810-64-virtio-scsi.qcow2 \
    -device scsi-hd,drive=drive_image1,bus=virtio_scsi_pci0.0,id=disk0 \
    -device pcie-root-port,id=pcie.0-root-port-7,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
    -device virtio-net-pci,mac=9a:34:35:36:37:38,id=idyb3F88,vectors=4,netdev=idTAFS0s,bus=pcie.0-root-port-7,addr=0x0  \
    -netdev tap,id=idTAFS0s,vhost=on \
    -m 4G  \
    -smp 12,maxcpus=12,cores=6,threads=1,sockets=2  \
    -cpu 'SandyBridge',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:4444,server,nowait \
    -device pcie-root-port,id=pcie.0-root-port-9,slot=9,chassis=9,addr=0x9,bus=pcie.0 \
    -object pr-manager-helper,id=helper0,path=/var/run/qemu-pr-helper.sock \
    -device virtio-scsi-pci,id=virtio_scsi_pci1,bus=pcie.0-root-port-9,addr=0x0 \
    -drive id=host_disk,if=none,format=raw,file=/dev/sdb,file.pr-manager=helper0 \
    -device scsi-block,drive=host_disk,bus=virtio_scsi_pci1.0,id=disk1 \

2. test persistent reservation in guest.

cat test-persistent.sh

#! /bin/sh
sg_persist --no-inquiry -v --out --register-ignore --param-sark 123aaa "$@"
sg_persist --no-inquiry --in -k "$@"
sg_persist --no-inquiry -v --out --reserve --param-rk 123aaa --prout-type 5 "$@"
sg_persist --no-inquiry --in -r "$@"
sg_persist --no-inquiry -v --out --release --param-rk 123aaa --prout-type 5 "$@"
sg_persist --no-inquiry --in -r "$@"
sg_persist --no-inquiry -v --out --register --param-rk 123aaa --prout-type 5 "$@"
sg_persist --no-inquiry --in -k "$@"

# sh test-persistent.sh /dev/sdb
    Persistent reservation out cdb: 5f 06 00 00 00 00 00 00 18 00
PR out: command (Register and ignore existing key) successful
  PR generation=0x4d, 1 registered reservation key follows:
    0x123aaa
    Persistent reservation out cdb: 5f 01 05 00 00 00 00 00 18 00
PR out: command (Reserve) successful
  PR generation=0x4d, Reservation follows:
    Key=0x123aaa
    scope: LU_SCOPE,  type: Write Exclusive, registrants only
    Persistent reservation out cdb: 5f 02 05 00 00 00 00 00 18 00
PR out: command (Release) successful
  PR generation=0x4d, there is NO reservation held
    Persistent reservation out cdb: 5f 00 05 00 00 00 00 00 18 00
PR out: command (Register) successful
  PR generation=0x4e, there are NO registered reservation keys


3. restart guest with multipath (pass-through /dev/mapper/mpatha)

# sh test-persistent.sh /dev/sdb

    Persistent reservation out cdb: 5f 06 00 00 00 00 00 00 18 00 
PR out: command (Register and ignore existing key) successful
  PR generation=0x4a, 2 registered reservation keys follow:
    0x123aaa
    0x123aaa
    Persistent reservation out cdb: 5f 01 05 00 00 00 00 00 18 00 
PR out: command (Reserve) successful
  PR generation=0x4a, Reservation follows:
    Key=0x123aaa
    scope: LU_SCOPE,  type: Write Exclusive, registrants only
    Persistent reservation out cdb: 5f 02 05 00 00 00 00 00 18 00 
PR out: command (Release) successful
  PR generation=0x4a, there is NO reservation held
    Persistent reservation out cdb: 5f 00 05 00 00 00 00 00 18 00 
PR out: command (Register) successful
  PR generation=0x4c, there are NO registered reservation keys

4. check status of qemu-pr-helper service, it works well.
(1) stop it on host
# systemctl stop qemu-pr-helper
(2) run persistent reservation on guest
# sh test-persistent.sh /dev/sdb
(3) check status in qmp moniotr
{"timestamp": {"seconds": 1569312110, "microseconds": 101189}, "event": "PR_MANAGER_STATUS_CHANGED", "data": {"connected": false, "id": "helper0"}}
{"execute":"query-pr-managers"}
{"return": [{"connected": false, "id": "helper0"}]}
(4) start it on host
# systemctl start qemu-pr-helper
(5) run persistent reservation on guest
(6) check status in qmp moniotr
{"timestamp": {"seconds": 1569312147, "microseconds": 148410}, "event": "PR_MANAGER_STATUS_CHANGED", "data": {"connected": true, "id": "helper0"}}
{"execute":"query-pr-managers"}
{"return": [{"connected": true, "id": "helper0"}]}

Comment 9 errata-xmlrpc 2019-11-06 07:19:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3723