Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1037953

Summary: RHEL7.0 guest can not shutdown via system_powerdown in qmp monitor
Product: Red Hat Enterprise Linux 7 Reporter: zhonglinzhang <zhzhang>
Component: qemu-kvmAssignee: Hai Huang <hhuang>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: medium    
Version: 7.0CC: acathrow, bdas, chayang, hhuang, juzhang, michen, sluo, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-05-14 13:24:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description zhonglinzhang 2013-12-04 06:31:15 UTC
Description of problem:
Boot rhel7.0 guest with a VF assigned, then fail to shutdown the guest.  

Version-Release number of selected component (if applicable):
kernel: 3.10.0-57.el7.x86_64
qemu-kvm: qemu-kvm-rhev-1.5.3-20.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Bring up VFs and boot a guest
/usr/libexec/qemu-kvm -M q35 -cpu SandyBridge -enable-kvm -m 4G \
-smp 1,sockets=1,cores=2,threads=1 -name test \
-rtc base=utc,clock=host,driftfix=slew -k en-us -boot menu=on -vga qxl \
-spice disable-ticketing,port=5931 -monitor stdio  -qmp tcp:0:4444,server,nowait \
-device ioh3420,bus=pcie.0,id=root.0 -device x3130-upstream,bus=root.0,id=upstream     -device xio3130-downstream,bus=upstream,id=downstream0,chassis=1 \
-drive file=/home/RHEL-Server-7.0-64.raw,if=none,id=drive-system-disk,media=disk,format=raw,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,bus=downstream0,drive=drive-system-disk,id=system-disk,bootindex=1 \
-device xio3130-downstream,bus=upstream,id=downstream2,chassis=2 
-net none -device vfio-pci,host=09:04.3,id=hostnet_VF3 \
-serial unix:/tmp/ttyS0,server,nowait

2. Shutdown the guest: 
Hang at "Actual results"

3. 

Actual results:
Fail to shutdown the guest:
[   54.560539] Bad IO access at port 0xac (return inl(port))
[   56.561120] Bad IO access at port 0xac (return inl(port))
[   57.662415] pciehp 0000:00:02.0:pcie04: Device 0000:01:00.0 already exists at 0000:01:00, cannot hot-add
[   57.662417] pciehp 0000:00:02.0:pcie04: Cannot add device at 0000:01:00
[   58.562020] Bad IO access at port 0xac (return inl(port))
[   60.563077] Bad IO access at port 0xac (return inl(port))
[   62.564146] Bad IO access at port 0xac (return inl(port))
[   62.767556] pciehp 0000:02:00.0:pcie24: Device 0000:03:00.0 already exists at 0000:03:00, cannot hot-add
[   62.767557] pciehp 0000:02:00.0:pcie24: Cannot add device at 0000:03:00
[   64.565160] Bad IO access at port 0xac (return inl(port))
[   66.566148] Bad IO access at port 0xac (return inl(port))
[   68.567039] Bad IO access at port 0xac (return inl(port))
[   70.568050] Bad IO access at port 0xac (return inl(port))
[   72.569040] Bad IO access at port 0xac (return inl(port))
[  114.590184] be2net 0000:00:03.0: POST timeout; stage=0xffff
[  114.590190] dpm_run_callback(): pci_pm_resume+0x0/0xb0 returns -1
[  114.590194] PM: Device 0000:00:03.0 failed to resume async: error -1


Cannot finalize remaining file systems and devices, giving up.
dracut Warning: Killing all remaining processes
Powering off.
[  160.068240] be2net 0000:00:03.0: FW not responding
[  160.072997] Power down.

Expected results:
Succeed to shutdown the guest.

Additional info:
Succeed to shutdown windows guest.

Comment 2 Alex Williamson 2013-12-16 17:53:29 UTC
What is the device being assigned?

Does the shutdown problem only happen with that VF or is it reproducible with other VFs (82576/82599)?

Does this problem only happen on q35 guests?

Comment 3 zhonglinzhang 2013-12-17 05:57:24 UTC
Q35:    Retest this issue with 3.10.0-54.el7.x86_64 kernel guest, After I finished to install the guest without PF assigned, First Inside guest with power can succeed to shutdown the guest. But second in qmp monitor,After system_powerdown command, the guest stop at login screen, Inside guest with poweroff, failed to shutdown the guest. Maybe regardless of PF device assigned. 

the command line:
/usr/libexec/qemu-kvm -M q35 -cpu SandyBridge -enable-kvm -m 4G \
-smp 1,sockets=1,cores=2,threads=1 -name test -rtc base=utc,clock=host,driftfix=slew -k en-us -boot menu=on -vga qxl \
-spice disable-ticketing,port=5931 -monitor stdio  -qmp tcp:0:4444,server,nowait -device ioh3420,bus=pcie.0,id=root.0 \
-device x3130-upstream,bus=root.0,id=upstream     -device xio3130-downstream,bus=upstream,id=downstream0,chassis=1 -drive file=/home/RHEL7.0-64.raw,if=none,id=drive-system-disk,media=disk,format=raw,aio=native,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream0,drive=drive-system-disk,id=system-disk,bootindex=1 -device xio3130-downstream,bus=upstream,id=downstream2,chassis=2 \
-device virtio-net-pci,netdev=hostnet0,id=net0,bus=downstream2,mac=52:54:00:13:10:20 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup \
-serial unix:/tmp/ttyS0,server,nowait

Actual Result:
Hang at this stage:
[root@unused ~]# [  165.847127] pciehp 0000:00:02.0:pcie04: Device 0000:01:00.0 already exists at 0000:01:00, cannot hot-add
[  165.847129] pciehp 0000:00:02.0:pcie04: Cannot add device at 0000:01:00
[  170.949180] pciehp 0000:02:00.0:pcie24: Device 0000:03:00.0 already exists at 0000:03:00, cannot hot-add
[  170.949182] pciehp 0000:02:00.0:pcie24: Cannot add device at 0000:03:00
[  170.949226] pciehp 0000:02:01.0:pcie24: Device 0000:04:00.0 already exists at 0000:04:00, cannot hot-add
[  170.949227] pciehp 0000:02:01.0:pcie24: Cannot add device at 0000:04:00
[  OK  ] Stopped LSB: Start the ipr init daemon.
.................................
Cannot finalize remaining file systems and devices, giving up.
dracut Warning: Killing all remaining processes
Powering off.
[  208.919331] Power down.


pc-i440fx-rhel7.0.0: 
Hit the same result, but hang without any information in nc command monitor.

Comment 4 Alex Williamson 2013-12-17 13:52:17 UTC
(In reply to Alex Williamson from comment #2)
> What is the device being assigned?
> 
> Does the shutdown problem only happen with that VF or is it reproducible
> with other VFs (82576/82599)?

Comment 5 zhonglinzhang 2013-12-18 01:24:25 UTC
without VF can not shutdown the guest.

Comment 6 Alex Williamson 2013-12-18 02:41:05 UTC
(In reply to zhonglinzhang from comment #5)
> without VF can not shutdown the guest.

Are you saying this happens even without device assignment of any kind?

Comment 7 zhonglinzhang 2013-12-18 02:45:18 UTC
(In reply to Alex Williamson from comment #6)
> (In reply to zhonglinzhang from comment #5)
> > without VF can not shutdown the guest.
> 
> Are you saying this happens even without device assignment of any kind?


Yes, 
system_powerdown in qmp monitor, stop at login screen. Inside guest power off after login in guest, then hang.

Comment 8 Alex Williamson 2013-12-18 02:58:57 UTC
Changing title and re-assigning back to default assignee since this has nothing to do with device assignment.

Comment 9 Sibiao Luo 2013-12-18 03:01:27 UTC
(In reply to Alex Williamson from comment #8)
> Changing title and re-assigning back to default assignee since this has
> nothing to do with device assignment.

This bug maybe duplicate to bug 1040307 - Qemu monitor "system_powerdown" command fail to power down guest 

Best Regards,
sluo

Comment 14 juzhang 2014-05-14 02:12:48 UTC
Hi Sluo,

Can you do some investigation and test and update comment13?

Best Regards,
Junyi

Comment 15 Sibiao Luo 2014-05-14 03:29:14 UTC
(In reply to juzhang from comment #14)
> Hi Sluo,
> 
> Can you do some investigation and test and update comment13?
> 
> Best Regards,
> Junyi

Retried this issue with the same command as comment #0 and comment #3 which just same results with bug 1040307. I think we can close it duplicate to bug 1040307.

host info:
# uname -r && rpm -q qemu-kvm
3.10.0-123.el7.x86_64
qemu-kvm-1.5.3-60.el7.x86_64
guest info:
# uname -r
3.10.0-123.el7.x86_64

Best Regards,
sluo

Comment 16 Hai Huang 2014-05-14 13:24:07 UTC

*** This bug has been marked as a duplicate of bug 1040307 ***