Bug 1373154 - Guest fails boot up with ivshmem-plain and virtio-pci device
Summary: Guest fails boot up with ivshmem-plain and virtio-pci device
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: seabios
Version: 7.3
Hardware: x86_64
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Marcel Apfelbaum
QA Contact: Pei Zhang
URL:
Whiteboard:
Depends On:
Blocks: 1393322
TreeView+ depends on / blocked
 
Reported: 2016-09-05 11:26 UTC by Pei Zhang
Modified: 2016-11-09 10:31 UTC (History)
15 users (show)

Fixed In Version: seabios-1.9.1-5.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1393322 (view as bug list)
Environment:
Last Closed: 2016-11-04 00:28:31 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2207 normal SHIPPED_LIVE seabios bug fix update 2016-11-03 13:23:27 UTC

Comment 3 Marcel Apfelbaum 2016-09-08 11:06:46 UTC
(In reply to Pei Zhang from comment #0)
> Description of problem:
> Boot guest with ivshmem-plain, guest will fail start up with  'No bootable
> device'. 
> 
> Version-Release number of selected component (if applicable):
> Host:
> 3.10.0-503.el7.x86_64
> qemu-kvm-rhev-2.6.0-22.el7.x86_64
> 
> Guest:
> 3.10.0-503.el7.x86_64
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. Boot guest with ivshmem-plain
> # /usr/libexec/qemu-kvm -name rhel7.3 \
> -cpu IvyBridge,check -m 4G \
> -smp 4,sockets=2,cores=2,threads=1 \
> -netdev tap,id=hostnet0 \
> -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=22:54:00:5c:77:61,
> rx_queue_size=256 \
> -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16 \
> -spice
> port=5902,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-
> migration=on \
> -monitor stdio \
> -serial unix:/tmp/monitor,server,nowait \
> -qmp tcp:0:5551,server,nowait \
> -drive
> file=/home/pezhang/rhel7.3.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,
> werror=stop,rerror=stop \
> -device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0 \
> -usbdevice tablet \
> -object
> memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on \
> -device ivshmem-plain,memdev=mem \
> 
> 
> Actual results:
> Guest fails start.
> 
> Expected results:
> Guest should start.
> 
> Additional info:
> 1. This is a regression bug.
> qemu-kvm-rhev-2.6.0-18.el7.x86_64   work
> qemu-kvm-rhev-2.6.0-19.el7.x86_64   doesn't work
> qemu-kvm-rhev-2.6.0-20.el7.x86_64   doesn't work
> qemu-kvm-rhev-2.6.0-22.el7.x86_64   doesn't work
> 
> 2. The exact commit which causes this bug.
> # git bisect bad
> eca64aee65e32ed710af1185cefda038f599c86c is the first bad commit
> commit eca64aee65e32ed710af1185cefda038f599c86c
> Author: Marcel Apfelbaum <marcel@redhat.com>
> Date:   Wed Aug 3 08:36:43 2016 +0200
> 
>     hw/virtio-pci: fix virtio behaviour


The mentioned BZ only enables the virtio 1.0 .
In order to find the root cause we need to bisect with :
  -device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0,disable-modern=off

I'll update when I'll know more.
Thanks,
Marcel

Comment 4 Marcel Apfelbaum 2016-09-11 17:28:09 UTC
If there is no room to map all MMIO BARs into the 32-bit PCI window,
SeaBIOS will re-allocate all 64-bit MMIO BARs into over-4G space.

Virtio 1.0 block devices (virtio-blk/virtio-scsi) use a 64-bit BAR
unusable by SeaBIOS if mapped over 4G space, preventing the system
to boot.

The simplest solution is to follow the xhci model and simply
skip migrating the virtio 1.0 modern bar into over-4G space.

Patch posted upstream for SeaBIOS: 
[PATCH] pci: don't map virtio 1.0 storage devices above 4G
(No link yet)

If the solution is accepted I'll re-assign the BZ to SeaBIOS component.

Thanks,
Marcel

Comment 5 Pei Zhang 2016-09-12 01:40:53 UTC
(In reply to Marcel Apfelbaum from comment #3)
[...]
> 
> The mentioned BZ only enables the virtio 1.0 .
> In order to find the root cause we need to bisect with :
>   -device
> virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0,disable-modern=off
> 
> I'll update when I'll know more.
> Thanks,
> Marcel

Note: Sorry for missed this comment before. And put the re-test results here in case it's useful.

Re-test with adding 'disable-modern=off'.

1. As bug[1] is fixed in qemu-kvm-rhev-2.6.0-18.el7, so re-test this version.
[1]Bug 1333318 - ivshmem-plain support in RHEL 7.3

qemu-kvm-rhev-2.6.0-18.el7.x86_64   doesn't work('No bootable device')

2. Also tried the first version of 2.6. (ivshmem-plain has showed up)

qemu-kvm-rhev-2.6.0-1.el7.x86_64    doesn't work('No bootable device')

So seems disable-modern=off is incompatible with ivshmem-plain, even with it's first supported version.


Best Regards,
Pei

Comment 6 Marcel Apfelbaum 2016-09-13 07:47:40 UTC
It seems there is already work in progress dealing with the same problem.

From an offline conversation with Gerd:

Look at these upstream commits:
  c579d2f85b4835e39490d862a18143cf130c4078
  0e21548b15e25e010c362ea13d170c61a3fcc899

They implement support for accessing virtio bars via pci config space,
which will make virtio devices work in seabios even if mapped over 4G.

Unfortunately they depend on some pci cleanups done on the master
branch.  They are not clean cherry-picks for 1.9-stable, and because of
that the 1.9.x releases do not have them.

I have an older version of those patches laying around in a local
branch, they do apply to 1.9.x, pushed it to my repo:
  https://www.kraxel.org/cgit/seabios/log/?h=virtio-old

Alternatively we could make seabios skip virtio devices when mapping
bars over 4G.  Have a look at pci_region_migrate_64bit_entries(), there
is already an exception for usb (so xhci isn't mapped high).

Comment 9 Marcel Apfelbaum 2016-09-13 12:53:15 UTC
The upstream commit fixing it: 0e21548b15e25e010c362ea13d170c61a3fcc899).

Comment 10 Marcel Apfelbaum 2016-09-13 12:56:13 UTC
(In reply to Pei Zhang from comment #5)
> (In reply to Marcel Apfelbaum from comment #3)
> [...]
> > 
> > The mentioned BZ only enables the virtio 1.0 .
> > In order to find the root cause we need to bisect with :
> >   -device
> > virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0,disable-modern=off
> > 
> > I'll update when I'll know more.
> > Thanks,
> > Marcel
> 
> Note: Sorry for missed this comment before. And put the re-test results here
> in case it's useful.
> 
> Re-test with adding 'disable-modern=off'.
> 
> 1. As bug[1] is fixed in qemu-kvm-rhev-2.6.0-18.el7, so re-test this version.
> [1]Bug 1333318 - ivshmem-plain support in RHEL 7.3
> 
> qemu-kvm-rhev-2.6.0-18.el7.x86_64   doesn't work('No bootable device')
> 
> 2. Also tried the first version of 2.6. (ivshmem-plain has showed up)
> 
> qemu-kvm-rhev-2.6.0-1.el7.x86_64    doesn't work('No bootable device')
> 
> So seems disable-modern=off is incompatible with ivshmem-plain, even with
> it's first supported version.
> 

Thanks for checking it!

Please check (without the disable-modern=off) using the seabios package
from the brew build:
    https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11741451 

> 
> Best Regards,
> Pei

Comment 11 Pei Zhang 2016-09-14 01:50:05 UTC
(In reply to Marcel Apfelbaum from comment #10)
[...]
> Please check (without the disable-modern=off) using the seabios package
> from the brew build:
>     https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11741451 
[...] 

Re-test:
Versions:
qemu-kvm-rhev-2.6.0-24.el7.x86_64

Steps:
1. Install above seabios build in host

2. Boot guest without disable-modern=off, guest can boot successfully.
[...]
-device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0 \

So with this seabios build, no error occurs.

Best Regards,
Pei

Comment 14 Miroslav Rezanina 2016-09-15 06:52:53 UTC
Fix included in seabios-1.9.1-5.el7

Comment 16 Pei Zhang 2016-09-15 12:40:00 UTC
==Verification==

Versions:
Host:
3.10.0-506.rt56.411.el7.x86_64
qemu-kvm-rhev-2.6.0-25.el7.x86_64
seabios-1.9.1-5.el7.x86_64

Guest:
3.10.0-506.rt56.411.el7.x86_64


Steps:
1. Boot guest with ivshmem-plain, guest works well.
Same as Description.

So this bug has been fixed well. Thank you.

Comment 18 errata-xmlrpc 2016-11-04 00:28:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2207.html


Note You need to log in before you can comment on or make changes to this bug.