Bug 1459170

Summary: SLOF: Can't boot from virtio-scsi disk behind pci-bridge: E3405: No such device
Product: Red Hat Enterprise Linux 7 Reporter: Andrea Bolognani <abologna>
Component: qemu-kvmAssignee: Thomas Huth <thuth>
Status: CLOSED ERRATA QA Contact: xianwang <xianwang>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.4-AltCC: bugproxy, hannsj_uhl, juzhang, knoel, michen, qzhang, rbalakri, virt-maint, xianwang, zhengtli
Target Milestone: rcKeywords: Patch
Target Release: 7.4-Alt   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-2.9.0-15.el7a Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-09 11:28:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1440030    

Description Andrea Bolognani 2017-06-06 12:58:16 UTC
I have a guest configured with

  spapr-pci-host-bridge
    pci-bridge
      virtio-scsi-pci
        scsi-hd

and SLOF can't seem to be able to boot from the disk:

  SLOF **********************************************************************
  QEMU Starting
   Build Date = May 19 2017 07:43:48
   FW Version = mockbuild@ release 20170303
   Press "s" to enter Open Firmware.

  Populating /vdevice methods
  Populating /vdevice/vty@30000000
  Populating /vdevice/nvram@71000000
  Populating /pci@800000020000000
                       00 0800 (B) : 1b36 0001    pci*
                       01 0800 (D) : 1af4 1004    virtio [ scsi ]
  Populating /pci@800000020000000/pci@1/scsi@1
         SCSI: Looking for devices
            100000000000000 DISK     : "QEMU     QEMU HARDDISK    2.5+"
                       00 1000 (D) : 1af4 1000    virtio [ net ]
                       00 1800 (D) : 1af4 1003    virtio [ serial ]
                       00 2000 (D) : 1af4 1002    unknown-legacy-device*
  No NVRAM common partition, re-initializing...
  Scanning USB
  Using default console: /vdevice/vty@30000000

    Welcome to Open Firmware

    Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
    This program and the accompanying materials are made available
    under the terms of the BSD License available at
    http://www.opensource.org/licenses/bsd-license.php


  Trying to load:  from: /pci@800000020000000/pci-bridge@1/scsi@1/disk@100000000000000 ...
  E3405: No such device

  E3407: Load failed

    Type 'boot' and press return to continue booting the system.
    Type 'reset-all' and press return to reboot the system.


  Ready!
  0 >

If I plug the virtio-scsi-pci straight into pci.0 the guest
boots successfully.

libvirt XML:

  <domain type='kvm'>
    <name>abologna-pegas</name>
    <uuid>51a2947e-015d-4adb-9cae-e5d86e71394c</uuid>
    <memory unit='KiB'>2097152</memory>
    <currentMemory unit='KiB'>2097152</currentMemory>
    <vcpu placement='static'>8</vcpu>
    <os>
      <type arch='ppc64le' machine='pseries-rhel7.4.0'>hvm</type>
      <boot dev='hd'/>
    </os>
    <cpu mode='host-passthrough' check='none'/>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <emulator>/usr/libexec/qemu-kvm</emulator>
      <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2'/>
        <source file='/var/lib/libvirt/images/abologna-pegas.qcow2'/>
        <target dev='sda' bus='scsi'/>
        <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      </disk>
      <controller type='pci' index='0' model='pci-root'/>
      <controller type='pci' index='1' model='pci-bridge'>
        <model name='pci-bridge'/>
        <target chassisNr='1'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
      </controller>
      <controller type='scsi' index='0' model='virtio-scsi'>
        <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/>
      </controller>
      <controller type='usb' index='0' model='none'/>
      <controller type='virtio-serial' index='0'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      </controller>
      <interface type='network'>
        <mac address='52:54:00:03:fb:e5'/>
        <source network='default'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </interface>
      <serial type='pty'>
        <target port='0'/>
        <address type='spapr-vio' reg='0x30000000'/>
      </serial>
      <console type='pty'>
        <target type='serial' port='0'/>
        <address type='spapr-vio' reg='0x30000000'/>
      </console>
      <channel type='unix'>
        <target type='virtio' name='org.qemu.guest_agent.0'/>
        <address type='virtio-serial' controller='0' bus='0' port='1'/>
      </channel>
      <memballoon model='virtio'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
      </memballoon>
      <panic model='pseries'/>
    </devices>
  </domain>

QEMU command line:

  /usr/libexec/qemu-kvm \
  -name guest=abologna-pegas,debug-threads=on \
  -S \
  -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-6-abologna-pegas/master-key.aes \
  -machine pseries-rhel7.4.0,accel=kvm,usb=off,dump-guest-core=off \
  -cpu host \
  -m 2048 \
  -realtime mlock=off \
  -smp 8,sockets=8,cores=1,threads=1 \
  -uuid 51a2947e-015d-4adb-9cae-e5d86e71394c \
  -display none \
  -no-user-config \
  -nodefaults \
  -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-6-abologna-pegas/monitor.sock,server,nowait \
  -mon chardev=charmonitor,id=monitor,mode=control \
  -rtc base=utc \
  -no-shutdown \
  -boot strict=on \
  -device pci-bridge,chassis_nr=1,id=pci.1,bus=pci.0,addr=0x1 \
  -device virtio-scsi-pci,id=scsi0,bus=pci.1,addr=0x1 \
  -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
  -drive file=/var/lib/libvirt/images/abologna-pegas.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 \
  -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 \
  -netdev tap,fd=25,id=hostnet0,vhost=on,vhostfd=27 \
  -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:03:fb:e5,bus=pci.0,addr=0x2 \
  -chardev pty,id=charserial0 \
  -device spapr-vty,chardev=charserial0,reg=0x30000000 \
  -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-6-abologna-pegas/org.qemu.guest_agent.0,server,nowait \
  -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 \
  -msg timestamp=on

Packages:

  SLOF-20170303-4.git66d250e.el7.noarch
  qemu-kvm-rhev-2.9.0-7.el7.ppc64le

Comment 3 Thomas Huth 2017-06-07 06:49:20 UTC
The problem is the "pci-bridge" string in the boot path "/pci@800000020000000/pci-bridge@1/scsi@1/disk@100000000000000" ... SLOF does not use "pci-bridge" but "pci" instead:

0 > devalias disk 
 /pci@800000020000000/pci@1/scsi@1/disk@100000000000000

Looks like we've got to teach QEMU to pass a different string for PCI bridges...

Comment 4 Thomas Huth 2017-06-07 08:22:35 UTC
Suggested patch upstream:
https://marc.info/?i=1496823627-17609-1-git-send-email-thuth@redhat.com

Comment 5 Miroslav Rezanina 2017-06-27 11:56:37 UTC
Fix included in qemu-kvm-2.9.0-15.el7a

Comment 7 xianwang 2017-07-06 06:31:23 UTC
This bug is reproduced on qemu-kvm-rhev-2.9.0-7.el7.ppc64le, and works well on  qemu-kvm-2.9.0-16.el7a.ppc64le.
Bug reproduction:
Host:
4.11.0-10.el7a.ppc64le
qemu-kvm-rhev-2.9.0-7.el7.ppc64le
SLOF-20170303-4.git66d250e.el7.noarch

Guest:
(4.11.0-10.el7a.ppc64le) 7.4 (Pegas)

steps:
1.Boot guest with following qemu cli:
/usr/libexec/qemu-kvm \
    -name 'Pegas7.4'  \
    -sandbox off  \
    -machine pseries,accel=kvm \
    -nodefaults  \
    -device pci-bridge,chassis_nr=1,id=pci.1,bus=pci.0,addr=0x1 \
    -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=06 \
    -device virtio-scsi-pci,id=controller_scsi,bus=pci.1,addr=0x1 \
    -device virtio-net-pci,mac=9a:4f:50:51:52:53,id=id9HRc5V,vectors=4,netdev=idjlQN53,bus=pci.0,addr=09  \
    -netdev tap,id=idjlQN53,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
    -drive file=/home/xianwang/dhcp_server.qcow2,id=drive_scsi1,if=none,snapshot=off,aio=native,cache=none,werror=stop,rerror=stop \
    -device scsi-hd,id=scsi1,drive=drive_scsi1,bus=controller_scsi.0,bootindex=1 \
    -drive file=/home/xianwang/Pegas-7.4-20170530.n.0-Server-ppc64le-dvd1.iso,id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom \
    -device scsi-cd,id=cd1,drive=drive_cd1,bus=controller_scsi.0 \
    -m 4096 \
    -monitor stdio \
    -vnc :1 \
    -qmp tcp:0:8881,server,nowait \
    -smp 4,maxcpus=4 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=2  \
    -device usb-mouse,id=usb-mouse,bus=usb1.0,port=3 \
    -rtc base=utc,clock=host  \
    -boot order=cdn,once=n,menu=on,strict=on \
    -enable-kvm  \
    -chardev socket,id=console1,path=/tmp/console1,server,nowait \
    -device spapr-vty,chardev=console1 \

2.Open a console port in the same host to check the boot info:
# nc -U /tmp/console1

result:
slof can't boot from the disk which with "pci-bridge" string in the boot path
Trying to load:  from: /pci@800000020000000/pci-bridge@1/scsi@1/disk@100000000000000 ... 
E3405: No such device

E3407: Load failed

  Type 'boot' and press return to continue booting the system.
  Type 'reset-all' and press return to reboot the system.


Ready! 
0 > ^C

Bug verification:
Host:
4.11.0-10.el7a.ppc64le
qemu-kvm-2.9.0-16.el7a.ppc64le
SLOF-20170303-4.git66d250e.el7.noarch

Guest:
(4.11.0-10.el7a.ppc64le) 7.4 (Pegas)

steps are same as bug reproduction.

result:
slof can boot from hard disk which with "pci" string in the boot path
Trying to load:  from: /pci@800000020000000/pci@1/scsi@1/disk@100000000000000 ...   Successfully loaded

      Red Hat Enterprise Linux Server (4.11.0-10.el7a.ppc64le) 7.4 (Pegas)      
      Red Hat Enterprise Linux Server (4.11.0-5.el7.ppc64le) 7.4 (Pegas)       
      Red Hat Enterprise Linux Server (0-rescue-bdba9f21cc794630bbabb06c45e2c8>


So, this bug is fixed.

Comment 9 errata-xmlrpc 2017-11-09 11:28:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3169