Bug 667345

Summary: Cannot add/start a PPC virtual machine (Bus 'pci.0' not found)
Product: [Fedora] Fedora Reporter: Helmut Schlattl <helmut.schlattl>
Component: libvirtAssignee: Libvirt Maintainers <libvirt-maint>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: low    
Version: 14CC: aquini, berrange, clalance, crobinso, hbrock, itamar, jforbes, ndevos, rharwood, robatino, tristan.matthews, veillard, virt-maint
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-07-14 12:10:35 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
Use pci.0 as bus->name for PCI-busses qemu-kvm does
none
Use QEMUD_CMD_FLAG_PCI_MULTIBUS for passing "bus=pci" or "bus=pci.0"
none
qemu-system-ppc -nodefaults -nodefconfig -monitor stdio - info qtree - info pci
none
Reserve PCI addresses 3 and 4 on qemu-system-ppc
none
Screen shot of the kernel panic none

Description Helmut Schlattl 2011-01-05 05:35:45 EST
Description of problem:
A virtual machine with PPC-architecture cannot be installed/started.

Version-Release number of selected component (if applicable):
virt-manager-0.8.5-1.fc14

How reproducible:
Always

Steps to Reproduce:
1. Add new virtual machine
2. Choose as architecture ppc (type: qemu)
3. Try to finalize
  
Actual results:

Error message:
Unable to complete install '<class 'libvirt.libvirtError'> internal error Process exited while reading console log output: char device redirected to /dev/pts/3
qemu-system-ppc: -device rtl8139,vlan=0,id=net0,mac=52:54:00:bf:e0:d5,bus=pci.0,addr=0x3: Bus 'pci.0' not found

Expected results:
Create new virtual machine

Additional info:
It's not only a problem with the network card, but with all devices, since the virt-manager always adds "bus=pci.0" to the qemu call. It appears that qemu-system-ppc cannot cope with the same arguments as qemu-kvm. virt-manager should account for this! By the way, the default serial device added by virt-manager is also not recognized by qemu-system-ppc.
Comment 1 Niels de Vos 2011-01-27 05:44:59 EST
This is the same error as the one I get with qemu-system-arm.

http://fedoraproject.org/wiki/Architectures/ARM/HowToQemu points me to http://cdot.senecac.on.ca/arm/FEDORA13-WRAPPER-qemu-system-arm which effectively replaces bus=pci.0 by bus=pci. The script contains a 2-step description on how to use it. I guess it works the same for PPC.

This is far from optimal of course, but it might be okay for a workaround.
Comment 2 Niels de Vos 2011-01-28 11:17:52 EST
Created attachment 475826 [details]
Use pci.0 as bus->name for PCI-busses qemu-kvm does

Hmm, I've looked further into this and am of the opinion that this is a qemu issue and not libvirt's.

(posting my notes here)
With help from gdb and manually executing qemu-system-arm and comparing it with qemu-kvm, it seems that the virtual hardware is configured differently. Most noticible (for me) seems to be the name of the PCI-bus, on qemu-system-arm it is pci and on qemu-kvm it is pci.0. The function qbus_find_recursive() can be used to breakpoint and to check the names of the available busses (bus->name).

I would assume that at least the busses have the same names for emulated hardware, so either bus=pci or bus=pci.0 should work with any qemu-* command. Unfortunately it is unclear to me how qemu-kvm constructs the virtual hardware, the qemu-system-* are more transparent. Depending on the machine that is being emulated the hardware is 'connected' when the machine is created. The virtual hardware is constructed by qemu-kvm/hw/*.c and depends on the type of machine that is created.

Under qemu-kvm/hw/*.c there are some uses of pci_register_bus() where pci as name for the PCI-bus is passed. It seems easy to rename the PCI-busses to pci.0. This change will break any existing scripts that pass bus=pci, so the solution is not the best.
(end of notes)

I have built qemu with this patch for testing. For me ARM emulation works now with virt-manager (PPC should work too). The test packages are available here:
- http://repos.fedorapeople.org/repos/devos/qemu/

An email to gather thoughts on how this could be fixed alternatively just has been sent to the qemu-devel list.
Comment 3 Daniel Berrange 2011-01-28 11:20:56 EST
While the root cause is certainly a QEMU issue, previous attempts to address this at QEMU were rejected & changing it now may cause more pain than good. So we'll likely end up needing to fix it in libvirt by using either  'pci' or 'pci.0' as appropriate per arch.
Comment 4 Niels de Vos 2011-01-30 13:10:38 EST
Created attachment 476071 [details]
Use QEMUD_CMD_FLAG_PCI_MULTIBUS for passing "bus=pci" or "bus=pci.0"

This new patch makes the change in libvirt instead. The generated test packages work for me running x86_64, i686 and ARM guests. Details on how the patch works can be read in the commit message.

I'd happily receive feedback on the packages, which are available here:
- http://repos.fedorapeople.org/repos/devos/libvirt/


Note: This patch is for the current Fedora 14 release of libvirt (0.8.3) and does not apply on the current upstream version (0.8.7) without changes (work in progress).
Comment 5 Daniel Berrange 2011-01-31 08:48:24 EST
Ahh nice approach to fixing this.  I was thinking we'd have to pass the virDomainDefPtr into every single method, but setting a fake flag in qemudCmdFlags is much simpler. The patch looks sane as it is to me.
Comment 6 Helmut Schlattl 2011-01-31 14:16:31 EST
The patch of Niels solved the pci-problem on my PPC only partly. Now, 
I get a new error:

Unable to complete install '<class 'libvirt.libvirtError'> internal error Process exited while reading console log output: char device redirected to /dev/pts/0
qemu-system-ppc: -device rtl8139,vlan=0,id=net0,mac=52:54:00:f5:4e:1f,bus=pci,addr=0x3: PCI: slot 3 function 0 not available for rtl8139, in use by macio
qemu-system-ppc: -device rtl8139,vlan=0,id=net0,mac=52:54:00:f5:4e:1f,bus=pci,addr=0x3: Device 'rtl8139' could not be initialized

Is it the addr-statement ?
Comment 7 Daniel Berrange 2011-02-01 05:10:46 EST
This address in use error means that the machine has some piece of hardware enabled by default that is grabbing a PCI slot that we think is free. On x86 targets there is only the PIIX (slot 1) and VGA card (slot 2), so we start allocating from 3. It sounds like the PPC board has something already in slot three which we'd need to take into account. 

Can you run   qemu-system-ppc -nodefaults -nodefconfig -monitor stdio, and then run 'info qtree' and 'info pci' and attach the outpout of those to this bug.
Comment 8 Niels de Vos 2011-02-01 10:57:17 EST
Created attachment 476426 [details]
qemu-system-ppc -nodefaults -nodefconfig -monitor stdio - info qtree - info pci

For me qemu-system-ppc behaves the same way.

Attahching the output of the requested commands.
Comment 9 Niels de Vos 2011-02-02 06:03:05 EST
(In reply to comment #4)
> Note: This patch is for the current Fedora 14 release of libvirt (0.8.3) and
> does not apply on the current upstream version (0.8.7) without changes (work in
> progress).

The same patch for the current version of libvirt:
- https://www.redhat.com/archives/libvir-list/2011-February/msg00023.html
Comment 10 Niels de Vos 2011-02-03 05:36:21 EST
(In reply to comment #9)
> (In reply to comment #4)
> > Note: This patch is for the current Fedora 14 release of libvirt (0.8.3) and
> > does not apply on the current upstream version (0.8.7) without changes (work in
> > progress).
> 
> The same patch for the current version of libvirt:
> - https://www.redhat.com/archives/libvir-list/2011-February/msg00023.html

The "bus=pci" patch has been applied and pushed:
- http://libvirt.org/git/?p=libvirt.git;a=commit;h=0b864eb103274e43441ea784bfc7f99a0771a0f5
Comment 11 Niels de Vos 2011-02-05 09:57:04 EST
Created attachment 477203 [details]
Reserve PCI addresses 3 and 4 on qemu-system-ppc

The attached patch reserves PCI-slot 3 and 4, additional PCI-devices will get slot 5 and up.

Slot 3 is in use by macio, slot 4 is the OHCI controller.

New test-packages are available on the same repository, please report back if this works for you too:
- http://repos.fedorapeople.org/repos/devos/libvirt/

The version that has both patches is libvirt-0.8.3-2.fc14.0.bz667345.1.

Note that by default a ISA serial port is added to the guest definition. qemu-system-ppc does not provide an emulated ISA-bus, therefore you'll need to remove the serial port before starting the guest.


For my feeling this patch is really ugly, but I have not seen a better way to reserve PCI-slots per architecture. Its up to the libvirt developers to ACK/NACK a similar patch that I will send shortly.
Comment 12 Niels de Vos 2011-02-07 05:45:11 EST
btw, the patch sent upstream can be found here:
- http://permalink.gmane.org/gmane.comp.emulators.libvirt/33585
Comment 13 Helmut Schlattl 2011-02-07 15:28:29 EST
(In reply to comment #11)
> 
> The version that has both patches is libvirt-0.8.3-2.fc14.0.bz667345.1.
> 
> Note that by default a ISA serial port is added to the guest definition.
> qemu-system-ppc does not provide an emulated ISA-bus, therefore you'll need to
> remove the serial port before starting the guest.

I disabled the serial port, and the qemu starts without problems.

Now, after the kernel boots (tried Fedora 12 PPC), a kernel panic appears. I am
not sure whether this is a wrong setting from libvirt, or more a qemu-problem
(I presume the latter). 

Anyway, I create an attachment with the screen shot of the kernel panic. 

Thanks for all your efforts to get qemu-system-ppc running with libvirt!
Comment 14 Helmut Schlattl 2011-02-07 15:29:39 EST
Created attachment 477497 [details]
Screen shot of the kernel panic
Comment 15 Fedora Admin XMLRPC Client 2011-09-22 13:58:17 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 16 Fedora Admin XMLRPC Client 2011-09-22 14:01:59 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 17 Fedora Admin XMLRPC Client 2011-11-30 14:58:06 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 18 Fedora Admin XMLRPC Client 2011-11-30 14:59:11 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 19 Fedora Admin XMLRPC Client 2011-11-30 15:04:12 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 20 Fedora Admin XMLRPC Client 2011-11-30 15:04:50 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 21 Cole Robinson 2012-01-24 17:35:26 EST
F14 is EOL, please reopen if this is still relevant in a more recent fedora.
Comment 22 Tristan Matthews 2013-06-25 13:22:36 EDT
This bug is still present in Fedora 19, reproduced in virt-manager as above with ppc architecture.

Unable to complete install: 'internal error process exited while connecting to monitor: char device redirected to /dev/pts/2 (label charserial0)
qemu-system-ppc: -device piix3-usb-uhci,id=usb,bus=pci,addr=0x1.0x2: PCI: single function device can't be populated in function 1.2
qemu-system-ppc: -device piix3-usb-uhci,id=usb,bus=pci,addr=0x1.0x2: Device 'piix3-usb-uhci' could not be initialized
'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 100, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/create.py", line 1920, in do_install
    guest.start_install(False, meter=meter)
  File "/usr/share/virt-manager/virtinst/Guest.py", line 1134, in start_install
    noboot)
  File "/usr/share/virt-manager/virtinst/Guest.py", line 1202, in _create_guest
    dom = self.conn.createLinux(start_xml or final_xml, 0)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2782, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)
libvirtError: internal error process exited while connecting to monitor: char device redirected to /dev/pts/2 (label charserial0)
qemu-system-ppc: -device piix3-usb-uhci,id=usb,bus=pci,addr=0x1.0x2: PCI: single function device can't be populated in function 1.2
qemu-system-ppc: -device piix3-usb-uhci,id=usb,bus=pci,addr=0x1.0x2: Device 'piix3-usb-uhci' could not be initialized
Comment 23 Robbie Harwood 2014-07-14 12:00:06 EDT
This bug is still present in Fedora 20, reproduced in virt-manager using the ppc architecture.

Unable to complete install: 'internal error: process exited while connecting to monitor: qemu-system-ppc: -device piix3-usb-uhci,id=usb,bus=pci,addr=0x2: Bus 'pci' not found
'

Unable to complete install: 'internal error: process exited while connecting to monitor: qemu-system-ppc: -device piix3-usb-uhci,id=usb,bus=pci,addr=0x2: Bus 'pci' not found
'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 91, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/create.py", line 1787, in do_install
    guest.start_install(meter=meter)
  File "/usr/share/virt-manager/virtinst/guest.py", line 403, in start_install
    noboot)
  File "/usr/share/virt-manager/virtinst/guest.py", line 467, in _create_guest
    dom = self.conn.createLinux(start_xml or final_xml, 0)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2897, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)
libvirtError: internal error: process exited while connecting to monitor: qemu-system-ppc: -device piix3-usb-uhci,id=usb,bus=pci,addr=0x2: Bus 'pci' not found
Comment 24 Cole Robinson 2014-07-14 12:10:35 EDT
Please open a new bug report and report all package versions, libvirt XML, and /var/log/libvirt/qemu/$vmname.log