Bug 1449265

Summary: Libvirt uses a bad QEMU device for the <serial> XML tag on s390x
Product: Red Hat Enterprise Linux 7 Reporter: Thomas Huth <thuth>
Component: libvirtAssignee: Pino Toscano <ptoscano>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: low Docs Contact:
Priority: high    
Version: 7.5-AltCC: acarter, arubin, david, dzheng, fweimer, jsuchane, lfriedma, mtessun, rbalakri, rjones, salmy
Target Milestone: beta   
Target Release: 7.5-Alt   
Hardware: s390x   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-3.9.0-4.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-10 10:43:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1436140    
Bug Blocks: 1485496    

Description Thomas Huth 2017-05-09 13:45:44 UTC
Description of problem:
When adding the <serial> tag to the XML definition of a KVM guest on s390x, libvirt translates this into a "isa-serial" device when starting QEMU. Since this device does not exist in the s390x version of QEMU, the guest can not be launched.

Version-Release number of selected component (if applicable):
libvirt-3.1.0-2.el7.s390x
qemu-kvm-rhev-2.9.0-1.el7.s390x

How reproducible:
100%

Steps to Reproduce:
1. Install qemu-kvm and libvirt on a s390x system
2. Create a guest definition that contains a <serial> tag, for example:

<domain type="kvm">
  <name>guest</name>
  <memory>786432</memory>
  <vcpu>1</vcpu>
  <os>
    <type arch="s390x" machine="s390-ccw-virtio">hvm</type>
    <boot dev="hd"/>
  </os>
  <clock offset="utc"/>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/guest.qcow2"/>
      <target dev="sda" bus="scsi"/>
    </disk>
    <interface type="bridge">
      <source bridge="virbr0"/>
      <model type="virtio"/>
    </interface>
    <serial type='pty'/>
    <console type="pty"/>
  </devices>
</domain>

3. Start the guest with "virsh start guest"

Actual results:
error: Failed to start domain zkvmtest
error: internal error: qemu unexpectedly closed the monitor:
[...]
2017-05-09T13:42:08.553851Z qemu-kvm: -device isa-serial,chardev=charserial0,id=serial0: 'isa-serial' is not a valid device model name

Expected results:
Guest can be started successfully

Additional info:
Since s390x does not have an ISA bus, the 'isa-serial' device can not be used here. I suggest to map the <serial> tag to the 'virtio-serial' device on s390x instead.

Comment 2 Richard W.M. Jones 2017-05-16 18:07:49 UTC
(In reply to Thomas Huth from comment #0)
> Since s390x does not have an ISA bus, the 'isa-serial' device can not be
> used here. I suggest to map the <serial> tag to the 'virtio-serial' device
> on s390x instead.

There's a danger of missing vital early debugging if we do this.

Comment 3 Thomas Huth 2017-05-17 06:51:38 UTC
(In reply to Richard W.M. Jones from comment #2)
> (In reply to Thomas Huth from comment #0)
> > Since s390x does not have an ISA bus, the 'isa-serial' device can not be
> > used here. I suggest to map the <serial> tag to the 'virtio-serial' device
> > on s390x instead.
> 
> There's a danger of missing vital early debugging if we do this.

Early debugging text should go to the <console> on s390x, not to the <serial> device, and the <console> tag is already working as expected, so I don't think there is any danger here.

Comment 4 Richard W.M. Jones 2017-05-17 12:12:37 UTC
(In reply to Thomas Huth from comment #3)
> (In reply to Richard W.M. Jones from comment #2)
> > (In reply to Thomas Huth from comment #0)
> > > Since s390x does not have an ISA bus, the 'isa-serial' device can not be
> > > used here. I suggest to map the <serial> tag to the 'virtio-serial' device
> > > on s390x instead.
> > 
> > There's a danger of missing vital early debugging if we do this.
> 
> Early debugging text should go to the <console> on s390x, not to the
> <serial> device, and the <console> tag is already working as expected, so I
> don't think there is any danger here.

Yup, I've confirmed this is true on the s/390x machine too, thanks.

Comment 8 Ann Marie Rubin 2017-11-14 18:35:23 UTC
can we get devel and qa acks on this bug.

Comment 9 Pino Toscano 2017-11-15 12:04:32 UTC
Patch series upstream, which fixes the behaviour of serial consoles on non-x86 architectures:
https://www.redhat.com/archives/libvir-list/2017-November/msg00545.html

Comment 11 Pino Toscano 2017-11-22 17:08:36 UTC
(In reply to Pino Toscano from comment #9)
> Patch series upstream, which fixes the behaviour of serial consoles on
> non-x86 architectures:
> https://www.redhat.com/archives/libvir-list/2017-November/msg00545.html

v2 of the work:
https://www.redhat.com/archives/libvir-list/2017-November/msg00831.html

Comment 12 Ann Marie Rubin 2017-11-27 18:24:32 UTC
Is this fixed? Please move to MODIFIED.

Comment 13 Jaroslav Suchanek 2017-11-28 09:41:56 UTC
(In reply to Ann Marie Rubin from comment #12)
> Is this fixed? Please move to MODIFIED.

Not yet ready upstream, v3 under review
https://www.redhat.com/archives/libvir-list/2017-November/msg01027.html

Comment 14 Pino Toscano 2017-11-29 12:20:32 UTC
This is fixed upstream with commit:

commit 21332bf6587c23409fecb06ab81dbc14dd52c10b
Author: Pino Toscano <ptoscano>
Date:   Tue Nov 14 16:27:04 2017 +0100

    conf: add VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_SCLP
    
    Introduce specific a target types with two models for the console
    devices (sclp and sclplm) used in s390 and s390x guests, so isa-serial
    is no more used for them.
    
    This makes <serial> usable on s390 and s390x guests, with at most only
    a single sclpconsole and one sclplmconsole devices usable in a single
    guest (due to limitations in QEMU, which will enforce already at
    runtime).
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1449265
    
    Signed-off-by: Pino Toscano <ptoscano>
    Reviewed-by: Andrea Bolognani <abologna>
    Reviewed-by: Pavel Hrdina <phrdina>

Also some more commits are required for it.

Comment 19 Dan Zheng 2017-12-13 08:10:24 UTC
Test packages:

libvirt-3.9.0-5.el7.s390x
qemu-kvm-ma-2.10.0-11.el7.s390x
kernel-4.14.0-18.el7a.s390x


Case1: sclpconsole
Edit guest below and start guest OK

    <serial type='pty'>
    </serial>

qemu: 
-chardev pty,id=charserial0 -device sclpconsole,chardev=charserial0,id=serial0 

# virsh dumpxml vm1|grep serial -A7
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target type='sclp-serial' port='0'>
        <model name='sclpconsole'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>

sclpconsole console can be connected and operated.

Case 2:  sclpconsole + virtio console
Guest can start with below setting. 
    <serial type='pty'>
        <target type='sclp-serial' port='0'>
        <model name='sclpconsole'/>
      </target>
    </serial>
    <console type='pty'>
    </console>
    <console type='pty'>
      <target type='virtio' port='1'/>
    </console>

qemu: -chardev pty,id=charserial0 -device sclpconsole,chardev=charserial0,id=serial0 -chardev pty,id=charconsole1 -device virtconsole,chardev=charconsole1,id=console1

Case 3: sclpconsole + sclplmconsole + virtio console
Guest can start with below setting.
    <serial type='pty'>
      <target type='sclp-serial' port='0'>
        <model name='sclpconsole'/>
      </target>
    </serial>
    <serial type='pty'>
      <target type='sclp-serial' port='1'>
        <model name='sclplmconsole'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <console type='pty'>
      <target type='virtio' port='1'/>
    </console>

qemu: 
-chardev pty,id=charserial0 -device sclpconsole,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device sclplmconsole,chardev=charserial1,id=serial1 -chardev pty,id=charconsole1 -device virtconsole,chardev=charconsole1,id=console1 

Case 4:  sclplmconsole (better use s390 machine, not s390x)
Guest can start with below setting.
    <serial type='pty'>
      <target type='sclp-serial' port='0'>
        <model name='sclplmconsole'/>
      </target>
    </serial>

Dumpxml:
    <serial type='pty'>
      <target type='sclp-serial' port='0'>
        <model name='sclplmconsole'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
qemu:
chardev pty,id=charserial0 -device sclplmconsole,chardev=charserial0,id=serial0 -
sclplmconsole can be connected.

All tests are PASS.

Comment 23 errata-xmlrpc 2018-04-10 10:43:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704