Bug 1257813

Summary: Domain can not start with virtio console
Product: Red Hat Enterprise Linux 7 Reporter: Dan Zheng <dzheng>
Component: libvirtAssignee: Andrea Bolognani <abologna>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact: Jiri Herrmann <jherrman>
Priority: unspecified    
Version: 7.2CC: dgibson, dyuan, gsun, jdenemar, jherrman, jsuchane, lvivier, michal.skrivanek, mzhan, rbalakri
Target Milestone: rc   
Target Release: ---   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-3.0.0-1.el7 Doc Type: Known Issue
Doc Text:
Non-functional QEMU virtio console on IBM Power Systems Currently, the virtio console does not work when QEMU is used on IBM Power Systems. As a consequence, it is impossible to start a guest that is configured to use the virtio console instead of the default spapr-vty console. To work around this issue, use the spapr-vty console.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 17:06:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dan Zheng 2015-08-28 06:34:23 UTC
Description of problem:
Guest fails to start after attach-device with a virtio console.
This issue happens only on PPC64LE.

Version-Release number of selected component (if applicable):
libvirt-1.2.17-6.el7.ppc64le
qemu-kvm-rhev-2.3.0-18.el7.ppc64le
kernel-3.10.0-302.el7.ppc64le

How reproducible:
100%

Steps to Reproduce:
1. Define and run a guest
# virsh  list --all
 Id    Name                           State
----------------------------------------------------
 5     virt-tests-vm1                 running

2. Attach a console device
The attach.xml is as below:
<console type="pty"><target port="1" type="virtio" /></console>

# virsh attach-device --domain virt-tests-vm1 --file attach.xml --persistent
Device attached successfully

3. Check the dumpxml of guest, a new <console> section is inserted for type virtio.

# virsh dumpxml virt-tests-vm1
    <serial type='pty'>
      <source path='/dev/pts/2'/>
      <target type='isa-serial' port='0'/>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </serial>
    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </console>
 +   <console type='pty'>
 +       <source path='/dev/pts/3'/>
 +       <target type='virtio' port='1'/>
 +       <alias name='console1'/>
 +    </console>

4. Destroy the guest and fail to start it again.

# virsh destroy virt-tests-vm1
Domain virt-tests-vm1 destroyed

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     virt-tests-vm1                 shut off

# virsh start  virt-tests-vm1
error: Failed to start domain virt-tests-vm1
error: internal error: no assigned pty for device console1



Actual result:
See above

Expected result:
Guest should be able to start successfully with the virtio console.

Additional information:
---Libvirtd.log:
2015-08-28 06:26:27.125+0000: 22876: debug : virEventPollMakePollFDs:401 : Prepare n=6 w=7, f=15 e=0 d=0
2015-08-28 06:26:27.125+0000: 22878: error : qemuProcessLookupPTYs:1921 : internal error: no assigned pty for device console1

Comment 2 Dan Zheng 2015-08-28 06:51:45 UTC
There is a similar error for channel.
Same packages as above.

1. attach_channel.xml 
<channel type="pty">
  <target name="virt-test.unix.channel" type="virtio" />
</channel>

2. With a running guest, do attach-device
# virsh attach-device --domain virt-tests-vm1 --file attach_channel.xml --persistent
Device attached successfully

3. DumpXML
# virsh dumpxml  virt-tests-vm1
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/virt-tests-vm1.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
+    <channel type='pty'>
+      <source path='/dev/pts/2'/>
+      <target type='virtio' name='virt-test.unix.channel'/>
+      <alias name='channel1'/>
+      <address type='virtio-serial' controller='0' bus='0' port='2'/>
+    </channel>

4. Destroy the guest and restart it, failed.
# virsh start virt-tests-vm1
error: Failed to start domain virt-tests-vm1
error: internal error: no assigned pty for device channel1

Comment 6 David Gibson 2015-11-05 03:46:32 UTC
Smells to me like libvirt on power isn't correctly generating the command lines for these cases.  Can someone extract the qemu command lines that are generated in the failing cases so we can check?

Comment 7 Dan Zheng 2015-12-07 08:23:24 UTC
Hi, David

libvirt-1.2.17-13.el7_2.2.ppc64le
qemu-kvm-rhev-2.3.0-31.el7_2.3.ppc64le
kernel-3.10.0-327.4.1.el7.ppc64le


1. Attach a console device successfully to a running guest
2. Destroy the guest
3. DumpXML for the guest
# virsh dumpxml virt-tests-vm1
    <serial type='pty'>
      <target port='0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </console>
    <console type='pty'>
      <target type='virtio' port='1'/>
    </console>
4. Fail to start the guest and below is got from guest log.

2015-12-07 08:15:38.481+0000: starting up libvirt version: 1.2.17, package: 13.el7_2.2 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2015-11-23-07:44:56, ppc-029.build.eng.bos.redhat.com), qemu version: 2.3.0 (qemu-kvm-rhev-2.3.0-31.el7_2.3)
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name virt-tests-vm1 -S -machine pseries-rhel7.2.0,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 5,sockets=1,cores=5,threads=1 -uuid cb5bb9dd-3567-464c-8090-4690e80fe363 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-virt-tests-vm1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2015-12-07T08:15:37,clock=host -no-shutdown -boot strict=on -device spapr-vscsi,id=scsi0,reg=0x1000 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x1 -usb -drive file=/home/zhwang/rhel7.2_http_virtio.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=22,id=hostnet0 -device spapr-vlan,netdev=hostnet0,id=net0,mac=52:54:00:c4:e7:14,reg=0x2000 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,reg=0x30000000 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/virt-tests-vm1.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev pty,id=charconsole1 -device virtconsole,chardev=charconsole1,id=console1 -device usb-kbd,id=input0 -device usb-mouse,id=input1 -device usb-tablet,id=input2 -vnc 0.0.0.0:0 -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -object rng-random,id=objrng0,filename=/dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=1234,period=2000,bus=pci.0,addr=0x5 -global spapr-nvram.reg=0x3000 -msg timestamp=on
char device redirected to /dev/pts/6 (label charserial0)
char device redirected to /dev/pts/7 (label charconsole1)

Comment 8 Laurent Vivier 2016-02-22 14:30:47 UTC
This is really a libvirt bug as virtconsole works well with QEMU ppc64le.

Moreover, if we don't stop/start the guest after adding the virtconsole, it can be used to login (if we have a kernel with the good module and getty configured for this console).

The generated command line seems correct:

-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x1
...
-chardev pty,id=charserial0 \
-device spapr-vty,chardev=charserial0,reg=0x30000000
...
-chardev pty,id=charconsole1 \
-device virtconsole,chardev=charconsole1,id=console1

It looks like libvirt is not able to start a guest with two PTY (of different kind?). Perhaps the missing "id=" for the spapr-vty can be a problem for libvirt?

My tests have been done using virt-manager.

Comment 10 Andrea Bolognani 2017-01-02 09:08:55 UTC
Fixed upstream.

commit 5f65c96e8de95518d7a8d4cf3cedd9d6ab581f4f
Author: Shivaprasad G Bhat <sbhat.ibm.com>
Date:   Wed Oct 19 18:59:02 2016 +0530

    Allow virtio-console on PPC64
    
    virQEMUCapsSupportsChardev existing checks returns true
    for spapr-vty alone. Instead verify spapr-vty validity
    and let the logic to return true for other device types
    so that virtio-console passes.
    
    The non-pseries machines dont have spapr-vio-bus. So, the
    function always returned false for them before.
    
    Fixes - https://bugzilla.redhat.com/show_bug.cgi?id=1257813
    
    Signed-off-by: Shivaprasad G Bhat <sbhat.ibm.com>

v2.5.0-254-g5f65c96

Comment 12 Dan Zheng 2017-03-07 07:03:41 UTC
Test packages:

libvirt-3.1.0-1.el7.ppc64le
qemu-kvm-rhev-2.8.0-5.el7.ppc64le
kernel-3.10.0-578.el7.ppc64le

Steps:
1. Define and run a guest
# virsh  list --all
 Id    Name                           State
----------------------------------------------------
 5     dd               running

2. Attach a console device
The attach.xml is as below:
<console type="pty"><target port="1" type="virtio" /></console>

# virsh attach-device --domain dd --file attach.xml --persistent
Device attached successfully

3. Check the dumpxml of guest, a new <console> section is inserted for type virtio.
# virsh dumpxml dd
...

4. Destroy the guest and restart it again and no problem.
5. Also try to add another virtio channel.. The guest can start successfully and normal operations in the guest are ok.
    <console type='pty'>
      <source path='/dev/pts/3'/>
      <target type='virtio' port='1'/>
      <alias name='console1'/>
    </console>

    <channel type='pty'>
      <source path='/dev/pts/2'/>
      <target type='virtio' name='virt-test.unix.channel' state='disconnected'/>
      <alias name='channel1'/>
      <address type='virtio-serial' controller='0' bus='0' port='2'/>
    </channel>

So mark it pass.

Comment 13 errata-xmlrpc 2017-08-01 17:06:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 14 errata-xmlrpc 2017-08-01 23:48:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 15 errata-xmlrpc 2017-08-02 01:25:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846