Bug 781467

I'm also adding the guest xml, just for reference.
=====================
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE 
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh edit f16rguest1
or other application using the libvirt API.
-->

<domain type='kvm'>
  <name>f16rguest1</name>
  <uuid>b0a6672e-2c0f-4708-878f-14128360b810</uuid>
  <memory>3145728</memory>
  <currentMemory>3145728</currentMemory>
  <vcpu>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc-0.14'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu match='exact'>
    <model>Penryn</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='dca'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='ds_cpl'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='vme'/>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/var/lib/libvirt/images/f16rguest1.dsk'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <interface type='bridge'>
      <mac address='52:54:00:17:00:37'/>
      <source bridge='virbr0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='tcp'>
      <source mode='bind' host='127.0.0.1' service='56862'/>
      <protocol type='raw'/>
      <target port='1'/>
    </serial>
    <console type='tcp'>
      <source mode='bind' host='127.0.0.1' service='56862'/>
      <protocol type='raw'/>
      <target type='serial' port='1'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>
=====================

(Side note: I added the <cpu> attribute intentionally as part of testing other things.)

Just to confirm: This is reproducible, I tried with another fresh guest(created using oz (I used a quick script -- http://kashyapc.fedorapeople.org/virt/oz-guest.bash)

Result:
----------------
[root@moon qemu]# virsh console reg-guest1
Connected to domain reg-guest1
Escape character is ^]
error: Unable to open stream for '127.0.0.1': No such file or directory

[root@moon qemu]#
----------------

Vague suspicion: The only thing I can think of for now is the new 'oz' version which I used to create these guest -- oz-0.8.0-1.fc16.noarch

Okay, we are not opening TCP ports as files any longer:

commit 4716138229ae47c5492c13a8622b779889560fd7
Author:     Peter Krempa <pkrempa>
AuthorDate: Thu Oct 6 12:24:47 2011 +0200
Commit:     Peter Krempa <pkrempa>
CommitDate: Mon Feb 27 15:05:17 2012 +0100

    qemu: Add ability to abort existing console while creating new one
    
    This patch fixes console corruption, that happens if two concurrent
    sessions are opened for a single console on a domain. Result of this
    corruption was that each of the console streams recieved just a part
    of the data written to the pipe so every console rendered unusable.
    
    New helper function for safe console handling is used to establish the
    console stream connection. This function ensures that no other libvirt
    client is using the console (with the ability to disconnect consoles of
    libvirt clients) and that no UUCP style lockfile is placed on the PTY
    device.
    
    * src/qemu/qemu_domain.h
            - add data structure to domain's private data dealing with
              console connections
    * src/qemu/qemu_domain.c:
            - allocate/free domain's console data structure
    * src/qemu/qemu_driver.c
            - use the new helper function for console handling

v0.9.10-54-g4716138

commit dde91ab9172c7aa929b98df74c16afd296fe435a
Author:     Jan Kiszka <jan.kiszka>
AuthorDate: Tue Mar 13 16:48:27 2012 +0100
Commit:     Eric Blake <eblake>
CommitDate: Mon Apr 16 22:24:20 2012 -0600

    Do not enforce source type of console[0]
    
    If console[0] is an alias for serial[0], do not enforce the former to
    have a PTY source type. This breaks serial consoles on stdio and makes
    no sense.
    
    Signed-off-by: Jan Kiszka <jan.kiszka>

v0.9.11-71-gdde91ab

In other words, one will obtain much more precise error message here:
# virsh start --console f16
Domain f16 started
Connected to domain f16
Escape character is ^]
error: internal error character device (null) is not using a PTY

However, we still don't handle TCP consoles and allow only PTY ones. I assume it's aim of this bug, to learn libvirt to handle TCP consoles, isn't?

I'm backporting the error reporting fixes to 0.9.6-maint, but let's move this to the upstream tracker to track tcp console support in virsh

Duh, I missed this bug.

The aim of the original bug description was - serial console access was failing when a guest was created with Oz (from the XML snippet I posted, it was accessing PTY consoles)

Now it's resolved I tried with oz-0.10.0-1.fc19

$ rpm -q libvirt-daemon-kvm qemu
libvirt-daemon-kvm-1.0.6-2.fc19.x86_64
qemu-1.5.0-8.fc19.x86_64

$ sudo virsh console f19-jeos
Connected to domain f19-jeos
Escape character is ^]

Fedora release 19 (Schrödinger’s Cat)
Kernel 3.9.5-301.fc19.x86_64 on an x86_64 (ttyS0)

localhost login: root
Password: 
[root@localhost ~]#

Created attachment 766492 [details]
Successful Oz guest creation with a PTY console. Added for reference.

I have to question why Oz is using TCP based consoles at all. Even after binding them to 127.0.0.1, they can't really be considered secure, because any other user on the host can connect to this console, at the very least DOS'ing the app which was trying to use it, since QEMU only allows a single TCP connection.  IMHO oz should just use 'pty' based consoles, or 'unix' based consoles. I'd support addition of support for 'unix' based consoles to virDomainOpenConsole, but I don't think we should deal with TCP/UDP consoles at all. They're just an all-round bad idea.

Just to be clear, Oz actually has two serial devices attached.  One is the serial console, and is created with this libxml2 code:

        # serial console pseudo TTY
        serial = devices.newChild(None, "serial", None)
        serial.setProp("type", "pty")
        serialTarget = serial.newChild(None, "target", None)
        serialTarget.setProp("port", "0")

The second one is the "listener", and is what is used to report the guest IP address to oz-install when doing customization.  It is created with this libxml2 code:

        serial = dev.newChild(None, "serial", None)
        serial.setProp("type", "tcp")
        serialSource = serial.newChild(None, "source", None)
        serialSource.setProp("mode", "bind")
        serialSource.setProp("host", "127.0.0.1")
        serialSource.setProp("service", str(self.listen_port))
        serialProtocol = serial.newChild(None, "protocol", None)
        serialProtocol.setProp("type", "raw")
        serialTarget = serial.newChild(None, "target", None)
        serialTarget.setProp("port", "1")

It's this latter one that uses TCP serial devices.  There is an open bug against Oz to change this; I just haven't had the time to fix it.  I'm guessing that the unix based console is the way to go for what Oz wants to do, which is simple communication between the guest and the host without using the network.

(In reply to Chris Lalancette from comment #8)
> It's this latter one that uses TCP serial devices.  There is an open bug
> against Oz to change this; I just haven't had the time to fix it.  I'm
> guessing that the unix based console is the way to go for what Oz wants to
> do, which is simple communication between the guest and the host without
> using the network.

In that case I see no reason for libvirt to support TCP consoles with 'virsh console'. A patch to support unix consoles though would be ok.

*** Bug 1243474 has been marked as a duplicate of this bug. ***

Since openstack uses TCP consoles it's a bit more compelling to try and make virsh console support those as well, I suspect it's not all that much more work

Thank you for reporting this issue to the libvirt project. Unfortunately we have been unable to resolve this issue due to insufficient maintainer capacity and it will now be closed. This is not a reflection on the possible validity of the issue, merely the lack of resources to investigate and address it, for which we apologise. If you none the less feel the issue is still important, you may choose to report it again at the new project issue tracker https://gitlab.com/libvirt/libvirt/-/issues The project also welcomes contribution from anyone who believes they can provide a solution.