Bug 223112

Okay, I think the expected behaviour here is:

  - With and without xenfb, xen's console driver impersonates /dev/tty[1-63]
    if no xencons= is specified on the cmdline, but only writes to /dev/tty1
    goes to the backend

  - Without console= or with console=tty xen's console driver makes 
    /dev/console equivalent to /dev/tty1

  - By default (i.e. without xencons=xvc specified), console=xvc is ignored

  - When you add xenfb into the equation, it creates a console but doesn't
    allocate a tty by virtue of the fact that the xen console driver sets
    console_use_vt = 0

  - The kernel writes its printk messages to all consoles, even ones without
    a tty driver allocated

  - We run a getty on /dev/tty[1-6] by default

  - The kernel exec()s init such that it writes to /dev/console

  - rhgb changes init's console to a pty which in turn displays inside X

So, to go through each of your observed behaviours:

  1) The fact that you don't see init messages makes sense, you should be
     seeing them in the VNC console. There's no reason why you shouldn't
     be seeing a login prompt - /dev/tty1 should be hooked up to the backend
     and we should be running a getty on /dev/tty1

  2) Bizarre. rhgb now isn't around to change the console, so init should
     be writing to /dev/console and those writes should be going to the
     console backend. The lack of a login prompt is just as strange as
     in (1).

3/4) Here you're booting without xenfb and seeing both init and the login
     process is visible.

One possible conclusion is that by loading xenfb, /dev/tty1 and /dev/console are
now going elsewhere. That would explain both (1) and (2) ... cc-ing Markus, this
could well be a known/old/fixed bug.

Chris, some more info would be useful:

  - What version of kernel-xen was this?

  - Is it still reproducible with the latest kernel?

  - dmesg logs from 1, 2 and 3 might shed some light on this

Hmm, I misread something ...

>   2) Bizarre. rhgb now isn't around to change the console, so init should
>      be writing to /dev/console and those writes should be going to the
>      console backend. The lack of a login prompt is just as strange as
>      in (1).

It's pretty clear that we were getting a login prompt in this case, which is
very strange - so /dev/tty1 is working, but not /dev/console, and rhgb is
somehow the influencing factor  ?

Console tty is the Linux virtual console.  It's accessible through
xenfb, when that's enabled.  Device files are /dev/tty[0-9]* (major 4)
and /dev/{tty,console,ptmx} (major 5).

Console xvc is the xen console, accessible through xm console.  Device
file is /dev/xvc0 (major 204, minor 191).  It can be made to mug the
real tty (xencons=tty), but that's a bad idea.  In fact, this
misfeature is currently *broken* in our kernels.  That's because the
upstream changeset that fixes it breaks stuff we actually care about.

Where init runs getty is outside the area of my expertise.  On my FC-6
system, I get a getty on /dev/xvc0 (i.e. in xm console) regardless of
kernel parameters, with and without xenfb.

Kernel messages (printks) go to all registered consoles with flag
CON_ENABLED set.  I get them in both xvc (xm console) and tty (xenfb).

Output to /dev/console goes to the registered console that is
specified last on the command line, if any.  Else it goes to the first
console that registers, which is tty.  The kernel is not prepared to
copy the output to more than one console.

User space can redirect /dev/console to a file (ioctl TIOCCONS).  This
could conceivably be used to tee console output to multiple places.  I
expect that to play badly with existing uses of TIOCCONS.

Ah, I see this is all different in RHEL-5 from the rawhide kernel I was looking
at - xencons=xvc is the default and kudzu has code for adding a getty for
/dev/xvc0, so ...

Chris: we really need an update from you on whether this is all still just as
broken with the latest RHEL-5 kernel ...

Re-assigning to Markus

I am testing this out right now...

I used the following command to install a xen guest:
#virt-install -n rhel5test1 -r 512 -f /var/lib/xen/images/rhel5test1.dsk -s 10
--vnc -p -l
nfs:curly.devel.redhat.com:/vol/engineering/devarchive/redhat/rel-eng/RHEL5-Server-20070208.0/tree-x86_64

[root@x460 xen]# uname -a
Linux x460.gsslab.rdu.redhat.com 2.6.18-8.1.1.el5xen #1 SMP Mon Feb 26 20:51:53
EST 2007 x86_64 x86_64 x86_64 GNU/Linux

It is still reproducible. I am not able to launch the console:
[root@x460 xen]# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0    13121    16 r-----    733.6
rhel5test1                                 4      511     1 -b----     26.2
[root@x460 xen]# xm console rhel5test1
xenconsole: Could not read tty from store: No such file or directory
[root@x460 xen]#

If I remove the rhgb quiet, then I get a login prompt, but I do not see any
kernel messages. If I have  "console=tty console=xvc" in the kernel line then I
see the INIT messages and the login prompt.

I am also attaching the dmesg from cases 1, 2 and 3

Thanks,
Archana

Created attachment 151811 [details]
dmesg with default graphical install

Created attachment 151812 [details]
Case 2- xm_dmesg-without-rhgb_quiet

Created attachment 151813 [details]
Case 3- with "console=tty console=xvc"

Comments #1 and #7 do not make it perfectly clear to me whether case (3) has
rhgb quiet.  Please advise.

For all cases where `INIT messages' are not visible in xm console: do they show
up in xenfb?  If you can't connect your VNC viewer quickly enough to see all the
action, try SDL.

OK, here we go again.  Based on a conversation with Markus on IRC, here's the
data from another experiment booting a RHEL-5 paravirt guest.  In the following,
I have 3 different message categories (kernel messages, init messages, and
gettys), and I report whether they show up in the VNC console (PVFB), the serial
console (xvc), or both.

1.  Boot freshly installed RHEL-5 guest (/proc/cmdline = ro
root=/dev/VolGroup00/LogVol00 rhgb quiet): Because of quiet, only a few kernel
console messages are printed, both on the VNC console and on the serial console.
 All init messages are printed on VNC console, but not in serial console. 
Gettys are available on the VNC console, but not the serial console.

2.  Boot RHEL-5 guest (/proc/cmdline = ro root=/dev/VolGroup00/LogVol00 rhgb):
Many more kernel messages, all printed on both serial console and VNC console. 
Init messages printed on VNC console, not in serial console.  Gettys available
in VNC console, but not the serial console.

3.  Boot RHEL-5 guest (/proc/cmdline = ro root=/dev/VolGroup00/LogVol00): Kernel
messages are all printed on both serial console and VNC console.  Init messages
printed on VNC console, not in serial console.  Gettys available in VNC console,
but not in the serial console.

4.  Boot RHEL-5 guest (/proc/cmdline = ro root=/dev/VolGroup00/LogVol00
console=tty console=xvc): All kernel messages show up in both VNC console and
serial console.  Init messages printed on serial console, not in VNC console. 
Gettys available in both VNC console and serial console.

As a side test, I did another install with --nographics; in that case, anaconda
*automatically* adds "console=xvc0" to the kernel command line, which is why the
serial console works by default in that setup.

Markus, I believe this is the information you were looking for; if I missed
anything, please let me know.

Chris Lalancette

Thanks, Chris!

Your VNC console is known to the kernel as console tty.

Your serial console is known to the kernel as console xvc.

The preferred console (last on command line, or else first to register) is tty
(VNC console) in cases 1-3, and xvc (serial console) in case 4.

Now let me rearrange your data a bit:

* Kernel messages go to all registered consoles.  Works.

* Init messages go to the preferred console.  Works.

* getty runs on all consoles in case 4.  Works.

* getty runs only on console tty (VNC console) in cases 1-3.  Bug.  It should
run on console xvc (serial console) as well.

So the only bug here is the failure to run getty on console xvc (/dev/xvc0) in
cases 1-3.  Not a kernel-xen problem.  Could be a kudzu problem.

This is exactly how every other platform that we support works... a getty gets
set up on the primary console (be it the graphical console or a serial console
like xvc).  Setting up a getty on an available serial line could potentially be
a security problem in some circumstances.

The right thing to do here is to actually make virsh console connect to the
primary active console instead of pretending that the only thing that's a
"console" is the serial device.

I disagree - 'virsh console' is only intended to provide access to locally
accessible text based character devices exported from the guest, be they a
serial console, the Xen xvc console, a parport mapped to a PTY, or the QEMU
monitor mapped to a PTY. 

A graphical console does not fit into this model & will give two different UI
models - one inline in the controlling terminal, the other popping up a separate
(VNC) window. The virsh console command doesn't really extend into the remote
management space either.

What I think we really want is a standalone 'virt-console' program which can
provide a lightweight VNC console for those who don't want to run the full
virt-manager app. This can also transparently map the serial 'console'  into a
VTE widget - so whether using VNC or text PTY we can provide a reasonably
consistent graphical UI for the console. This needs to be done upstream first
though.

Re-attaching the Issue Tracker.

I see virt-viewer-0.0.2-2.el5 in Update 2, and it does provide a virtual
console viewer as suggested.

So I think this bug is fixed, by the current release of virt-viewer,

Daniel