Bug 223112 - xenfb breaks /dev/tty1 and /dev/console ?
xenfb breaks /dev/tty1 and /dev/console ?
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: libvirt (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Daniel Veillard
Depends On:
Blocks: 234654 227613 240441 449099
  Show dependency treegraph
Reported: 2007-01-17 17:09 EST by Chris Lalancette
Modified: 2011-01-24 16:37 EST (History)
4 users (show)

See Also:
Fixed In Version: virt-viewer-0.0.2-2.el5
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-05-23 08:50:50 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
dmesg with default graphical install (15.59 KB, application/octet-stream)
2007-04-05 17:47 EDT, Archana K. Raghavan
no flags Details
Case 2- xm_dmesg-without-rhgb_quiet (15.46 KB, application/octet-stream)
2007-04-05 17:49 EDT, Archana K. Raghavan
no flags Details
Case 3- with "console=tty console=xvc" (15.07 KB, application/octet-stream)
2007-04-05 17:51 EDT, Archana K. Raghavan
no flags Details

  None (edit)
Description Chris Lalancette 2007-01-17 17:09:23 EST
Description of problem:

From an email I sent:

OK, here's what I found out:

1)  Using this command-line:

virt-install -n rhel5test -r 512 -f /var/lib/xen/images/rhel5test.dsk -s 10
--vnc -p -l nfs:bigpapi:/vol/engineering/redhat/rhel5-server-i386

and all of the default options during install, I installed RHEL5 fresh.
When the domain rebooted, it was exactly as you described to me; I could connect
to the console using "xm console <dom>", but I only got kernel messages up to
SELinux, and never saw INIT messages or got a login prompt.  Looking at
/proc/cmdline, I can see the domain was booted with "rhgb quiet".

2)  I then booted the same domain without "rhgb quiet".  I saw exactly the same
(kernel messages, but no INIT messages), but this time I actually got a login
prompt as well.

3)  I then booted the same domain with "console=tty console=xvc" (which is the
recommended commandline for getting serial console access).  With this command
line combination I saw all of the kernel messages, all of the INIT messages, and
I got a login console.

4)  As another test, I decided to try installing with no graphical console; that
is, I used this virt-install line:

virt-install -n rhel5test -r 512 -f /var/lib/xen/images/rhel5test.dsk -s 10
--nographics -p -l nfs:bigpapi:/vol/engineering/redhat/rhel5-server-i386

I then booted into the domain, and the console worked perfectly (that is, I saw
kernel messages, INIT messages, and got a login prompt).

So it seems to me the only case that doesn't work (that is, at *least* give a
login prompt) is the default "rhgb quiet" case with graphical console.

IBM is asking that we make it so that, by default (that is, with "rhgb quiet"),
you get a login prompt when connecting to the domain with "xm console <dom>". 
I'm not sure if this is appropriately filed against the xen component; feel free
to move it.

Chris Lalancette
Comment 2 Mark McLoughlin 2007-04-02 22:02:58 EDT
Okay, I think the expected behaviour here is:

  - With and without xenfb, xen's console driver impersonates /dev/tty[1-63]
    if no xencons= is specified on the cmdline, but only writes to /dev/tty1
    goes to the backend

  - Without console= or with console=tty xen's console driver makes 
    /dev/console equivalent to /dev/tty1

  - By default (i.e. without xencons=xvc specified), console=xvc is ignored

  - When you add xenfb into the equation, it creates a console but doesn't
    allocate a tty by virtue of the fact that the xen console driver sets
    console_use_vt = 0

  - The kernel writes its printk messages to all consoles, even ones without
    a tty driver allocated

  - We run a getty on /dev/tty[1-6] by default

  - The kernel exec()s init such that it writes to /dev/console

  - rhgb changes init's console to a pty which in turn displays inside X

So, to go through each of your observed behaviours:

  1) The fact that you don't see init messages makes sense, you should be
     seeing them in the VNC console. There's no reason why you shouldn't
     be seeing a login prompt - /dev/tty1 should be hooked up to the backend
     and we should be running a getty on /dev/tty1

  2) Bizarre. rhgb now isn't around to change the console, so init should
     be writing to /dev/console and those writes should be going to the
     console backend. The lack of a login prompt is just as strange as
     in (1).

3/4) Here you're booting without xenfb and seeing both init and the login
     process is visible.

One possible conclusion is that by loading xenfb, /dev/tty1 and /dev/console are
now going elsewhere. That would explain both (1) and (2) ... cc-ing Markus, this
could well be a known/old/fixed bug.

Chris, some more info would be useful:

  - What version of kernel-xen was this?

  - Is it still reproducible with the latest kernel?

  - dmesg logs from 1, 2 and 3 might shed some light on this
Comment 3 Mark McLoughlin 2007-04-03 02:49:47 EDT
Hmm, I misread something ...

>   2) Bizarre. rhgb now isn't around to change the console, so init should
>      be writing to /dev/console and those writes should be going to the
>      console backend. The lack of a login prompt is just as strange as
>      in (1).

It's pretty clear that we were getting a login prompt in this case, which is
very strange - so /dev/tty1 is working, but not /dev/console, and rhgb is
somehow the influencing factor  ?
Comment 4 Markus Armbruster 2007-04-05 05:56:49 EDT
Console tty is the Linux virtual console.  It's accessible through
xenfb, when that's enabled.  Device files are /dev/tty[0-9]* (major 4)
and /dev/{tty,console,ptmx} (major 5).

Console xvc is the xen console, accessible through xm console.  Device
file is /dev/xvc0 (major 204, minor 191).  It can be made to mug the
real tty (xencons=tty), but that's a bad idea.  In fact, this
misfeature is currently *broken* in our kernels.  That's because the
upstream changeset that fixes it breaks stuff we actually care about.

Where init runs getty is outside the area of my expertise.  On my FC-6
system, I get a getty on /dev/xvc0 (i.e. in xm console) regardless of
kernel parameters, with and without xenfb.

Kernel messages (printks) go to all registered consoles with flag
CON_ENABLED set.  I get them in both xvc (xm console) and tty (xenfb).

Output to /dev/console goes to the registered console that is
specified last on the command line, if any.  Else it goes to the first
console that registers, which is tty.  The kernel is not prepared to
copy the output to more than one console.

User space can redirect /dev/console to a file (ioctl TIOCCONS).  This
could conceivably be used to tee console output to multiple places.  I
expect that to play badly with existing uses of TIOCCONS.
Comment 5 Mark McLoughlin 2007-04-05 11:46:35 EDT
Ah, I see this is all different in RHEL-5 from the rawhide kernel I was looking
at - xencons=xvc is the default and kudzu has code for adding a getty for
/dev/xvc0, so ...

Chris: we really need an update from you on whether this is all still just as
broken with the latest RHEL-5 kernel ...

Re-assigning to Markus
Comment 6 Archana K. Raghavan 2007-04-05 12:32:01 EDT
I am testing this out right now...
Comment 7 Archana K. Raghavan 2007-04-05 17:45:36 EDT
I used the following command to install a xen guest:
#virt-install -n rhel5test1 -r 512 -f /var/lib/xen/images/rhel5test1.dsk -s 10
--vnc -p -l

[root@x460 xen]# uname -a
Linux x460.gsslab.rdu.redhat.com 2.6.18-8.1.1.el5xen #1 SMP Mon Feb 26 20:51:53
EST 2007 x86_64 x86_64 x86_64 GNU/Linux

It is still reproducible. I am not able to launch the console:
[root@x460 xen]# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0    13121    16 r-----    733.6
rhel5test1                                 4      511     1 -b----     26.2
[root@x460 xen]# xm console rhel5test1
xenconsole: Could not read tty from store: No such file or directory
[root@x460 xen]#

If I remove the rhgb quiet, then I get a login prompt, but I do not see any
kernel messages. If I have  "console=tty console=xvc" in the kernel line then I
see the INIT messages and the login prompt.

I am also attaching the dmesg from cases 1, 2 and 3

Comment 8 Archana K. Raghavan 2007-04-05 17:47:50 EDT
Created attachment 151811 [details]
dmesg with default graphical install
Comment 9 Archana K. Raghavan 2007-04-05 17:49:13 EDT
Created attachment 151812 [details]
Case 2- xm_dmesg-without-rhgb_quiet
Comment 10 Archana K. Raghavan 2007-04-05 17:51:10 EDT
Created attachment 151813 [details]
Case 3- with "console=tty console=xvc"
Comment 11 Markus Armbruster 2007-04-06 10:13:41 EDT
Comments #1 and #7 do not make it perfectly clear to me whether case (3) has
rhgb quiet.  Please advise.
Comment 12 Markus Armbruster 2007-04-06 10:28:51 EDT
For all cases where `INIT messages' are not visible in xm console: do they show
up in xenfb?  If you can't connect your VNC viewer quickly enough to see all the
action, try SDL.
Comment 14 Chris Lalancette 2007-04-23 09:57:45 EDT
OK, here we go again.  Based on a conversation with Markus on IRC, here's the
data from another experiment booting a RHEL-5 paravirt guest.  In the following,
I have 3 different message categories (kernel messages, init messages, and
gettys), and I report whether they show up in the VNC console (PVFB), the serial
console (xvc), or both.

1.  Boot freshly installed RHEL-5 guest (/proc/cmdline = ro
root=/dev/VolGroup00/LogVol00 rhgb quiet): Because of quiet, only a few kernel
console messages are printed, both on the VNC console and on the serial console.
 All init messages are printed on VNC console, but not in serial console. 
Gettys are available on the VNC console, but not the serial console.

2.  Boot RHEL-5 guest (/proc/cmdline = ro root=/dev/VolGroup00/LogVol00 rhgb):
Many more kernel messages, all printed on both serial console and VNC console. 
Init messages printed on VNC console, not in serial console.  Gettys available
in VNC console, but not the serial console.

3.  Boot RHEL-5 guest (/proc/cmdline = ro root=/dev/VolGroup00/LogVol00): Kernel
messages are all printed on both serial console and VNC console.  Init messages
printed on VNC console, not in serial console.  Gettys available in VNC console,
but not in the serial console.

4.  Boot RHEL-5 guest (/proc/cmdline = ro root=/dev/VolGroup00/LogVol00
console=tty console=xvc): All kernel messages show up in both VNC console and
serial console.  Init messages printed on serial console, not in VNC console. 
Gettys available in both VNC console and serial console.

As a side test, I did another install with --nographics; in that case, anaconda
*automatically* adds "console=xvc0" to the kernel command line, which is why the
serial console works by default in that setup.

Markus, I believe this is the information you were looking for; if I missed
anything, please let me know.

Chris Lalancette
Comment 15 Markus Armbruster 2007-04-23 10:54:29 EDT
Thanks, Chris!

Your VNC console is known to the kernel as console tty.

Your serial console is known to the kernel as console xvc.

The preferred console (last on command line, or else first to register) is tty
(VNC console) in cases 1-3, and xvc (serial console) in case 4.

Now let me rearrange your data a bit:

* Kernel messages go to all registered consoles.  Works.

* Init messages go to the preferred console.  Works.

* getty runs on all consoles in case 4.  Works.

* getty runs only on console tty (VNC console) in cases 1-3.  Bug.  It should
run on console xvc (serial console) as well.

So the only bug here is the failure to run getty on console xvc (/dev/xvc0) in
cases 1-3.  Not a kernel-xen problem.  Could be a kudzu problem.
Comment 16 Jeremy Katz 2007-04-23 11:05:28 EDT
This is exactly how every other platform that we support works... a getty gets
set up on the primary console (be it the graphical console or a serial console
like xvc).  Setting up a getty on an available serial line could potentially be
a security problem in some circumstances.

The right thing to do here is to actually make virsh console connect to the
primary active console instead of pretending that the only thing that's a
"console" is the serial device.
Comment 17 Daniel Berrange 2007-04-23 11:24:43 EDT
I disagree - 'virsh console' is only intended to provide access to locally
accessible text based character devices exported from the guest, be they a
serial console, the Xen xvc console, a parport mapped to a PTY, or the QEMU
monitor mapped to a PTY. 

A graphical console does not fit into this model & will give two different UI
models - one inline in the controlling terminal, the other popping up a separate
(VNC) window. The virsh console command doesn't really extend into the remote
management space either.

What I think we really want is a standalone 'virt-console' program which can
provide a lightweight VNC console for those who don't want to run the full
virt-manager app. This can also transparently map the serial 'console'  into a
VTE widget - so whether using VNC or text PTY we can provide a reasonably
consistent graphical UI for the console. This needs to be done upstream first
Comment 18 Konrad Rzeszutek 2007-05-18 16:26:03 EDT
Re-attaching the Issue Tracker.
Comment 22 Daniel Veillard 2008-05-23 08:50:50 EDT
I see virt-viewer-0.0.2-2.el5 in Update 2, and it does provide a virtual
console viewer as suggested.

So I think this bug is fixed, by the current release of virt-viewer,


Note You need to log in before you can comment on or make changes to this bug.