Bug 1149651 - Carriage return instead of line break when entering long command in serial console
Summary: Carriage return instead of line break when entering long command in serial co...
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: bash
Version: 25
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
Assignee: Siteshwar Vashisht
QA Contact: BaseOS QE - Apps
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-10-06 11:45 UTC by Martin Frodl
Modified: 2017-12-12 10:14 UTC (History)
12 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-12 10:14:48 UTC
Type: Bug
Embargoed:
kdudka: needinfo-


Attachments (Terms of Use)

Description Martin Frodl 2014-10-06 11:45:53 UTC
Description of problem:

When entering a command which is long enough not to fit on a single line, bash should perform a line break and continue printing the characters on the following line. This is not the case when bash is run over serial console, though. Instead, the 'overflowed' characters are displayed at the beginning of the very same line.

In case I have not made myself quite clear, here goes a simple illustration. I am assuming the width of the serial console is 80 characters. What bash normally displays in this way...

[root@rhel7vm ~]# yum install java-1.6.0-openjdk java-1.6.0-openjdk-demo java-1.6.0-openjdk-devel

...is displayed like this in serial console:

6.0-openjdk-devel yum install java-1.6.0-openjdk java-1.6.0-openjdk-demo java-1.


Version-Release number of selected component (if applicable):
bash-4.2.45-5.el7.x86_64

How reproducible:
100%

Steps to Reproduce:

1. Connect to a RHEL7 machine over serial console. The easiest way is probably to run a (properly configured) virtual machine and use virsh to connect to it.

2. Enter an arbitrary command (or any other string, for that matter) long enough not to fit on a single line. 

Actual results:
After reaching the end of line, the subsequent characters appear at the beginning of the very same line ('\r'-like behaviour).

Expected results:
They should appear on the next line ('\n'-like behaviour).

Comment 3 Ondrej Oprala 2015-04-28 09:00:02 UTC
Hmm, I'm not entirely sure this is caused by bash, could you please dump
your 'stty -a' output, when connected to the virtual machine?

Thanks

Comment 4 Martin Frodl 2015-04-28 10:41:49 UTC
Thanks for asking. I noticed the described behaviour only takes place in serial consoles whose width is not exactly 80 (both wider and narrower terminals make troubles). I am still convinced this is a bash bug, though -- when a long command is entered in ksh, line breaking works correctly.

Here is the 'stty -a' output if it helps:

[root@localhost ~]# stty -a
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon ixoff
-iuclc -ixany -imaxbel iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon -iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
-echoctl echoke

Comment 5 Ondrej Oprala 2015-08-18 08:39:07 UTC
Is that the stty -a output of the host or the machine? I have a similar bug reported, where the behaviour is exactly opposite, so I'm wondering...

Comment 8 Siteshwar Vashisht 2016-05-23 16:28:38 UTC
Carriage return is added by below code in _rl_move_cursor_relative() function in bash :

1975   if (dpos == 0 || CR_FASTER (dpos, _rl_last_c_pos) ||
1976       (_rl_term_autowrap && i == _rl_screenwidth))
1977     {
1978 #if defined (__MSDOS__)
1979       putc ('\r', rl_outstream);
1980 #else
1981       tputs (_rl_term_cr, 1, _rl_output_character_function);
1982 #endif /* !__MSDOS__ */
1983       cpos = _rl_last_c_pos = 0;
1984     }


_rl_term_autowrap is set to true when both 'am' and 'xn' flags are set for a terminal :

516   _rl_term_autowrap = tgetflag ("am") && tgetflag ("xn");

So it looks like even though both "am" and "xn" flags are set for serial terminal, serial terminal actually do no support automatic line wrapping.

Comment 9 Siteshwar Vashisht 2016-05-24 12:03:06 UTC
Reassigning to systemd.

This issue is happening because systemd sets terminal type to 'vt220' while initializing serial terminals :

4027 const char *default_term_for_tty(const char *tty) {
4028         assert(tty);
4029 
4030         return tty_is_vc_resolve(tty) ? "TERM=linux" : "TERM=vt220";
4031 }

I believe systemd should specifically check for serail terminal devices and use 'serial' terminal type instead of 'vt220' for serial terminals.

Investigation of similar issue: https://bugzilla.redhat.com/show_bug.cgi?id=1252764#c10

Comment 11 Siteshwar Vashisht 2016-05-25 05:07:43 UTC
Upstream bug: https://github.com/systemd/systemd/issues/3342

Comment 12 Siteshwar Vashisht 2016-05-27 09:21:37 UTC
Transferring bug to kernel and assigning to skozina for further investigation of tty driver.

Comment 13 Stanislav Kozina 2016-06-03 10:47:03 UTC
After spending some more time on this I think it's quite clear.
Bash uses libtinfo to figure out if current $TERM supports line wrap:

    516   _rl_term_autowrap = tgetflag ("am") && tgetflag ("xn"); 

libtinfo only finds the right values for $TERM, for TERM=vt200 both 'am' and 'xn' flags are set.

The line wrap actually works fine (for me) *if* the terminal on the host side is not resized. In other words, following ioctl() always returns the original size of the window (for me) even after I resize the terminal on the host side:

ioctl(0, TIOCGWINSZ, &w);

I'm using libvirt which sets virtio-serial as the serial console. This is my qemu command and component versions:

2016-06-03 09:06:31.653+0000: starting up libvirt version: 1.2.18.2, package: 3.fc23 (Fedora Project, 2016-03-17-21:40:48, buildvm-22.phx2.fedoraproject.org), qemu version: 2.4.1 (qemu-2.4.1-8.fc23)
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/bin/qemu-kvm -name rhel7 -S -machine pc-i440fx-2.3,accel=kvm,usb=off -cpu Nehalem -m 1024 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid bd338edd-cbfd-4213-a89f-7cfe2fcde608 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/rhel7.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/var/lib/libvirt/images/rhel7.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-0-0,readonly=on,format=raw -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:15:7a:0d,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/rhel7.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on
char device redirected to /dev/pts/2 (label charserial0)

That said the virtio-console *can* support windows resize via the VIRTIO_CONSOLE_RESIZE event which is supported if the device supports VIRTIO_CONSOLE_F_MULTIPORT feature. I think this is not the case, so changing component to qemu-kvm. This is the device as seen in the guest:

00:05.0 Communication controller: Red Hat, Inc Virtio console
	Subsystem: Red Hat, Inc Device 0003
	Physical Slot: 5
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 10
	Region 0: I/O ports at c080 [size=32]
	Region 1: Memory at fc057000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [40] MSI-X: Enable+ Count=2 Masked-
		Vector table: BAR=1 offset=00000000
		PBA: BAR=1 offset=00000800
	Kernel driver in use: virtio-pci
00: f4 1a 03 10 07 05 10 00 00 00 80 07 00 00 00 00
10: 81 c0 00 00 00 70 05 fc 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 03 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 00 00
40: 11 00 01 80 01 00 00 00 01 08 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Please, can we get the VIRTIO_CONSOLE_RESIZE messages work?

Comment 14 Stanislav Kozina 2016-06-03 10:48:24 UTC
In the text above please s/vt200/vt220/ for TERM. Thanks!

Comment 15 Dr. David Alan Gilbert 2016-06-17 09:14:25 UTC
Hi,
  I can repeat this in a little f20 image I've got here with serial console.

  VIRTIO_CONSOLE_RESIZE won't work because in this case you're using a plain old ISA serial console that just behaves the same as decades old PC serial ports - so there's no way of communicating VIRTIO_CONSOLE_RESIZE down that.

  Now, you could try using virtio-console, but it's generally not preferred.

My reading of this is that the problem is the rows/columns size isn't set - as seen in comment 4.

If I set that manually then this bug seems to go away, so I think the challenge is trying to figure out how to get rows/columns set properly.

There's one thing I found works and another that doesn't work and we should understand why:

  a) The thing that works is to do the following in the guest:
          eval $(resize) 

     - that's using the resize command that's in the xterm package;  if you add that to your .bash_profile on the guest it should run it at login.

   b) bash has a shell option 'checkwinsize' and my reading is that's set by default in an interactive shell after login; this is supposed to read the size after every command to update the ROWS/COLUMNS sizes;  why isn't this solving the problem for us?

Ondrej: Can you explain why checkwinsize isn't doing this?

Dave

Comment 16 Dr. David Alan Gilbert 2016-06-17 09:22:55 UTC
Oops, looks like Ondrej has gone; Siteshwar - can you offer any idea of why checkwinsize doesn't work?

Comment 17 Laszlo Ersek 2016-06-17 23:01:36 UTC
Dave: my guess is that bash doesn't do the ugly trick that xterm's resize does :)

http://blog.sam.liddicott.com/2015/02/finding-xterm-terminal-window-size-for.html

> The string emitted [...] instructs the xterm to save the current cursor
> position, move the cursor to 999,999 which should be beyond the terminal
> bounds and so instead move to the bottom right corner; it then instructs the
> xterm to report the current cursor position, and then restore the previous
> cursor position.

Whereas from bash's "lib/sh/winsize.c":

void
get_new_window_size (from_sig, rp, cp)
     int from_sig;
     int *rp, *cp;
{
#if defined (TIOCGWINSZ)
  struct winsize win;
  int tty;

  tty = input_tty ();
  if (tty >= 0 && (ioctl (tty, TIOCGWINSZ, &win) == 0) &&
      win.ws_row > 0 && win.ws_col > 0)
    {
      sh_set_lines_and_columns (win.ws_row, win.ws_col);
#if defined (READLINE)
      rl_set_screen_size (win.ws_row, win.ws_col);
      if (rp)
        *rp = win.ws_row;
      if (cp)
        *cp = win.ws_col;
#endif
    }
#endif
}

So, bash's checkwinsize is based entirely on TIOCGWINSZ returning positive row and column numbers. Unfortunately, those numbers are zero when the ioctl() is called on ttyS0. The stty utility calls the same ioctl() -- I straced it --, and stty -a reports:

rows 0; columns 0;

Instead, xterm's "resize" utility exchanges information with xterm itself, using special escape sequences, and brings that information into the shell environment.

As the linked blog post shows however, the same can be done with a very clever PS1 prompt.

Comment 18 Dr. David Alan Gilbert 2016-06-20 08:17:47 UTC
(In reply to Laszlo Ersek from comment #17)
> Dave: my guess is that bash doesn't do the ugly trick that xterm's resize
> does :)
> 
> http://blog.sam.liddicott.com/2015/02/finding-xterm-terminal-window-size-for.
> html
> 
> > The string emitted [...] instructs the xterm to save the current cursor
> > position, move the cursor to 999,999 which should be beyond the terminal
> > bounds and so instead move to the bottom right corner; it then instructs the
> > xterm to report the current cursor position, and then restore the previous
> > cursor position.

Ah, that explains something I'd always noticed - resize always causes the cursor to flicker in an odd way :-)

> Whereas from bash's "lib/sh/winsize.c":
> 
> void
> get_new_window_size (from_sig, rp, cp)
>      int from_sig;
>      int *rp, *cp;
> {
> #if defined (TIOCGWINSZ)
>   struct winsize win;
>   int tty;
> 
>   tty = input_tty ();
>   if (tty >= 0 && (ioctl (tty, TIOCGWINSZ, &win) == 0) &&
>       win.ws_row > 0 && win.ws_col > 0)
>     {
>       sh_set_lines_and_columns (win.ws_row, win.ws_col);
> #if defined (READLINE)
>       rl_set_screen_size (win.ws_row, win.ws_col);
>       if (rp)
>         *rp = win.ws_row;
>       if (cp)
>         *cp = win.ws_col;
> #endif
>     }
> #endif
> }
> 
> So, bash's checkwinsize is based entirely on TIOCGWINSZ returning positive
> row and column numbers. Unfortunately, those numbers are zero when the
> ioctl() is called on ttyS0. The stty utility calls the same ioctl() -- I
> straced it --, and stty -a reports:
> 
> rows 0; columns 0;
> 
> Instead, xterm's "resize" utility exchanges information with xterm itself,
> using special escape sequences, and brings that information into the shell
> environment.
> 
> As the linked blog post shows however, the same can be done with a very
> clever PS1 prompt.

OK; so that explains why bash isn't doing it.

Right; so I'd advise against the shell prompt stuff; it is noticeable, but I think that a call to resize in the bash_login is appropriate - especially if you can tell you're on a serial port.
It's probably worth asking the xterm guys nicely so that resize shouldn't be in their package and then we dont end up adding a dependency/

Dave

Comment 20 Amit Shah 2016-08-03 06:39:00 UTC
So this doesn't have anything to do with the virtio-serial device, nor with virtualization.  Re-assigning to bash.  As Dave mentions in comment 18, this could be handled in xterm too.

Comment 21 Siteshwar Vashisht 2016-11-11 10:44:55 UTC
This can not be considered a bug with bash. As comment #8 mentions, bash checks for specific flags to verify if the terminal supports auto line wrapping. If serial terminals do not support line wrapping, these flags should not be set.

Comment 22 Dr. David Alan Gilbert 2016-11-11 12:10:32 UTC
+Siteshwar: I think this is fixable by making a call to the 'resize' program (currnetly in the xterm package) in something that gets run at bash startup I'm not sure if that means /etc/skel/.bash_profile or /etc/skel/.bashrc - those are owned by bash.

Comment 23 Kamil Dudka 2016-11-11 12:40:30 UTC
bash cannot depend on xterm and hacking bash profile scripts to work around the bug (which is not in bash anyway) could have side effects inappropriate for an update in RHEL.

Comment 24 Dr. David Alan Gilbert 2016-11-11 13:07:42 UTC
As I remember calling resize was standard practice on serial and remote terminals, even when I was using Sun systems ~25 years ago - it's the normal way to get the right terminal size.

I'd agree that it shouldn't be in the xterm package, but that sounds like it should be fixed separately from using it in a standard script.

I don't think anyone has actually identified a 'bug' in any one package anywhere in the 2 years and ~4 packages that this bz has been through.

Comment 25 Kamil Dudka 2016-11-11 13:38:06 UTC
resize is documented as xterm-specific.  Are you implying it is more than that?  In any case this needs to fixed in Fedora/upstream first.  We are not going to experiment with such workarounds in RHEL-7.

Comment 26 Dr. David Alan Gilbert 2016-11-11 13:44:49 UTC
(In reply to Kamil Dudka from comment #25)
> resize is documented as xterm-specific.  Are you implying it is more than
> that?

My understanding was that it's an ANSI escape sequence that should work on most terminal types (i.e. all ANSI/vt100 like ones)

>  In any case this needs to fixed in Fedora/upstream first.  We are not
> going to experiment with such workarounds in RHEL-7.

Yeh that's reasonable.  I'm just wondering if any old version of RHEL used to do this.

Comment 27 Siteshwar Vashisht 2016-11-11 14:07:26 UTC
I would prefer to document any such workarounds rather than trying to "fix" it in bash.

Comment 28 Kamil Dudka 2016-11-11 14:17:36 UTC
(In reply to Dr. David Alan Gilbert from comment #26)
> I'm just wondering if any old version of RHEL used to do this.

Not that I know of.  We added 'shopt -s checkwinsize' to /etc/bashrc in 2002 but, as you know, it does not solve the problem on its own:

http://pkgs.fedoraproject.org/cgit/rpms/setup.git/commit/?id=0d9c43cd

Comment 29 Dr. David Alan Gilbert 2016-11-16 11:31:59 UTC
I created https://bugzilla.redhat.com/show_bug.cgi?id=1395661 in Fedora for resize to be moved somewhere other than the xterm package;  if anyone has any good ideas where it might be an idea to add a comment.

Comment 30 Dr. David Alan Gilbert 2016-11-23 20:04:34 UTC
bz 1349582 split resize out of the xterm package in Fedora 25; it's now in the xterm-resize package.

Comment 31 Fedora End Of Life 2017-11-16 18:55:07 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 32 Fedora End Of Life 2017-12-12 10:14:48 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.