Bug 892674

Summary: broken ncurses output after upgrading from F17 to F18
Product: [Fedora] Fedora Reporter: Jaromír Cápík <jcapik>
Component: ncursesAssignee: Miroslav Lichvar <mlichvar>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: high    
Version: rawhideCC: dickey, mlichvar, ovasik
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-01-30 08:44:56 EST Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Attachments:
Description Flags
curs_test.c none

Description Jaromír Cápík 2013-01-07 10:09:01 EST
Description of problem:
Hello. I recently received a report from Jim Warner (author of the procps-ng/top tool) where he experiences an issue with broken ncurses output in some circumstances. The issue was very probably introduced in ncurses-5.9.20120825.
The attached C source demonstrates the problem and contains useful notes about Jim's workaround.

I was able to reproduce the issue with:
ncurses-5.9-7.20121017.fc19.x86_64

I was unable to reproduce the issue with:
ncurses-5.9-4.20120204.fc17.x86_64

How reproducible:
always

Steps to Reproduce:
gcc -lncurses -o curs_test curs_test.c
./curs_test
  
Actual results:
line # 1line # 2line # 3line # 4line # 5line # 6line # 7line # 8line # 9line #10line #11line #12line #13line #14line #15line #16line #17line #18line #19line #20line #21line #22line #23line #24line #25line #26line #27line #28line #29line #30line #31line #32line #33line #34line #35line #36line #37line #38line #39line #40line #41line #42line #43line #44line #45line #46line #47line #48line #49line #50line #51line #52line #53line #54

Expected results:
line # 1
line # 2
line # 3
line # 4
...
...
line #51
line #52
line #53
Comment 1 Jaromír Cápík 2013-01-07 10:11:54 EST
Created attachment 674121 [details]
curs_test.c
Comment 2 Jaromír Cápík 2013-01-07 10:16:48 EST
Note: Jim confirmed the issue persists with the latest upstream sources ...
Comment 3 Thomas E. Dickey 2013-01-07 16:48:09 EST
The cause is from my changes here:

http://invisible-island.net/ncurses/NEWS.html#t20120825

	+ change output buffering scheme, using buffer maintained by ncurses
	  rather than stdio, to avoid problems with SIGTSTP handling (report
	  by Brian Bloniarz).

Mixing stdio calls with ncurses calls has not really ever been supported.
The terminfo layer has putp, which the example uses in one case, but uses
the stdio putchar in another.  The latter is buffered, the former currently
is not.  (Note that setupterm takes a file descriptor parameter, and there's
been no documentation asserting that it is buffered I/O). 

Since it is likely to be a rare application which can justify the mix,
I would expect the applications to do an fflush(stdout) before resuming
with ncurses.

The same issue will happen with other mixes of stdio and ncurses calls.
Comment 4 Thomas E. Dickey 2013-01-07 16:55:16 EST
For the given example, the straightforward solution would be to
use putp, and stay within the library's documented interfaces.
Comment 5 Thomas E. Dickey 2013-01-07 17:05:39 EST
hmm - the manpage claims that

       The  putp routine calls tputs(str, 1, putchar).  Note that
       the output of putp always  goes  to  stdout,  not  to  the
       fildes specified in setupterm.

However that (use of putchar) has not been true for some time.
I can look up that, if it's an issue - several years at any rate.

It's been an internal function owned by ncurses to solve
other signal-handling issues.  For SIGTSTP, there was no other
solution that I could see.

The 20120825 patch basically replaced the way buffering was done,
without introducing new I/O streams.
Comment 6 Jaromír Cápík 2013-01-08 08:08:45 EST
Hello Thomas.

This is unfortunately quite commonly seen scenario. Developers often build their software on top of undocumented features (because it somehow works) and then cry when such undocumented features stop working. Jim has seen your response and wrote a patch, that modifies the top tool to avoid mixing stdio and ncurses calls and feels satisfied about that. Anyway ... more tools like that could exist and if the previous undocumented behaviour can be easily achieved in ncurses without having a negative performance impact and if that doesn't go against your conviction, I would say it's more generous to do the change in ncurses. What do you think?

Thank you.

Regards,
Jaromir.
Comment 7 Thomas E. Dickey 2013-01-08 16:34:52 EST
At the moment I'm considering modifying ncurses so that putp and the
other low-level calls
   http://invisible-island.net/ncurses/man/curs_terminfo.3x.html

are buffered, while those from the ncurses library use the newer
scheme.  That would solve both problems (I think).
Comment 8 Jaromír Cápík 2013-01-09 08:12:55 EST
That's great. Thank you. J.
Comment 9 Thomas E. Dickey 2013-01-13 09:45:27 EST
no problem...

I made that change yesterday - 
http://lists.gnu.org/archive/html/bug-ncurses/2013-01/msg00007.html
Comment 10 Jaromír Cápík 2013-01-17 05:02:19 EST
Ahoj Mirku.

Do you think it would be possible to introduce the changes made by Thomas to F19/Rawhide?

Please, let me know.

Thanks,
Jaromir.
Comment 11 Miroslav Lichvar 2013-01-17 06:33:54 EST
Definitely, I'm planning to update the ncurses package soon.
Comment 12 Thomas E. Dickey 2013-01-27 06:37:42 EST
I made updates yesterday for this in mvcur (I noticed that dots_mvcur
test-program was still incorrect).  The other test-programs look ok,
though I'm thinking how to document the limitations of this hybrid
approach.
Comment 13 Miroslav Lichvar 2013-01-30 08:44:56 EST
ncurses-5.9-8.20130126.fc19 was built for rawhide. Thanks.