Bug 17204

Summary: /bin/sort broken in RH6.2 updates
Product: [Retired] Red Hat Linux Reporter: ellson
Component: textutilsAssignee: Bernhard Rosenkraenzer <bero>
Status: CLOSED NOTABUG QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 6.2CC: alecm, jakub, konrad, menthos, shishz
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2000-11-22 12:09:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
list of color names and rgb values none

Description ellson 2000-09-03 15:28:03 UTC
/bin/sort from textutils-2.0e-6.i386.rpm (in RH6.2 updates) is broken
and produces incorrect results.

This is probably the same problem as #10570, but the resolution should be
applied to RH6.2 updates because the bug is causing difficult-to-find
errors when sort is used in Makefiles.

textutils-2.0a-2.i386.rpm is OK (base RH6.2 release)
textutils-2.0e-7.i386.rpm is OK (RH6.9.5 release)

Sorting the attached "color_lib" file results incorrectly in
"red1" and "red2" before "red"

Since this problem doesn't seem to exist in textutils-2.0e-7.i386.rpm
I would guess that this is no longer a textutils problem, but the
textutils-2.0e-6.i386.rpm in RH6.2 Updates should be upgraded.

----------------------color_lib

purple 160 32 240
purple1 155 48 255
purple2 145 44 238
purple3 125 38 205
purple4 85 26 139
red 255 0 0
red1 255 0 0
red2 238 0 0
red3 205 0 0
red4 139 0 0
rosybrown 188 143 143
rosybrown1 255 193 193
rosybrown2 238 180 180
rosybrown3 205 155 155
rosybrown4 139 105 105

Comment 1 ellson 2000-09-03 15:29:47 UTC
Created attachment 3198 [details]
list of color names and rgb values

Comment 2 Bernhard Rosenkraenzer 2000-10-24 08:49:23 UTC
Fixed package built; waiting for QA approval.

Comment 3 Trond Eivind Glomsrxd 2000-10-24 13:54:05 UTC
Your package is broken. The problem seems to be that sort strips whitespace
before comparing when used with a non-C locale:

[dlt@test123 dlt]$ LC_ALL=en_US sort file2
red
red1
red 1
red12
red2
red 3
[dlt@test123 dlt]$ LC_ALL=C sort file2
red
red 1
red 3
red1
red12
red2
[dlt@test123 dlt]$

This may well be in glibc...

Comment 4 Jakub Jelinek 2000-10-24 13:58:31 UTC
What's wrong about that? That's the expected behaviour in most locales.
Take any vocabulary (be it english, german, czech or norwegian) and look
how entries are sorted.

Comment 5 ellson 2000-10-24 21:15:23 UTC
(reply to jakrub who doesn't seem convinced)

I don't see how this can be considered sorted?

red1 255 0 0
red2 238 0 0
red 255 0 0
red3 205 0 0
red4 139 0 0

In any case, the behaviour changed without explanation between releases, and
then changed back later.

Comment 6 Konrad J. Hambrick 2000-11-15 18:36:42 UTC
Just my $0.02 ...

I run the same script on RH5.2, RH6.1, Solaris 2.6 and
now RH 6.2.  The following is an intermediate file from
a departmentalized billing history report ( the algorithm
goes:  /bin/sh { gawk-gather -> sort -> gawk-format } )


######################## cut here ########################
~~~~~~~~|~~~~~|15102.65|1553.06|1341.51|453.09|214.58|0.00|18664.89
20001110|12|1145.67|151.02|11.29|34.37|7.96|0.00|1350.31
20001110|13|1142.50|87.41|116.76|34.28|34.54|0.00|1415.49
20001110|14|1212.31|92.74|126.77|36.37|65.61|0.00|1533.80
20001110|~~~~~|15102.65|1553.06|1341.51|453.09|214.58|0.00|18664.89
20001110|15|11186.17|1190.07|1082.30|335.59|74.04|0.00|13868.17
20001110|16|416.00|31.82|4.39|12.48|32.43|0.00|497.12
######################## cut here ########################

The data should come out as ( and does everywhere except on RH6.2 ) 
######################## cut here ########################
20001110|12|1145.67|151.02|11.29|34.37|7.96|0.00|1350.31
20001110|13|1142.50|87.41|116.76|34.28|34.54|0.00|1415.49
20001110|14|1212.31|92.74|126.77|36.37|65.61|0.00|1533.80
20001110|15|11186.17|1190.07|1082.30|335.59|74.04|0.00|13868.17
20001110|16|416.00|31.82|4.39|12.48|32.43|0.00|497.12
20001110|~~~~~|15102.65|1553.06|1341.51|453.09|214.58|0.00|18664.89
~~~~~~~~|~~~~~|15102.65|1553.06|1341.51|453.09|214.58|0.00|18664.89
######################## cut here ########################

Please help !

Thanks 

-- kjh

Comment 7 Konrad J. Hambrick 2000-11-15 18:45:17 UTC
Dooh !

Here are my particulars:

[konrad@dualie redhat]$ rpm -q textutils
textutils-2.0e-6

[konrad@dualie redhat]$ uname -a
Linux dualie.teamamerica.com 2.2.16-3smp #1 SMP Mon Jun 19 19:00:35 EDT 2000
i686 unknown

Sorry for the incomplete info.

-- kjh ( for now running /bin/sort from RH6.1 )

Comment 8 Konrad J. Hambrick 2000-11-15 19:12:03 UTC
Hmmm ... a clue ...

I just copied sort 2.0 from my RH6.1 laptop ( this
one works on my laptop ) onto a server running RH6.2 

Sort 2.0 is now broken on the RH6.2 server.

Sounds like a library issue to me.

I did make sort work by taking sort from a RH5.2 box

  [konrad@server1 konrad]$ sort --version
  sort (GNU textutils) 1.22

hth.

-- kjh

Comment 9 Alec Muffett 2000-11-22 12:09:34 UTC
It's not a library issue; I had installed all the updates 
(including textutils-2.0e-6) on a virgin RH6.2 machine, and
did:

( echo aa ; echo AB; echo ac; echo BA; echo bb ) | sort

...which should yield:

	AB
	BA
	aa
	ac
	bb

...but instead I got:
	$ rpm -q -f /bin/sort
	textutils-2.0e-6

	$ ( echo aa ; echo AB; echo ac; echo BA; echo bb ) |/bin/sort
	aa
	AB
	ac
	BA
	bb

...which is very VERY wrong (basically acting as if the "-f" flag was
permanently on) - and which breaks lots of things.

I worked around this by "rpm -Uvh --force" of the original textutils-2.0a-2
which is distributed with RH6.2; this gave me the expected behaviour again.

ergo: this is not a library issue.

This bug-in-the-update really *is* dangerous, as quite a lot of things have a
serious dependency upon the way that "sort" works by default.




Comment 10 Bernhard Rosenkraenzer 2000-11-22 12:16:13 UTC
Actually, this is the intended and POSIX compliant behavior.
If you don't like it, use LC_ALL=C and LC_COLLATE=C.

Comment 11 Konrad J. Hambrick 2000-11-22 15:01:49 UTC
alecm --

Thanks for the FB.

bero --

Why does the data file I submitted in message 2000-11-15 01:36
sort correctly everywhere I have tried it except on RH 6.2 ?

( everywhere includes RH5.2, RH6.1, Slack 7.0, Solaris 2.6,
  Solaris 2.7, SCO UnixWare and even on DOS )

Note that I, like alecm, have Installed all available patches
on the box ... unfortunately I did not try the programs before
installing the RH6.2 Updates.

IMNSHO, if the sort works the same everywhere but on RH6.2 with
all patches installed then sort is broken on RH6.2.

This will break 'lots' of programs for me, several that have been
running correctly for 5+ years on boxen without 'correct POSIX
behaviour'. 

Please do reconsider your assessment, POSIX be damned, sort is 
broken.

Thanks.

-- kjh

Comment 12 Alec Muffett 2000-11-22 15:11:46 UTC
I have a different take on the matter; I cannot check until tonight
when i get home after work, but I *believe* that I have no LC_* variables
set at all.

If that is the case, *then* i will consider the program to be broken
because I will not have opted-in to any bizzare locale-specific options.

More info when I have it.

	- alec


Comment 13 Konrad J. Hambrick 2000-11-27 15:57:36 UTC
All --

I checked my env on the RH6.2 box:

[konrad@sd_server konrad]$ env
LESSOPEN=|/usr/bin/lesspipe.sh %s
USERNAME=
HISTSIZE=1000
HOSTNAME=sd_server.xxx.com
LOGNAME=konrad
SSH_TTY=/dev/pts/1
MAIL=/var/spool/mail/konrad
TERM=xterm
HOSTTYPE=i386
PATH=/usr/bin:/bin:/usr/X11R6/bin:/home/local/bin:/home/konrad/bin
KDEDIR=/usr
PRINTER=rpt
HOME=/home/konrad
SHELL=/bin/bash
USER=konrad
LESS=-iXG -e -m -Pm ?f%f .?lbLine %lb of %L:?pb%pb\%:?bbByte %bb:-...
BASH_ENV=/home/konrad/.bashrc
QTDIR=/usr/lib/qt-2.1.0
LANG=en_US
SSH_CLIENT=XXX.XXX.XXX.XXX 63033 22
OSTYPE=Linux
SHLVL=1
_=/usr/bin/env

There are no LC_* env varbs set.

When I do set and export LC_ALL=C and LC_COLLATE=C, sort 
works as usual on the file I appended above.

Where is the proper place to set these varbs 
( /etc/profile ?  /etc/bashrc ? someplace else ? )

Thanks !

-- kjh

Comment 14 Konrad J. Hambrick 2000-12-14 00:09:23 UTC
bero --

Where is the proper place to set LC_ALL=C ; LC_COLLATE=C ?

Thanks.

-- kjh

Comment 15 Christian Rose 2001-11-15 11:11:38 UTC
This is a feature, not a bug. Broken scripts that expect the output of a certain
locale without setting it should get fixed.
Yes, the lack of docs on this is a problem, but a seperate problem.