Bug 1270480 - *fatal* sort broken: yet again!
*fatal* sort broken: yet again!
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: coreutils (Show other bugs)
22
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Ondrej Vasik
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-10 05:22 EDT by Ali Akcaagac
Modified: 2015-10-14 11:09 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-10-14 09:22:10 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
old.txt (9.66 KB, text/plain)
2015-10-10 05:22 EDT, Ali Akcaagac
no flags Details
new.txt (9.66 KB, text/plain)
2015-10-10 05:23 EDT, Ali Akcaagac
no flags Details
correct old.txt (9.66 KB, text/plain)
2015-10-10 05:48 EDT, Ali Akcaagac
no flags Details
correct new.txt (9.66 KB, text/plain)
2015-10-10 05:49 EDT, Ali Akcaagac
no flags Details

  None (edit)
Description Ali Akcaagac 2015-10-10 05:22:39 EDT
Created attachment 1081562 [details]
old.txt

It came to my attention, that coreutils 8.23-11 has a broken sort utility while coreutirls 8.23-9 has a working sort utility.

This is inacceptable. Please test your patches before applying them. Sorry for sounding a bit harsh here but *sort* is a core element and tons of scripts use sort to sort lists or pipe sort output that get's processed later on. The current sort totally misbehaves, gives wrong output and if you use the outout of the new sort to diff e.g. lists that has been done with old sort, then the result is - to say it in good words - fatal!

Please find 2 attachments:

old.txt -> coreutils 8.23-9
new.txt -> coreutils 8.23-11

diff -ruN old.txt new.txt > diff.txt

... or simply load in an editor and you will see.

You can also find in "new.txt" that

a
b
c
d
e
                      : e
f
g
h

...

... but the long spaced line with " : e" is not the correct sorting after e because it has a bunch of spaces and a colon infront of it.
Comment 1 Ali Akcaagac 2015-10-10 05:23 EDT
Created attachment 1081563 [details]
new.txt

Commands issued:

old coreutirls
sudo dnf search ios | LC_ALL=C sort > old.txt

new coreutirls
sudo dnf search ios | LC_ALL=C sort > new.txt
Comment 2 Ali Akcaagac 2015-10-10 05:48 EDT
Created attachment 1081580 [details]
correct old.txt
Comment 3 Ali Akcaagac 2015-10-10 05:49 EDT
Created attachment 1081581 [details]
correct new.txt
Comment 4 Pádraig Brady 2015-10-10 06:52:31 EDT
I can't see what is going on here sorry.

From the initial description you want the lines with initial spaces sorted first, which is what you should get with LC_ALL=C.
However in old.txt the lines with initial spaces do not sort first.
I thought old.txt is what you expected, so I'm confused.

Anyway the difference between the two sorting orders are due to
different LC_ALL settings for each sort run. Can you double check
that the env vars are set appropriately for each run.

Also I downloaded both 8.23-9.fc22 and 8.23-11.fc22
and didn't notice any difference between them for the attached inputs.
Comment 5 Pádraig Brady 2015-10-10 07:18:38 EDT
For completeness, note the setlocale() calls in sort(1) are not checked,
so if one system had de_DE installed while another didn't and fell
back to the "good" C locale, it might explain things?
Comment 6 Kamil Dudka 2015-10-12 05:55:28 EDT
Please re-test.
Comment 7 Ali Akcaagac 2015-10-14 09:22:10 EDT
Yes you are right! I have to apologize here for not looking closer to the real problem. The problem was not coreutils or wrong locales being used. It was because a previous installation of glibc (GNU C Library) that has shown up a few days before coreutils update caused a segmentation fault during update. I figured this out once I repeated and reviewed the steps for the past days of dnf updates. The locale file stored in /usr/lib/locale/locale-archive (not the template) was null bytes in size. I had to re-create that one. Now the ouput is as expected again - corresponding my language used. Therefore closing as *NOTABUG*
Comment 8 Pádraig Brady 2015-10-14 11:09:00 EDT
Note the setlocale() calls in sort(1) don't warn upon failure, silently falling back to the "C" locale. This is due to too many warnings in various cases of incorrectly setup locales. Now it could be argued that locales are central to sort's operation, and so it shold at least warn. As an interim measure at least, we've added a warning upon setlocale() failure when --debug is specified, in the not yet released version of coreutils.

Note You need to log in before you can comment on or make changes to this bug.