Bug 733561 - "sha256sum: WARNING: 20 lines are improperly formatted" when using GPG signed checksum file
Summary: "sha256sum: WARNING: 20 lines are improperly formatted" when using GPG signed...
Alias: None
Product: Fedora
Classification: Fedora
Component: coreutils
Version: rawhide
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Ondrej Vasik
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2011-08-26 02:24 UTC by Andre Robatino
Modified: 2012-11-30 04:10 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2011-10-20 09:00:15 UTC
Type: ---

Attachments (Terms of Use)
bash script to deal with gpg-signed checksum files (874 bytes, text/plain)
2012-11-30 04:10 UTC, D. Hugh Redelmeier
no flags Details

Description Andre Robatino 2011-08-26 02:24:19 UTC
Description of problem:
[andre@compaq-pc i386]$ sha256sum -c Fedora-15-i386-CHECKSUM
sha256sum: Fedora-15-i386-DVD.iso: No such file or directory
Fedora-15-i386-DVD.iso: FAILED open or read
sha256sum: Fedora-15-i386-netinst.iso: No such file or directory
Fedora-15-i386-netinst.iso: FAILED open or read
sha256sum: WARNING: 20 lines are improperly formatted
sha256sum: WARNING: 2 listed files could not be read
[andre@compaq-pc i386]$

The line "sha256sum: WARNING: 20 lines are improperly formatted" comes from the signature. This only happens with F15's coreutils-8.10-2.fc15, not with F14's coreutils-8.5-7.fc14, F16's coreutils-8.12-2.fc16, or Rawhide's coreutils-8.12-6.fc17. Not a big deal, just a little disturbing. The signature itself is fine and this happens with either recent or old signature files, so it's purely a change in coreutils. Will F15's coreutils be updated to a version that gets rid of this?

Version-Release number of selected component (if applicable):

Comment 1 Ondrej Vasik 2011-08-26 05:43:10 UTC
Thanks for report, so just to be sure - you used various sha256sum utilities on the same checksum file?

I think coreutils-8.5-7.fc14 is without warning, because it was added 6 months after the release of 8.5 by upstream commit http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=407806b27f032b3bce828ee0a152b5f67e997e09 ... however it doesn't explain why the coreutils-8.12 doesn't print the warnings, I'm not aware of any relevant change in this area... but will check this further ...

Comment 2 Andre Robatino 2011-08-26 05:56:20 UTC
(In reply to comment #1)
> Thanks for report, so just to be sure - you used various sha256sum utilities on
> the same checksum file?

/usr/bin/sha256sum from coreutils is the only one I normally use - I'm not sure what other utilities you're talking about. The checksum files I tested were the standard Fedora ones, and as mentioned before, it didn't matter which ones I used.

Comment 3 Ondrej Vasik 2011-08-26 12:56:09 UTC
I checked the f15, f16 and f17 version of coreutils and in all these versions sha256sum shows these warnings... simply these files are not in correct format for sha256sum - as it expects only checksums and filenames (and no GPG check lines). You could verify this with f14 sha256sum with -w option - it will show warnings for every single gpgkey line.

Maybe this "warning summary" could be suppressed if no -w is specified. I'll ask the upstream ... anyway - I still don't understand why sha256sum from f16 and f17 coreutils package is not showing these messages on your machine - there is something suspicious...

Comment 4 Andre Robatino 2011-08-26 19:11:40 UTC
I apologize - I was certain that it didn't appear in 16 and Rawhide, but upon retesting, I see the warning in both of them. Don't know how I made that mistake (maybe I inadvertently looked at an unsigned file?). Anyway, the warning appears in 15, 16, and Rawhide.

I note that in the 16 sha256sum man page, it still says

      -w, --warn
              warn about improperly formatted checksum lines

The new behavior appears to be that -w adds an individual warning for each improperly formatted line. You always get a one-line warning giving the total number of unformatted lines, regardless of whether -w is used. So the explanation for -w should be clarified. Personally I think it would make more sense to suppress the warning summary by default, as you suggested. That way the man page could stay the same.

Also, if the one-line warning continues to exist regardless of whether -w is used, it may require minor tweaks in Fedora's documentation regarding verifying signed checksum files, since people may see the warning and think something is wrong.

Comment 5 Ondrej Vasik 2011-08-26 19:44:50 UTC
Thanks for confirmation. Suppressing this warning by default is oneliner patch and it seems to be the best option so far - but I'll wait for upstream opinion (maybe there is some reason for showing this by default).

Comment 6 Andre Robatino 2011-10-19 15:35:59 UTC
Any news on this? (Since F16 release is close.)

Comment 7 Ondrej Vasik 2011-10-19 16:46:33 UTC
Unfortunately no reply from Jim via email (sent Aug 26th) ... I'll ask him directly here.

Jim, what do you think? Could you please change the default behaviour back? ( so summary for n_misformatted_lines will be only printed when warn flag from -w short option is active. )

Comment 8 Jim Meyering 2011-10-20 08:35:15 UTC
(In reply to comment #7)
> Jim, what do you think? Could you please change the default behaviour back? (
> so summary for n_misformatted_lines will be only printed when warn flag from -w
> short option is active. )

Hi Ondrej,

The rationale for adding that warning is that an improperly formatted
(unrecognized) checksum line can represent a serious problem, potentially
security-relevant, so the existence of any non-checksum line is worth
warning about.

If you do not want such a warning, please consider stripping the gpg
clear-sign wrapper lines before invoking sha*sum -c.  You can do that
as part of the verification phase:

  gpg -d SHA256SUM > bare-checksums || report-gpg-sig-mismatch
  sha256sum -c bare-checksums || report-checksum-mismatch

Or, (slightly risky) for SHA256SUM, you could use this:

  grep -E '^[[:xdigit:]]{64}  [^ ]' SHA256SUM | sha256sum -c

Comment 9 Ondrej Vasik 2011-10-20 09:00:15 UTC
Thanks for explanation and hints, Jim.
Based on Jim's comment closing that one NOTABUG - it is intentional upstream change in behaviour due to concerns about ignored serious, potentially security problems.

Comment 10 D. Hugh Redelmeier 2012-11-30 04:08:38 UTC
This state of affairs is embarrassing.  People following <https://fedoraproject.org/en/verify> get hit with error messages that don't actually indicate a problem that they should know about -- very confusing as ones first exposure to Fedora.  In the process, any diagnostic that they should pay attention to might be ignored.

Furthermore, the verification is unnecessarily complicated: it involves two arcane steps (with an arbitrary tempfile, if done right, but that is not documented on the verify page).

The fix that makes the most sense to me is that sha256sum should know about gpg and be willing to invoke it.  But I can see that that fix might be resisted.

So, I propose that attached script "check-signed-checksums" (possibly named something else) be adopted and the verify page be simplified by using it.  This script takes as arguments signed checksum files.  It applies gpg to each and then passes on the bare checksum file to sha256sum.  I've tried to make it bullet-proof but I may have missed something.

Perhaps this is a request for enhancement.

Comment 11 D. Hugh Redelmeier 2012-11-30 04:10:24 UTC
Created attachment 654782 [details]
bash script to deal with gpg-signed checksum files

Note You need to log in before you can comment on or make changes to this bug.