This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours
Bug 733448 - abrt refuses to attach anaconda crash dump to automated anaconda bug reports
abrt refuses to attach anaconda crash dump to automated anaconda bug reports
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: abrt (Show other bugs)
16
All Linux
unspecified Severity medium
: ---
: ---
Assigned To: Jiri Moskovcak
Fedora Extras Quality Assurance
AcceptedBlocker
:
Depends On:
Blocks: F16Beta/F16BetaBlocker
  Show dependency treegraph
 
Reported: 2011-08-25 14:06 EDT by David Lehman
Modified: 2015-02-01 17:54 EST (History)
12 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-10-03 12:04:06 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
example anaconda traceback file (366.46 KB, text/plain)
2011-08-30 09:44 EDT, David Lehman
no flags Details

  None (edit)
Description David Lehman 2011-08-25 14:06:39 EDT
Description of problem:
When filing an automated bug report after an anaconda crash, abrt will not allow me to select anaconda-tb-XXXXXX file for inclusion in the report, which is the single most important piece of information needed for an effective report.

Version-Release number of selected component (if applicable):

F16 Alpha RC5
Comment 1 Denys Vlasenko 2011-08-30 06:41:57 EDT
anaconda-tb-XXXXXX file can not be selected to bugzilla reporting because it is a binary file, and bugzilla reporter doesn't attach binary files to reports.

IOW, we may have two problems here:
(1) is anaconda-tb-XXXXXX really a binary file, or abrt is mistaken?
(2) should we teach bugzilla reporter to attach binary files too?

Regarding (1): current code declares file to be binary if it is larger than 200k, or more than 2% bytes in first 4 kbytes are not ASCII.

Can you check your anaconda-tb-XXXXXX file? Does it pass both checks? Can you attach one such file to this bug?

Regarding (2): it is relatively easy to implement this, but after this,

    reporter-bugzilla -d DIR

on a DIR generated from segfaulting application will happily upload *core dump* too, which is (a) big and (b) a security risk.
Comment 2 David Lehman 2011-08-30 09:42:58 EDT
(In reply to comment #1)
> anaconda-tb-XXXXXX file can not be selected to bugzilla reporting because it is
> a binary file, and bugzilla reporter doesn't attach binary files to reports.

It is not a binary file.

> 
> IOW, we may have two problems here:
> (1) is anaconda-tb-XXXXXX really a binary file, or abrt is mistaken?

anaconda-tb-XXXXX is *never* a binary file. It is always plain text.

> (2) should we teach bugzilla reporter to attach binary files too?
> 
> Regarding (1): current code declares file to be binary if it is larger than
> 200k, or more than 2% bytes in first 4 kbytes are not ASCII.

anaconda-tb-XXXXX varies in size according to the sizes of the various log files it contains. It is quite often larger than 200k.

> 
> Can you check your anaconda-tb-XXXXXX file? Does it pass both checks? Can you
> attach one such file to this bug?
> 

I'll attach one shortly.
Comment 3 David Lehman 2011-08-30 09:44:02 EDT
Created attachment 520612 [details]
example anaconda traceback file
Comment 4 Denys Vlasenko 2011-08-30 12:09:11 EDT
Fixed in git by these commits:

commit 1ad17e37d25a416a1798778b0492c81443d860aa
Author: Denys Vlasenko <dvlasenk@redhat.com>
Date:   Tue Aug 30 18:04:37 2011 +0200

    reporter-bugzilla: add -b to make it possible to attach binary items

    Text file max size is increased to 500k.
    Both these measures should help with rhbz#733448.


commit f41ede51f823d8e1697805018aab02b3ef5b9490
Author: Denys Vlasenko <dvlasenk@redhat.com>
Date:   Tue Aug 30 18:07:47 2011 +0200

    Use option -b ("attach binary files too") in reporter-bugzilla invocations
Comment 5 David Lehman 2011-08-30 12:30:13 EDT
Why is there a size limit on text files? I don't understand the logic behind this seemingly arbitrary limitation.
Comment 6 David Lehman 2011-08-30 12:36:34 EDT
Why not use the magic module from the python standard library for a more
reliable and consistent result:

>>> import magic
>>> m = magic.open(magic.MAGIC_MIME)
>>> m.load()
0
>>> m.file("/tmp/anaconda-tb-j_1A4h")
'text/plain; charset=us-ascii'
Comment 7 Nikola Pajkovsky 2011-08-31 05:23:25 EDT
That's because the text file could be 1GB and upload 1GB to bugzilla is not what everyone whats
Comment 8 David Lehman 2011-08-31 11:29:24 EDT
Which is worse: uploading a known text file of a potentially large size, or insisting on calling text files binary files when they exceed a certain size and then allowing blind upload of any/all binary files, regardless of size? I think you would be better off just allowing people to upload the anaconda-tb-XXX files regardless of size without making up lies about what the file type is.
Comment 9 Peter Jones 2011-08-31 15:33:36 EDT
In this case I don't really think the file can be 1G, but even if it were, there's a reason for the information in it, and we really do always want all of it, and we want it as text/plain.
Comment 10 Fedora Update System 2011-09-05 10:16:45 EDT
libreport-2.0.5-7.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/libreport-2.0.5-7.fc16
Comment 11 Fedora Update System 2011-09-06 14:08:28 EDT
Package libreport-2.0.5-7.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing libreport-2.0.5-7.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/libreport-2.0.5-7.fc16
then log in and leave karma (feedback).
Comment 12 Adam Williamson 2011-09-07 17:57:16 EDT
"That's because the text file could be 1GB and upload 1GB to bugzilla is not
what everyone whats"

that seems like something Bugzilla should disallow, if it doesn't want files over a certain size - doing it in libreport doesn't seem like the right place.
Comment 13 Fedora Update System 2011-09-09 09:46:26 EDT
libreport-2.0.5-8.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/libreport-2.0.5-8.fc16
Comment 14 Adam Williamson 2011-09-09 13:40:38 EDT
Discussed at 2011-09-09 blocker review meeting, accepted as a blocker per Alpha criterion "The installer must be able to report failures to Bugzilla, with appropriate information included ".
Comment 15 Adam Williamson 2011-09-12 20:55:33 EDT
It looks like -7 was only a partial fix for this, and -8 fixed it better. Unfortunately we only got -7 into TC2. I did notice that the traceback was greyed out and could not be selected when I ran into https://bugzilla.redhat.com/show_bug.cgi?id=735730 on TC2 earlier today. So I think we may be able to confirm this fix only with RC1, and we should pull -8 into RC1.
Comment 16 Tim Flink 2011-09-14 13:12:12 EDT
I tested this with a custom boot.iso containing libreport-2.0.5-8.

Using the update img (with randomized tb) from the save anaconda traceback to bz test case [1], I generated a new bug [2] from anaconda with anaconda attached.

Anaconda devs confirmed that this bug exhibits desired behavior and I'm moving this to verified.

[1] https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzilla
[2] https://bugzilla.redhat.com/show_bug.cgi?id=738386
Comment 17 Denys Vlasenko 2011-09-19 08:53:36 EDT
(In reply to comment #8)
> Which is worse: uploading a known text file of a potentially large size, or
> insisting on calling text files binary files when they exceed a certain size

It's not only upload. Elements which we consider to be "text" are loaded in memory when we load problem data, whereas binary files aren't loaded. Subsequently, text elements may be displayed or edited in GUI.

Therefore, if multi-gigabyte file in problem directory will be detected as "text", this problem directory will likely cause GUI and CLI to die with out-of-memory. Which may prevent deletion of said problem directory using GUI/CLI, and even prevent GUI operations on _other_ problem directories.

IOW: it's a basic sanity check. "If it's insanely large, don't try to work with it as if it's a few lines of text".

> and then allowing blind upload of any/all binary files, regardless of size?

The decision to allow or disallow binary uploads is for admin to make. We do not enable it unconditionally - admin _can_ switch it off.
Comment 18 David Lehman 2011-09-21 09:39:03 EDT
(In reply to comment #17)
> (In reply to comment #8)
> > Which is worse: uploading a known text file of a potentially large size, or
> > insisting on calling text files binary files when they exceed a certain size
> 
> It's not only upload. Elements which we consider to be "text" are loaded in
> memory when we load problem data, whereas binary files aren't loaded.
> Subsequently, text elements may be displayed or edited in GUI.
> 
> Therefore, if multi-gigabyte file in problem directory will be detected as
> "text", this problem directory will likely cause GUI and CLI to die with
> out-of-memory. Which may prevent deletion of said problem directory using
> GUI/CLI, and even prevent GUI operations on _other_ problem directories.
> 
> IOW: it's a basic sanity check. "If it's insanely large, don't try to work with
> it as if it's a few lines of text".

Okay, but if what you are concerned with is the size of the file why not add a flag indicating the file's size is too great instead of pretending its content is something other than what it actually is?

If your problem is that you can't display the file in a text editor, don't display it in the text editor -- that does not mean you have to lie about what is in the file to all other possible users of the file.

> 
> > and then allowing blind upload of any/all binary files, regardless of size?
> 
> The decision to allow or disallow binary uploads is for admin to make. We do
> not enable it unconditionally - admin _can_ switch it off.

If we were talking about binary files this might matter to me, but they are not.
Comment 19 Adam Williamson 2011-10-03 12:04:06 EDT
libreport-2.0.5.982-1.fc16 was pushed stable, so we can close this.

Note You need to log in before you can comment on or make changes to this bug.