anaconda-tb-XXXXXX file can not be selected to bugzilla reporting because it is a binary file, and bugzilla reporter doesn't attach binary files to reports.

IOW, we may have two problems here:
(1) is anaconda-tb-XXXXXX really a binary file, or abrt is mistaken?
(2) should we teach bugzilla reporter to attach binary files too?

Regarding (1): current code declares file to be binary if it is larger than 200k, or more than 2% bytes in first 4 kbytes are not ASCII.

Can you check your anaconda-tb-XXXXXX file? Does it pass both checks? Can you attach one such file to this bug?

Regarding (2): it is relatively easy to implement this, but after this,

    reporter-bugzilla -d DIR

on a DIR generated from segfaulting application will happily upload *core dump* too, which is (a) big and (b) a security risk.

(In reply to comment #1)
> anaconda-tb-XXXXXX file can not be selected to bugzilla reporting because it is
> a binary file, and bugzilla reporter doesn't attach binary files to reports.

It is not a binary file.

> 
> IOW, we may have two problems here:
> (1) is anaconda-tb-XXXXXX really a binary file, or abrt is mistaken?

anaconda-tb-XXXXX is *never* a binary file. It is always plain text.

> (2) should we teach bugzilla reporter to attach binary files too?
> 
> Regarding (1): current code declares file to be binary if it is larger than
> 200k, or more than 2% bytes in first 4 kbytes are not ASCII.

anaconda-tb-XXXXX varies in size according to the sizes of the various log files it contains. It is quite often larger than 200k.

> 
> Can you check your anaconda-tb-XXXXXX file? Does it pass both checks? Can you
> attach one such file to this bug?
> 

I'll attach one shortly.

Created attachment 520612 [details]
example anaconda traceback file

Fixed in git by these commits:

commit 1ad17e37d25a416a1798778b0492c81443d860aa
Author: Denys Vlasenko <dvlasenk>
Date:   Tue Aug 30 18:04:37 2011 +0200

    reporter-bugzilla: add -b to make it possible to attach binary items

    Text file max size is increased to 500k.
    Both these measures should help with rhbz#733448.


commit f41ede51f823d8e1697805018aab02b3ef5b9490
Author: Denys Vlasenko <dvlasenk>
Date:   Tue Aug 30 18:07:47 2011 +0200

    Use option -b ("attach binary files too") in reporter-bugzilla invocations

Why is there a size limit on text files? I don't understand the logic behind this seemingly arbitrary limitation.

Why not use the magic module from the python standard library for a more
reliable and consistent result:

>>> import magic
>>> m = magic.open(magic.MAGIC_MIME)
>>> m.load()
0
>>> m.file("/tmp/anaconda-tb-j_1A4h")
'text/plain; charset=us-ascii'

That's because the text file could be 1GB and upload 1GB to bugzilla is not what everyone whats

Which is worse: uploading a known text file of a potentially large size, or insisting on calling text files binary files when they exceed a certain size and then allowing blind upload of any/all binary files, regardless of size? I think you would be better off just allowing people to upload the anaconda-tb-XXX files regardless of size without making up lies about what the file type is.

In this case I don't really think the file can be 1G, but even if it were, there's a reason for the information in it, and we really do always want all of it, and we want it as text/plain.

libreport-2.0.5-7.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/libreport-2.0.5-7.fc16

Package libreport-2.0.5-7.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing libreport-2.0.5-7.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/libreport-2.0.5-7.fc16
then log in and leave karma (feedback).

"That's because the text file could be 1GB and upload 1GB to bugzilla is not
what everyone whats"

that seems like something Bugzilla should disallow, if it doesn't want files over a certain size - doing it in libreport doesn't seem like the right place.

libreport-2.0.5-8.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/libreport-2.0.5-8.fc16

Discussed at 2011-09-09 blocker review meeting, accepted as a blocker per Alpha criterion "The installer must be able to report failures to Bugzilla, with appropriate information included ".

It looks like -7 was only a partial fix for this, and -8 fixed it better. Unfortunately we only got -7 into TC2. I did notice that the traceback was greyed out and could not be selected when I ran into https://bugzilla.redhat.com/show_bug.cgi?id=735730 on TC2 earlier today. So I think we may be able to confirm this fix only with RC1, and we should pull -8 into RC1.

I tested this with a custom boot.iso containing libreport-2.0.5-8.

Using the update img (with randomized tb) from the save anaconda traceback to bz test case [1], I generated a new bug [2] from anaconda with anaconda attached.

Anaconda devs confirmed that this bug exhibits desired behavior and I'm moving this to verified.

[1] https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzilla
[2] https://bugzilla.redhat.com/show_bug.cgi?id=738386

(In reply to comment #8)
> Which is worse: uploading a known text file of a potentially large size, or
> insisting on calling text files binary files when they exceed a certain size

It's not only upload. Elements which we consider to be "text" are loaded in memory when we load problem data, whereas binary files aren't loaded. Subsequently, text elements may be displayed or edited in GUI.

Therefore, if multi-gigabyte file in problem directory will be detected as "text", this problem directory will likely cause GUI and CLI to die with out-of-memory. Which may prevent deletion of said problem directory using GUI/CLI, and even prevent GUI operations on _other_ problem directories.

IOW: it's a basic sanity check. "If it's insanely large, don't try to work with it as if it's a few lines of text".

> and then allowing blind upload of any/all binary files, regardless of size?

The decision to allow or disallow binary uploads is for admin to make. We do not enable it unconditionally - admin _can_ switch it off.

(In reply to comment #17)
> (In reply to comment #8)
> > Which is worse: uploading a known text file of a potentially large size, or
> > insisting on calling text files binary files when they exceed a certain size
> 
> It's not only upload. Elements which we consider to be "text" are loaded in
> memory when we load problem data, whereas binary files aren't loaded.
> Subsequently, text elements may be displayed or edited in GUI.
> 
> Therefore, if multi-gigabyte file in problem directory will be detected as
> "text", this problem directory will likely cause GUI and CLI to die with
> out-of-memory. Which may prevent deletion of said problem directory using
> GUI/CLI, and even prevent GUI operations on _other_ problem directories.
> 
> IOW: it's a basic sanity check. "If it's insanely large, don't try to work with
> it as if it's a few lines of text".

Okay, but if what you are concerned with is the size of the file why not add a flag indicating the file's size is too great instead of pretending its content is something other than what it actually is?

If your problem is that you can't display the file in a text editor, don't display it in the text editor -- that does not mean you have to lie about what is in the file to all other possible users of the file.

> 
> > and then allowing blind upload of any/all binary files, regardless of size?
> 
> The decision to allow or disallow binary uploads is for admin to make. We do
> not enable it unconditionally - admin _can_ switch it off.

If we were talking about binary files this might matter to me, but they are not.

libreport-2.0.5.982-1.fc16 was pushed stable, so we can close this.