Bug 578969 - ABRT does not react to unhandled Python exceptions that don't have tracebacks (e.g. IndentationError)
Summary: ABRT does not react to unhandled Python exceptions that don't have tracebacks...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: abrt
Version: 15
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Miroslav Lichvar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: ABRTF17
TreeView+ depends on / blocked
 
Reported: 2010-04-01 22:15 UTC by Dave Malcolm
Modified: 2011-12-16 19:53 UTC (History)
9 users (show)

Fixed In Version: abrt-2.0.7-2.fc16
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-12-16 19:53:41 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Dave Malcolm 2010-04-01 22:15:01 UTC
Description of problem:
I followed the recipe here:
https://fedoraproject.org/wiki/QA:Testcase_ABRT_python

(albeit as a non-root user).


[david@f13 ~]$ cat /etc/abrt/plugins/Python.conf 
Enabled = yes

(is the fact that it's "yes" rather than "Yes" significant?)

[david@f13 ~]$ cat /etc/abrt/abrt.conf | grep Python
Python = Bugzilla, Logger

I edited the top of /usr/share/doc/smolt-1.4.2.2/lite2my.py to read:

#!/usr/bin/python
    "11" + 11
import sqlite
import sys
from urllib import urlencode

Upon executing it in gnome-terminal in a GNOME session I get this error:

[david@f13 ~]$ python /usr/share/doc/smolt-1.4.2.2/lite2my.py 
  File "/usr/share/doc/smolt-1.4.2.2/lite2my.py", line 2
    "11" + 11
    ^
IndentationError: unexpected indent

However, ABRT does not seem to respond; there is no notification from ABRT, and upon manually launching the graphical error browser (Applications > System Tools > Automatic B R T etc) there doesn't seem to be a report in the list view.

Version-Release number of selected component (if applicable):
abrt-addon-python-1.0.9-0.201003311.fc13.x86_64

[david@f13 ~]$ rpm -qa "abrt*"
abrt-1.0.9-0.201003311.fc13.x86_64
abrt-plugin-runapp-1.0.9-0.201003311.fc13.x86_64
abrt-addon-kerneloops-1.0.9-0.201003311.fc13.x86_64
abrt-libs-1.0.9-0.201003311.fc13.x86_64
abrt-addon-ccpp-1.0.9-0.201003311.fc13.x86_64
abrt-plugin-bugzilla-1.0.9-0.201003311.fc13.x86_64
abrt-gui-1.0.9-0.201003311.fc13.x86_64
abrt-desktop-1.0.9-0.201003311.fc13.x86_64
abrt-addon-python-1.0.9-0.201003311.fc13.x86_64
abrt-plugin-logger-1.0.9-0.201003311.fc13.x86_64


I'm using python-2.6.4-24_with_no_var_tracking_assignments.fc13.x86_64 , a scratch rebuild in Koji, but I don't believe that affects this.


Additional info:
How does this hook into Python?  I see:

[david@f13 ~]$ rpm -ql abrt-addon-python
/etc/abrt/plugins/Python.conf
/usr/lib/python2.6/site-packages/abrt.pth
/usr/lib/python2.6/site-packages/abrt_exception_handler.py
/usr/lib/python2.6/site-packages/abrt_exception_handler.pyc
/usr/lib/python2.6/site-packages/abrt_exception_handler.pyo
/usr/lib64/abrt/libPython.so
/usr/libexec/abrt-hook-python

Should there have been a site-customization hook?  Thanks.

Comment 1 Bertrand Juglas 2010-04-01 22:33:17 UTC
I'm using python-2.6.4-23.fc13.i686 from F13 official repositories and after following the same testing sequence i have exactly the same bug: ABRT doesn't show anything about the python backtrace.

Comment 2 Jiri Moskovcak 2010-04-01 23:07:13 UTC
> How does this hook into Python?  I see:
> /usr/lib/python2.6/site-packages/abrt.pth
- the .pth file is loaded at start of every python script so that's the place where the abrt's hook is instaled.


> File "/usr/share/doc/smolt-1.4.2.2/lite2my.py", line 2
>  "11" + 11
>    ^
>IndentationError: unexpected indent

- abrt doesn't catch IndentationError - AFAIK it's not even catchable exception, the test case is probably missleading, coz we don't want IndentationError, but:
TypeError: cannot concatenate 'str' and 'int' objects

J.

Comment 3 Dave Malcolm 2010-04-01 23:34:56 UTC
Thanks; looks like a bad test case then.

On changing the code to:
#!/usr/bin/python
print "11" + 11
import sqlite
import sys
from urllib import urlencode

and retrying, ABRT does indeed detect the error.

I've edited the test case to remove the misleading indentation, and to clarify the type of the expected error:
https://fedoraproject.org/wiki/QA:Testcase_ABRT_python_addon

Comment 4 Dave Malcolm 2010-04-01 23:48:54 UTC
Reopening: I believe  "IndentationError" is catchable, and it seems like a meaningful thing to detect.

I reset the file back to the state in comment #0, and then tried the following:

[david@f13 ~]$ PYTHONPATH=/usr/share/doc/smolt-1.4.2.2 python
Python 2.6.4 (r264:75706, Mar 25 2010, 22:46:56) 
[GCC 4.4.3 20100226 (Red Hat 4.4.3-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     import lite2my
... except IndentationError, e:
...     print e
... 
unexpected indent (lite2my.py, line 2)
>>> e
IndentationError('unexpected indent', ('/usr/share/doc/smolt-1.4.2.2/lite2my.py', 2, 4, '    print "11" + 11\n'))
>>> e.__class__
<type 'exceptions.IndentationError'>
>>> e.__class__.__mro__
(<type 'exceptions.IndentationError'>, <type 'exceptions.SyntaxError'>, <type 'exceptions.StandardError'>, <type 'exceptions.Exception'>, <type 'exceptions.BaseException'>, <type 'object'>)
>>> 

So it appears Python ought to be capable of detecting this error and handling it.

Comment 5 Jiri Moskovcak 2010-04-02 09:26:01 UTC
(In reply to comment #4)
> Reopening: I believe  "IndentationError" is catchable, and it seems like a
> meaningful thing to detect.
> 

Yes, you're right. I went thru the abrt-exception-handler code and it detect's this exception (you can see it in /var/log/messages), but the logic in the hook ignores exceptions without traceback.
 However, I can't think of any case when "IndentationError" can be thrown at runtime (you won't be able to run the python script with bad indentation even if you don't use the part where it goes wrong). So the only case would be when programmer doesn't even try to run the app and push it to the distro.

J.

Comment 6 Michal Nowak 2010-04-02 10:20:38 UTC
(In reply to comment #5)
> So the only case would be when
> programmer doesn't even try to run the app and push it to the distro.

However, it *may* happen. So on the other hand, is there a reason not to catch it?

Here comes bug list of 56 bugs where IndentationError played some role: 

https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&longdesc=IndentationError&longdesc_type=allwordssubstr

Comment 7 Jiri Moskovcak 2010-04-02 10:38:49 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > So the only case would be when
> > programmer doesn't even try to run the app and push it to the distro.
> 
> However, it *may* happen. So on the other hand, is there a reason not to catch
> it?
> 
> Here comes bug list of 56 bugs where IndentationError played some role: 
> 
> https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&longdesc=IndentationError&longdesc_type=allwordssubstr    

Still feels like updating some C program with broken syntax without even trying to compile it, but I agree, we should probably catch these because we're only humans and there is also a Murphy's law: "Anything that can go wrong will go wrong". :)

Adding this to our TODO list.

J.

Comment 8 Dave Malcolm 2010-04-02 16:32:41 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > So the only case would be when
> > > programmer doesn't even try to run the app and push it to the distro.
> > 
> > However, it *may* happen. So on the other hand, is there a reason not to catch
> > it?
> > 
> > Here comes bug list of 56 bugs where IndentationError played some role: 
> > 
> > https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&longdesc=IndentationError&longdesc_type=allwordssubstr    


Many of these are false positives (apparently due to Anaconda's exception printer printing all builtin vars, which includes the name "IndentationError" within the __builtins__ scope.

Having said that, examples of real bugs where invalid Python sources were built in an RPM and reached rawhide or testing can be seen in bug 523534 (Fedora, authconfig) and in bug 430074 (this one on RHEL), at least.  Ideally we ought to catch these earlier (e.g. during rpm-build), but it would be good if ABRT handled them gracefully.


> Still feels like updating some C program with broken syntax without even trying
> to compile it, but I agree, we should probably catch these because we're only
> humans and there is also a Murphy's law: "Anything that can go wrong will go
> wrong". :)
> 
> Adding this to our TODO list.

Thanks!  Renaming this bug accordingly.

Comment 9 Bug Zapper 2011-06-02 15:44:23 UTC
This message is a reminder that Fedora 13 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 13.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '13'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 13's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 13 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 10 Dave Malcolm 2011-06-02 16:30:38 UTC
This bug appears to still be present on F15:  if I simulate a package with broken indentation by hacking /usr/bin/yum to contain 
bad indentation at the top of the file e.g. adding this line:

 #!/usr/bin/python                                                                                                                                                                                                                            
+   print "this line is badly indented"
 import sys
 try:
     import yum
 except ImportError:
     print >> sys.stderr, """\  

then all "yum" invocations fail an IndentationError traceback, but ABRT does not seem to detect any of them.

This is with:
  abrt-addon-python-1.1.17-1.fc15.x86_64

Comment 11 Dave Malcolm 2011-06-02 16:31:46 UTC
Actually, it's not a "traceback" per se, but hopefully you get the general idea:

  [david@f15] $ yum
    File "/usr/bin/yum", line 2
      print "this line is badly indented"
      ^
  IndentationError: unexpected indent

Comment 12 Miroslav Lichvar 2011-10-17 11:52:25 UTC
Upstream commit 1a5924.

Comment 13 Fedora Update System 2011-12-10 11:06:03 UTC
abrt-2.0.7-2.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/abrt-2.0.7-2.fc16

Comment 14 Fedora Update System 2011-12-11 21:57:33 UTC
Package abrt-2.0.7-2.fc16, libreport-2.0.8-3.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing abrt-2.0.7-2.fc16 libreport-2.0.8-3.fc16'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-16990/libreport-2.0.8-3.fc16,abrt-2.0.7-2.fc16
then log in and leave karma (feedback).

Comment 15 Fedora Update System 2011-12-16 19:53:41 UTC
abrt-2.0.7-2.fc16, libreport-2.0.8-3.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.