Bug 862079 - [abrt] python-pep8-1.0.1-1.fc17: tokenize.py:357:generate_tokens:TokenError: ('EOF in multi-line statement', (7, 0))
Summary: [abrt] python-pep8-1.0.1-1.fc17: tokenize.py:357:generate_tokens:TokenError: ...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pep8
Version: 17
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Ian Weller
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: abrt_hash:f171d841baabec7bf7ed381de0b...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-10-01 19:44 UTC by MartinG
Modified: 2013-03-19 04:31 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-01-30 03:16:48 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: core_backtrace (465 bytes, text/plain)
2012-10-01 19:44 UTC, MartinG
no flags Details

Description MartinG 2012-10-01 19:44:32 UTC
Description of problem:
I'm using emacs with flymake for .py files. While typing, this package crashed.

This is the relevant code in my ~/.emacs

;;# yum install pyflakes python-pep8
;;~/.emacs.d/site-lisp:$ wget http://www.emacswiki.org/emacs/flymake-cursor.el
;;(add-to-list 'load-path "~/.emacs.d/site-lisp/")


(add-hook 'find-file-hook 'flymake-find-file-hook)
(when (load "flymake" t)
  (defun flymake-pyflakes-init ()
    (let* ((temp-file (flymake-init-create-temp-buffer-copy
		       'flymake-create-temp-inplace))
	   (local-file (file-relative-name
			temp-file
			(file-name-directory buffer-file-name))))
      (list "pycheckers"  (list local-file))))
     ;;(list "pylint"  (list local-file))))
  (add-to-list 'flymake-allowed-file-name-masks
	       '("\\.py\\'" flymake-pyflakes-init)))
(load-library "flymake-cursor")

The (invalid) code which caused the crash was:
#!/bin/python

if __name__ == "__main__":
    for i in range(1,100):
        print(i)


This is on a Fedora 17 x86_64 system.


Version-Release number of selected component:
python-pep8-1.0.1-1.fc17

Additional info:
libreport version: 2.0.14
abrt_version:   2.0.13
kernel:         3.5.4-1.fc17.x86_64

backtrace:
:tokenize.py:357:generate_tokens:TokenError: ('EOF in multi-line statement', (7, 0))
:
:Traceback (most recent call last):
:  File "/usr/bin/pep8", line 9, in <module>
:    load_entry_point('pep8==1.0.1', 'console_scripts', 'pep8')()
:  File "/usr/lib/python2.7/site-packages/pep8.py", line 1348, in _main
:    runner(path)
:  File "/usr/lib/python2.7/site-packages/pep8.py", line 1023, in input_file
:    errors = Checker(filename).check_all()
:  File "/usr/lib/python2.7/site-packages/pep8.py", line 949, in check_all
:    for token in tokenize.generate_tokens(self.readline_check_physical):
:  File "/usr/lib64/python2.7/tokenize.py", line 357, in generate_tokens
:    raise TokenError, ("EOF in multi-line statement", (lnum, 0))
:TokenError: ('EOF in multi-line statement', (7, 0))
:
:Local variables in innermost frame:
:lnum: 7
:pos: 0
:namechars: 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_'
:readline: <bound method Checker.readline_check_physical of <pep8.Checker object at 0x1c83790>>
:epos: (6, 15)
:end: 15
:initial: '\n'
:needcont: 0
:contstr: ''
:start: 14
:max: 0
:spos: (6, 14)
:indents: [0, 4, 8]
:contline: None
:line: ''
:continued: 0
:comment_token: '#!/bin/python'
:pseudomatch: <_sre.SRE_Match object at 0x1d0a240>
:column: 8
:nl_pos: 13
:token: '\n'
:numchars: '0123456789'
:parenlev: 1

Comment 1 MartinG 2012-10-01 19:44:35 UTC
Created attachment 619975 [details]
File: core_backtrace

Comment 2 Ian Weller 2013-01-30 03:16:48 UTC
This crash report appears to be for running pep8 on obviously incorrect code (i.e. syntax errors), and a traceback is to be expected.

Reporter, you may reopen this ticket -- with more information on why this is a bug -- if you believe the above is incorrect.

Comment 3 Noel Duffy 2013-03-19 04:31:01 UTC
This is absolutely a bug. The whole point of pep8 is that you run it on your Python script and it reports errors, syntactical or otherwise, bugs, and style problems. Having it crash on an input script just because that script is syntactically wrong is the equivalent of GCC crashing if fed a syntactically wrong C program.


Note You need to log in before you can comment on or make changes to this bug.