Bug 613657 - [abrt] crash in pylint-0.20.0-1.fc13: tokenize.py:352:generate_tokens:TokenError: ('EOF in multi-line statement', (3093, 0))
[abrt] crash in pylint-0.20.0-1.fc13: tokenize.py:352:generate_tokens:TokenEr...
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: pylint (Show other bugs)
13
i686 Linux
low Severity medium
: ---
: ---
Assigned To: Brian Lane
Fedora Extras Quality Assurance
abrt_hash:b3e572e3
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-07-12 10:13 EDT by Mads Kiilerich
Modified: 2010-07-13 13:26 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-07-13 12:27:12 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
File: backtrace (1.66 KB, text/plain)
2010-07-12 10:13 EDT, Mads Kiilerich
no flags Details

  None (edit)
Description Mads Kiilerich 2010-07-12 10:13:51 EDT
abrt 1.1.1 detected a crash.

architecture: i686
cmdline: /usr/bin/python -u /usr/bin/pylint --include-ids=y --disable-msg=W0511,W0612,W0603,W0142 /home/mk/src/hg/hgext/mq.py
component: pylint
executable: /usr/bin/pylint
kernel: 2.6.33.6-147.fc13.i686.PAE
package: pylint-0.20.0-1.fc13
reason: tokenize.py:352:generate_tokens:TokenError: ('EOF in multi-line statement', (3093, 0))
release: Fedora release 13 (Goddard)

backtrace
-----
tokenize.py:352:generate_tokens:TokenError: ('EOF in multi-line statement', (3093, 0))

Traceback (most recent call last):
  File "/usr/bin/pylint", line 4, in <module>
    lint.Run(sys.argv[1:])
  File "/usr/lib/python2.6/site-packages/pylint/lint.py", line 885, in __init__
    linter.check(args)
  File "/usr/lib/python2.6/site-packages/pylint/lint.py", line 496, in check
    self.check_astng_module(astng, checkers)
  File "/usr/lib/python2.6/site-packages/pylint/lint.py", line 570, in check_astng_module
    checker.process_module(stream)
  File "/usr/lib/python2.6/site-packages/pylint/checkers/__init__.py", line 125, in process_module
    self.process_tokens(tokenize.generate_tokens(stream.readline))
  File "/usr/lib/python2.6/site-packages/pylint/checkers/format.py", line 206, in process_tokens
    for (tok_type, token, start, _, line) in tokens:
  File "/usr/lib/python2.6/tokenize.py", line 352, in generate_tokens
    raise TokenError, ("EOF in multi-line statement", (lnum, 0))
TokenError: ('EOF in multi-line statement', (3093, 0))

Local variables in innermost frame:
lnum: 3093
pos: 0
namechars: 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_'
readline: <built-in method readline of file object at 0x91dad40>
strstart: (2675, 4)
epos: (3092, 49)
end: 49
initial: '\n'
needcont: 0
contstr: ''
start: 48
max: 0
endmatch: <_sre.SRE_Match object at 0x91b82c0>
spos: (3092, 48)
indents: [0, 4, 8, 12]
contline: None
line: ''
continued: 0
comment_token: '# Otherwise, they should form a linear path to a head.'
pseudomatch: <_sre.SRE_Match object at 0x83fc7b8>
column: 12
nl_pos: 66
endprog: <_sre.SRE_Pattern object at 0x8121658>
token: '\n'
numchars: '0123456789'
parenlev: 1
Comment 1 Mads Kiilerich 2010-07-12 10:13:54 EDT
Created attachment 431187 [details]
File: backtrace
Comment 2 Brian Lane 2010-07-12 17:57:07 EDT
Could you attach mq.py as well?
Comment 3 Mads Kiilerich 2010-07-12 21:22:23 EDT
It seems like the problem happened because I was updating (truncating) files (tracked by mercurial) while pylint was checking them, and pylint thus got so confused that it crashed.

I will try to fix Mercurial so it don't modify the files, but the issue here is that pylint should handle such errors with a (fatal) error message but without crashing.
Comment 4 Mads Kiilerich 2010-07-13 08:27:48 EDT
Reproducibel with

f=open('x.py', 'w')
f.write('+1\\\n'*100)
import os
os.system('pylint x.py &')
for i in range(f.tell(),1,-4):
    print i
    f.seek(i)
    f.truncate()
Comment 5 Brian Lane 2010-07-13 12:27:12 EDT
That's not something that I would expect to work.
Comment 6 Mads Kiilerich 2010-07-13 13:26:33 EDT
(In reply to comment #5)
> That's not something that I would expect to work.    

That's fair enough. But it happens in real life, and it gives a crash that abrt will report to you (or CC here).

I would expect it to fail in a civilized way.

Note You need to log in before you can comment on or make changes to this bug.