Bug 1135632 - utf8 error decoding broken in py2 after py3 patch
Summary: utf8 error decoding broken in py2 after py3 patch
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-urlgrabber
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Tomas 'Sheldon' Radej
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1135680 (view as bug list)
Depends On: 1138270
Blocks: TRACKER-bugs-affecting-libguestfs
TreeView+ depends on / blocked
 
Reported: 2014-08-29 20:02 UTC by James Antill
Modified: 2014-10-29 10:20 UTC (History)
6 users (show)

Fixed In Version: python-urlgrabber-0:3.10.1-6.fc22.noarch
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-10-29 10:20:32 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description James Antill 2014-08-29 20:02:39 UTC
Description of problem:

 The rawhide compose returned this after the py3 patch went in:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1307, in _retrieve
    self.fo.write(buf.decode('utf-8'))
  File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xfd in position 0: invalid start byte
Traceback (most recent call last):
  File "/usr/libexec/urlgrabber-ext-down", line 75, in <module>
    main()
  File "/usr/libexec/urlgrabber-ext-down", line 61, in main
    fo = PyCurlFileObject(opts.url, opts.filename, opts)
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1262, in __init__
    self._do_open()
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1613, in _do_open
    self._do_grab()
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1749, in _do_grab
    Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1307, in _retrieve
    self.fo.write(buf.decode('utf-8'))
  File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xfd in position 0: invalid start byte
Traceback (most recent call last):
  File "/usr/libexec/urlgrabber-ext-down", line 75, in <module>
    main()
  File "/usr/libexec/urlgrabber-ext-down", line 61, in main
    fo = PyCurlFileObject(opts.url, opts.filename, opts)
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1262, in __init__
    self._do_open()
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1613, in _do_open
    self._do_grab()
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1749, in _do_grab
    self._do_perform()
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1527, in _do_perform
    self._do_perform()
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1527, in _do_perform
    raise getattr(self, '_cb_error', KeyboardInterrupt)
KeyboardInterrupt
raise getattr(self, '_cb_error', KeyboardInterrupt)
KeyboardInterrupt


Exiting on user cancel
error: Child process exited with code 1


...which is:

                    if not six.PY3 or isinstance(self.fo, StringIO):
                        self.fo.write(buf.decode('utf-8'))
                    else:
                        self.fo.write(buf)

Comment 1 James Antill 2014-08-29 20:04:15 UTC
 It's been untagged, so it's not urgent to get a new build out ... see if you can speak to some rel-eng people so you can do a dry run before doing another build in rawhide.

Comment 2 Kevin Fenzi 2014-08-30 16:12:08 UTC
It actually didn't get untagged in time, but I just did so and am restarting rawhide compose.

Comment 3 Tomas 'Sheldon' Radej 2014-09-01 12:24:16 UTC
I'll take a look at this. Can you please give me logs or more information about where this problem arose, so that I can debug it? Thank you.

TR

Comment 4 Kevin Fenzi 2014-09-01 16:46:05 UTC
You can see the full traceback in the mash logs: 

http://koji.fedoraproject.org/mash/rawhide-20140830.broken/logs/mash.log

Comment 5 Valentina Mukhamedzhanova 2014-09-02 10:32:57 UTC
*** Bug 1135680 has been marked as a duplicate of this bug. ***

Comment 6 Richard W.M. Jones 2014-09-02 11:21:18 UTC
I filed the duplicate bug 1135680.  FWIW the problem has stopped
happening now in Rawhide & F21.  As far as I know I didn't change
anything my end.

Comment 7 Tomas 'Sheldon' Radej 2014-09-02 11:26:21 UTC
(In reply to Richard W.M. Jones from comment #6)
> I filed the duplicate bug 1135680.  FWIW the problem has stopped
> happening now in Rawhide & F21.  As far as I know I didn't change
> anything my end.

Both bugs were caused by my patch for python-urlgrabber. The build thereof was untagged, which is why it's working again. I have a patch ready for urlgrabber that should fix both the issues, and will be posting it shortly.

Comment 8 Valentina Mukhamedzhanova 2014-09-04 12:11:12 UTC
*** Bug 1138270 has been marked as a duplicate of this bug. ***

Comment 9 Tomas 'Sheldon' Radej 2014-10-29 10:20:32 UTC
The culpable code which added Python 3 compatibility was reverted, so I am marking this as fixed.


Note You need to log in before you can comment on or make changes to this bug.