Bug 1818842

Summary: urlgrabber.progress.TextMeter fails with "TypeError: '>' not supported between instances of 'NoneType' and 'int'"
Product: [Fedora] Fedora Reporter: John Haxby <jch>
Component: python-urlgrabberAssignee: Neal Gompa <ngompa13>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 31CC: brejoc, dmach, mdomonko, ngompa13, packaging-team-maint, pkratoch, sheltren
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-24 18:10:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description John Haxby 2020-03-30 13:50:10 UTC
Description of problem:

Trying to use urlgrabber with a text progress meter fails with "TypeError: '>' not supported between instances of 'NoneType' and 'int'"

Version-Release number of selected component (if applicable):
 python3-urlgrabber-4.0.0-4.fc31


How reproducible: Always

Steps to Reproduce:
1.
Run this program:
~~~
#!/usr/bin/python3

from urlgrabber.grabber import URLGrabber
from urlgrabber.progress import TextMeter

g = URLGrabber(progress_obj=TextMeter())
g.urlgrab('https://download.fedoraproject.org/pub/fedora/linux/releases/31/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-31-1.9.iso')
~~~

(use a different URL if you prefer)

2. Wait a short while
3.

Actual results:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1940, in _progress_update
    self.opts.progress_obj.update(downloaded)
  File "/usr/lib/python3.7/site-packages/urlgrabber/progress.py", line 149, in update
    self._do_update(amount_read, now)
  File "/usr/lib/python3.7/site-packages/urlgrabber/progress.py", line 251, in _do_update
    ave_dl = format_number(self.re.average_rate())
  File "/usr/lib/python3.7/site-packages/urlgrabber/progress.py", line 785, in format_number
    while number > thresh and depth < max_depth:
TypeError: '>' not supported between instances of 'NoneType' and 'int'
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1580, in _do_perform
    self.curl_obj.perform()
pycurl.error: (42, 'Callback aborted')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./grabber.py", line 7, in <module>
    g.urlgrab('https://download.fedoraproject.org/pub/fedora/linux/releases/31/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-31-1.9.iso')
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1256, in urlgrab
    return self._retry(opts, retryfunc, url, filename)
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1145, in _retry
    r = func(opts, *args)
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1241, in retryfunc
    fo = PyCurlFileObject(url, filename, opts)
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1343, in __init__
    self._do_open()
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1692, in _do_open
    self._do_grab()
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1826, in _do_grab
    self._do_perform()
  File "/usr/lib/python3.7/site-packages/urlgrabber/grabber.py", line 1615, in _do_perform
    raise KeyboardInterrupt
KeyboardInterrupt



Expected results:

A nice progress bar and, after a while, an iso.


Additional info:

I got 160KiB downloaded before the error message and nothing else was displayed at all. 

Although this is against Fedora31, I get the same error on Fedora 30 and using python2 instead of python3 everything works as expected both on Fedora31 and Fedora30.

Comment 1 John Haxby 2020-03-30 13:55:05 UTC
The same problem appears to be in python-urlgrabber-4.1.0

Comment 2 John Haxby 2020-03-30 14:06:09 UTC
Initialising ave_rate to zero instead of None in RateEstimator fixes that problem.   Changing "blen" to "int(blen)" in _term_add_bar() fixes the next one to show up.   Both of those are in /usr/lib/python3.7/site-packages/urlgrabber/progress.py

Hope that helps.

Comment 3 Jochen Breuer 2020-03-30 14:25:08 UTC
(In reply to John Haxby from comment #2)
> Initialising ave_rate to zero instead of None in RateEstimator fixes that
> problem.   Changing "blen" to "int(blen)" in _term_add_bar() fixes the next
> one to show up.   Both of those are in
> /usr/lib/python3.7/site-packages/urlgrabber/progress.py
> 
> Hope that helps.

Thanks for opening this issue! If you'd like to, you can open a PR against the GitHub repository[0]. Otherwise I can also do it as soon as possible.


[0] https://github.com/rpm-software-management/urlgrabber

Comment 4 John Haxby 2020-03-30 14:50:06 UTC
Done!

Comment 6 Ben Cotton 2020-11-03 16:32:55 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 7 Ben Cotton 2020-11-24 18:10:27 UTC
Fedora 31 changed to end-of-life (EOL) status on 2020-11-24. Fedora 31 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.