Bug 1710576 - /etc/cron.hourly/0yum-hourly.cron crashes - unhashable type: 'dict'
Summary: /etc/cron.hourly/0yum-hourly.cron crashes - unhashable type: 'dict'
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-urlgrabber
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Michal Domonkos
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-15 19:42 UTC by Tom Chiverton
Modified: 2019-06-06 01:05 UTC (History)
5 users (show)

Fixed In Version: python-urlgrabber-4.0.0-3.fc30
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-06 01:05:58 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github rpm-software-management urlgrabber issues 10#issuecomment-492832007 0 None None None 2019-05-16 17:10:30 UTC

Description Tom Chiverton 2019-05-15 19:42:17 UTC
Description of problem:
yum-hourly cron task fails

Version-Release number of selected component (if applicable):

# rpm -qf /etc/yum/yum-cron-hourly.conf
yum-cron-3.4.3-521.fc30.noarch
# rpm -qa urlgrabber
python2-urlgrabber-4.0.0-1.fc30.noarch

How reproducible:
every time 

Steps to Reproduce:
1. run /etc/cron.hourly/0yum-hourly.cron

Actual results:
crash

Expected results:
no crash

Additional info:

# /etc/cron.hourly/0yum-hourly.cron 
Traceback (most recent call last):
  File "/usr/sbin/yum-cron", line 713, in <module>
    main()
  File "/usr/sbin/yum-cron", line 710, in main
    base.updatesCheck()
  File "/usr/sbin/yum-cron", line 602, in updatesCheck
    self.populateUpdateMetadata()
  File "/usr/sbin/yum-cron", line 409, in populateUpdateMetadata
    self.pkgSack # honor skip_if_unavailable
  File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1095, in <lambda>
    pkgSack = property(fget=lambda self: self._getSacks(),
  File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 799, in _getSacks
    self.repos.populateSack(which=repos)
  File "/usr/lib/python2.7/site-packages/yum/repos.py", line 344, in populateSack
    self.doSetup()
  File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup
    self.retrieveAllMD()
  File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD
    dl = repo._async and repo._commonLoadRepoXML(repo)
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1460, in _commonLoadRepoXML
    result = self._getFileRepoXML(local, text)
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1234, in _getFileRepoXML
    size=102400) # setting max size as 100K
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1014, in _getFile
    result = self.grab.urlgrab(misc.to_utf8(relative), local,
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 698, in <lambda>
    grab = property(lambda self: self._getgrab())
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 693, in _getgrab
    self._setupGrab()
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 659, in _setupGrab
    failure_callback=mirror_failure)
  File "/usr/lib/python2.7/site-packages/urlgrabber/mirror.py", line 288, in __init__
    self.mirrors.sort(key=estimate, reverse=True)
  File "/usr/lib/python2.7/site-packages/urlgrabber/mirror.py", line 282, in estimate
    speed, fail = _TH.estimate(m['mirror'])
  File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 2547, in estimate
    host = urlparse.urlsplit(baseurl).netloc.split(b'@')[-1] or baseurl
  File "/usr/lib64/python2.7/urlparse.py", line 176, in urlsplit
    cached = _parse_cache.get(key, None)
TypeError: unhashable type: 'dict'

Comment 1 Tom Chiverton 2019-05-16 17:10:30 UTC
Found an upstream bug (linked), closed sometime ago saying they don't care.
so fedora should drop urlgrabber? Means rewriting core yum features..

Comment 2 Michal Domonkos 2019-05-17 08:15:30 UTC
Fedora actually *is* dropping yum (the original implementation, version 3) starting with F31.

That said, in F30, we still have yum-3, so this should really be fixed.  I'll take that on.

Comment 3 Michal Domonkos 2019-05-20 13:55:18 UTC
Fixed upstream: https://github.com/rpm-software-management/urlgrabber/pull/13

Comment 4 Fedora Update System 2019-05-21 16:30:18 UTC
python-urlgrabber-4.0.0-3.fc30 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-372212061d

Comment 5 Fedora Update System 2019-05-22 01:44:20 UTC
python-urlgrabber-4.0.0-3.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-372212061d

Comment 6 Tom Chiverton 2019-05-23 20:52:48 UTC
Sorted.

Comment 7 Tom Chiverton 2019-05-27 14:40:35 UTC
However, dnf upgrade now reports issues :

[root@bookcase boinc]# dnf upgrade
ROCm                                                                                            16 kB/s | 2.9 kB     00:00    
Fedora Modular 30 - x86_64                                                                      14 kB/s |  24 kB     00:01    
Fedora Modular 30 - x86_64 - Updates                                                            45 kB/s |  23 kB     00:00    
Fedora 30 - x86_64 - Updates                                                                    47 kB/s |  24 kB     00:00    
Fedora 30 - x86_64                                                                              29 kB/s |  24 kB     00:00    
RPM Fusion for Fedora 30 - Free - Updates                                                       57 kB/s |  10 kB     00:00    
RPM Fusion for Fedora 30 - Free                                                                 24 kB/s |  10 kB     00:00    
Dependencies resolved.

 Problem: package fuse-2.9.9-3.fc30.x86_64 requires fuse-common = 3.4.2, but none of the providers can be installed
  - cannot install both fuse-common-3.5.0-1.fc30.x86_64 and fuse-common-3.4.2-6.fc30.x86_64
  - cannot install both fuse-common-3.4.2-3.fc30.x86_64 and fuse-common-3.5.0-1.fc30.x86_64
  - cannot install the best update candidate for package fuse-common-3.4.2-6.fc30.x86_64
  - cannot install the best update candidate for package fuse-2.9.9-3.fc30.x86_64
===============================================================================================================================
 Package                         Architecture               Version                          Repository                   Size
===============================================================================================================================
Skipping packages with conflicts:
(add '--best --allowerasing' to command line to force their upgrade):
 fuse-common                     x86_64                     3.4.2-3.fc30                     fedora                       19 k
 fuse-common                     x86_64                     3.5.0-1.fc30                     updates                     7.4 k

Transaction Summary
===============================================================================================================================
Skip  2 Packages

Nothing to do.
Complete!
[root@bookcase boinc]# dnf upgrade --best --allowerasing
Last metadata expiration check: 0:00:30 ago on Mon 27 May 2019 15:36:09 BST.
Error: 
 Problem: cannot install the best update candidate for package fuse-2.9.9-3.fc30.x86_64
  - problem with installed package fuse-2.9.9-3.fc30.x86_64
  - package fuse-2.9.9-3.fc30.x86_64 requires fuse-common = 3.4.2, but none of the providers can be installed
  - cannot install both fuse-common-3.5.0-1.fc30.x86_64 and fuse-common-3.4.2-6.fc30.x86_64
  - cannot install both fuse-common-3.4.2-3.fc30.x86_64 and fuse-common-3.5.0-1.fc30.x86_64
  - cannot install the best update candidate for package fuse-common-3.4.2-6.fc30.x86_64
  - problem with installed package fuse-common-3.4.2-6.fc30.x86_64
(try to add '--skip-broken' to skip uninstallable packages)
[root@bookcase boinc]# dnf upgrade --best --allowerasing --skip-broken
Last metadata expiration check: 0:01:13 ago on Mon 27 May 2019 15:36:09 BST.
Error: 
 Problem: cannot install the best update candidate for package fuse-2.9.9-3.fc30.x86_64
  - problem with installed package fuse-2.9.9-3.fc30.x86_64
  - package fuse-2.9.9-3.fc30.x86_64 requires fuse-common = 3.4.2, but none of the providers can be installed
  - cannot install both fuse-common-3.5.0-1.fc30.x86_64 and fuse-common-3.4.2-6.fc30.x86_64
  - cannot install both fuse-common-3.4.2-3.fc30.x86_64 and fuse-common-3.5.0-1.fc30.x86_64
  - cannot install the best update candidate for package fuse-common-3.4.2-6.fc30.x86_64
  - problem with installed package fuse-common-3.4.2-6.fc30.x86_64
[root@bookcase boinc]# dnf remove fuse-common
Dependencies resolved.
===============================================================================================================================
 Package                          Architecture             Version                            Repository                  Size
===============================================================================================================================
Removing:
 fuse-common                      x86_64                   3.4.2-6.fc30                       @updates                    38  
Removing dependent packages:
 borgbackup                       x86_64                   1.1.10-2.fc30                      @updates                   4.8 M
Removing unused dependencies:
 fuse                             x86_64                   2.9.9-3.fc30                       @fedora                    224 k
 python3-llfuse                   x86_64                   1.3.5-2.fc30                       @fedora                    1.5 M

Transaction Summary
===============================================================================================================================
Remove  4 Packages

Freed space: 6.5 M
Is this ok [y/N]: n
Operation aborted.
[root@bookcase boinc]#

Comment 8 Tom Chiverton 2019-05-27 14:43:46 UTC
Looks like you need the fuse package from updates-testing as well:

[root@bookcase boinc]# sudo dnf upgrade --enablerepo=updates-testing fuse
Last metadata expiration check: 0:00:50 ago on Mon 27 May 2019 15:40:57 BST.
Dependencies resolved.
===============================================================================================================================
 Package                 Architecture              Version                            Repository                          Size
===============================================================================================================================
Upgrading:
 fuse                    x86_64                    2.9.9-7.fc30                       updates-testing                     79 k

Transaction Summary
===============================================================================================================================
Upgrade  1 Package

Total download size: 79 k
Is this ok [y/N]: y
Downloading Packages:
fuse-2.9.9-7.fc30.x86_64.rpm                                                                   280 kB/s |  79 kB     00:00    
-------------------------------------------------------------------------------------------------------------------------------
Total                                                                                           86 kB/s |  79 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                       1/1 
  Upgrading        : fuse-2.9.9-7.fc30.x86_64                                                                              1/2 
update
  Cleanup          : fuse-2.9.9-3.fc30.x86_64                                                                              2/2 
  Running scriptlet: fuse-2.9.9-3.fc30.x86_64                                                                              2/2 
  Verifying        : fuse-2.9.9-7.fc30.x86_64                                                                              1/2 
  Verifying        : fuse-2.9.9-3.fc30.x86_64                                                                              2/2 

Upgraded:
  fuse-2.9.9-7.fc30.x86_64                                                                                                     

Complete!
[root@bookcase boinc]# dnf update
Last metadata expiration check: 0:06:13 ago on Mon 27 May 2019 15:36:09 BST.
Dependencies resolved.
===============================================================================================================================
 Package                         Architecture               Version                          Repository                   Size
===============================================================================================================================
Upgrading:
 fuse-common                     x86_64                     3.5.0-1.fc30                     updates                     7.4 k

Transaction Summary
===============================================================================================================================
Upgrade  1 Package

Total download size: 7.4 k
Is this ok [y/N]: y
Downloading Packages:
fuse-common-3.4.2-6.fc30_3.5.0-1.fc30.x86_64.drpm                                               26 kB/s | 4.6 kB     00:00    
[DRPM] fuse-common-3.4.2-6.fc30_3.5.0-1.fc30.x86_64.drpm: done                                                                
-------------------------------------------------------------------------------------------------------------------------------
Total                                                                                          8.7 kB/s | 4.6 kB     00:00     
Delta RPMs reduced 0.0 MB of updates to 0.0 MB (38.1% saved)
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                       1/1 
  Upgrading        : fuse-common-3.5.0-1.fc30.x86_64                                                                       1/2 
  Cleanup          : fuse-common-3.4.2-6.fc30.x86_64                                                                       2/2 
  Verifying        : fuse-common-3.5.0-1.fc30.x86_64                                                                       1/2 
  Verifying        : fuse-common-3.4.2-6.fc30.x86_64                                                                       2/2 

Upgraded:
  fuse-common-3.5.0-1.fc30.x86_64                                                                                              

Complete!
[root@bookcase boinc]# dnf update
Last metadata expiration check: 0:06:43 ago on Mon 27 May 2019 15:36:09 BST.
Dependencies resolved.
Nothing to do.
Complete!
[root@bookcase boinc]#

Comment 9 Michal Domonkos 2019-05-27 15:06:57 UTC
Please note that DNF does not use python-urlgrabber at all.

Comment 10 Ben Cotton 2019-05-28 18:55:09 UTC
Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 11 Fedora Update System 2019-06-06 01:05:58 UTC
python-urlgrabber-4.0.0-3.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.