Bug 1527417

Description of problem:

Some packagers tend to upload a newer rpm to their repository with the same name-version-release combination as the existing rpm. That leads to different checksums and might also lead to different filesizes.

reposync fails when:

(a) the remote size is bigger than the local copy

here, reposync only checks whether or not the file already exists _and_ the file sizes match. If that's the case, remove the file from the list. But not really, because there's still a checksum verification (see (b)). Anyways, if the local file size differs from the remote size, the file gets downloaded with a "Range:" header. If the remote file is smaller than the local copy, this results in a "416 range not satisfiable".

(b) the remote size is equal to the local copy but the checksum differs

reposync would skip this package but still the yum checksum verification kicks. Since yum doesn't do anything to mitigate that situation, reposync fails.


Version-Release number of selected component (if applicable):
yum-3.4.3-154
yum-utils-1.1.31-42


How reproducible:
always


Steps to Reproduce:
(a)
1. echo "foo" >> $existingpackage
2. reposync it
3. see reposync fail with a 416 error

(b)
1. dd if=/dev/urandom of=$existingpackage bs=100 count=1 conv=notrunc
2. reposync it
3. see reposync fail with "$package FAILED\n$package: [Errno 256] No more mirrors to try."


Actual results:
reposync doesn't redownload the file and fails


Expected results:
reposync removes the local copies and succeeds


Additional info:

In #1337957 Jens Kuehnel suggests to patch reposync to remove the local copy if its size is bigger than the remote copy's.
I think yum should handle this. In both cases, if the files differs in either size and/or content, the checksum will fail.
Yum should delete the local copy, I thought of something like that:

$ diff -up /usr/lib/python2.7/site-packages/yum/__init__.py{.orig,}
--- /usr/lib/python2.7/site-packages/yum/__init__.py.orig
+++ /usr/lib/python2.7/site-packages/yum/__init__.py
@@ -2469,6 +2469,8 @@ much more problems).
                     adderror(po, _('package fails checksum but caching is '
                         'enabled for %s') % po.repo.id)
                     return False
+                self.verbose_logger.debug("checksum failed for %s, removing local copy" % po)
+                os.remove(local)
             if downloadonly:
                 po.localpath += '.%d.tmp' % os.getpid()
                 try: os.rename(local, po.localpath)


examples for such packages:

(a) newer package is smaller
$ stat -c '%n: %s' libgeotiff-1.4.0-1.rhel7.x86_64.rpm*
libgeotiff-1.4.0-1.rhel7.x86_64.rpm: 725296
libgeotiff-1.4.0-1.rhel7.x86_64.rpm.orig: 725332
$ diff -y <(rpm -qip libgeotiff-1.4.0-1.rhel7.x86_64.rpm) <(rpm -qip libgeotiff-1.4.0-1.rhel7.x86_64.rpm.orig)
Name        : libgeotiff                                        Name        : libgeotiff
Version     : 1.4.0                                             Version     : 1.4.0
Release     : 1.rhel7                                           Release     : 1.rhel7
Architecture: x86_64                                            Architecture: x86_64
Install Date: (not installed)                                   Install Date: (not installed)
Group       : System Environment/Libraries                      Group       : System Environment/Libraries
Size        : 5635275                                         | Size        : 5635254
License     : MIT                                               License     : MIT
Signature   : DSA/SHA1, Mo 18 Dez 2017 20:38:36 CET, Key ID 1 | Signature   : DSA/SHA1, Fr 15 Dez 2017 04:02:22 CET, Key ID 1
Source RPM  : libgeotiff-1.4.0-1.rhel7.src.rpm                  Source RPM  : libgeotiff-1.4.0-1.rhel7.src.rpm
Build Date  : Mo 18 Dez 2017 20:38:30 CET                     | Build Date  : Fr 15 Dez 2017 04:02:19 CET
Build Host  : koji-rhel7-x86-64-pgbuild                         Build Host  : koji-rhel7-x86-64-pgbuild
Relocations : (not relocatable)                                 Relocations : (not relocatable)
Vendor      : PostgreSQL Global Development Group               Vendor      : PostgreSQL Global Development Group
URL         : http://trac.osgeo.org/geotiff/                    URL         : http://trac.osgeo.org/geotiff/
Summary     : GeoTIFF format library                            Summary     : GeoTIFF format library
Description :                                                   Description :
GeoTIFF represents an effort by over 160 different remote sen   GeoTIFF represents an effort by over 160 different remote sen
GIS, cartographic, and surveying related companies and organi   GIS, cartographic, and surveying related companies and organi
to establish a TIFF based interchange format for georeference   to establish a TIFF based interchange format for georeference
raster imagery.                                                 raster imagery.

(b) same package size:
$ stat -c '%n: %s' libgeotiff-debuginfo-1.4.0-1.rhel7.x86_64.rpm*
libgeotiff-debuginfo-1.4.0-1.rhel7.x86_64.rpm: 199736
libgeotiff-debuginfo-1.4.0-1.rhel7.x86_64.rpm.orig: 199736
$ diff -y <(rpm -qip libgeotiff-debuginfo-1.4.0-1.rhel7.x86_64.rpm) <(rpm -qip libgeotiff-debuginfo-1.4.0-1.rhel7.x86_64.rpm.orig)
Name        : libgeotiff-debuginfo                              Name        : libgeotiff-debuginfo
Version     : 1.4.0                                             Version     : 1.4.0
Release     : 1.rhel7                                           Release     : 1.rhel7
Architecture: x86_64                                            Architecture: x86_64
Install Date: (not installed)                                   Install Date: (not installed)
Group       : Development/Debug                                 Group       : Development/Debug
Size        : 917315                                          | Size        : 917294
License     : MIT                                               License     : MIT
Signature   : DSA/SHA1, Mo 18 Dez 2017 20:38:36 CET, Key ID 1 | Signature   : DSA/SHA1, Fr 15 Dez 2017 04:02:22 CET, Key ID 1
Source RPM  : libgeotiff-1.4.0-1.rhel7.src.rpm                  Source RPM  : libgeotiff-1.4.0-1.rhel7.src.rpm
Build Date  : Mo 18 Dez 2017 20:38:30 CET                     | Build Date  : Fr 15 Dez 2017 04:02:19 CET
Build Host  : koji-rhel7-x86-64-pgbuild                         Build Host  : koji-rhel7-x86-64-pgbuild
Relocations : (not relocatable)                                 Relocations : (not relocatable)
Vendor      : PostgreSQL Global Development Group               Vendor      : PostgreSQL Global Development Group
URL         : http://trac.osgeo.org/geotiff/                    URL         : http://trac.osgeo.org/geotiff/
Summary     : Debug information for package libgeotiff          Summary     : Debug information for package libgeotiff
Description :                                                   Description :
This package provides debug information for package libgeotif   This package provides debug information for package libgeotif
Debug information is useful when developing applications that   Debug information is useful when developing applications that
package or when debugging this package.                         package or when debugging this package.