Bug 1048837

Description of problem:

Repository sync-ed from cdn.redhat.com contains duplicate rpm's. One copy seems to have broken signatures(?)

This is just one package example, I see this issue with about half the packages yum update tries to download.

# http://yum-act.uk.XXX/pulp/repos/latest/rhel-x86_64-server-6/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=YYYY clean metadata
Trying other mirror.

bash-4.1$ pulp-admin rpm repo content rpm --repo-id rhel-x86_64-server-6 --match name=device-mapper-persistent-data --match version=0.2.8 --match release=2
Arch:         x86_64
Buildhost:    x86-002.build.bos.redhat.com
Checksum:     69c365f820d1d0a3504a27ea9b9acba444919463
Checksumtype: sha
Description:  thin-provisioning-tools contains check,dump,restore,repair,rmap
              and metadata_size tools to manage device-mapper thin provisioning
              target metadata devices; cache check,dump,restore and repair tools
              to manage device-mapper cache metadata devices are included.
Epoch:        0
Filename:     device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm
License:      GPLv3+
Name:         device-mapper-persistent-data
Provides:     device-mapper-persistent-data = 0.2.8-2.el6-0,
              device-mapper-persistent-data(x86-64) = 0.2.8-2.el6-0
Release:      2.el6
Requires:     expat, libc.so.6()(64bit), libc.so.6(GLIBC_2.2.5)(64bit),
              libc.so.6(GLIBC_2.3.4)(64bit), libc.so.6(GLIBC_2.4)(64bit),
              libexpat.so.1()(64bit), libgcc_s.so.1()(64bit),
              libgcc_s.so.1(GCC_3.0)(64bit), libm.so.6()(64bit),
              libm.so.6(GLIBC_2.2.5)(64bit), libstdc++.so.6()(64bit),
              libstdc++.so.6(CXXABI_1.3)(64bit),
              libstdc++.so.6(GLIBCXX_3.4)(64bit),
              libstdc++.so.6(GLIBCXX_3.4.11)(64bit),
              libstdc++.so.6(GLIBCXX_3.4.9)(64bit), rtld(GNU_HASH)
Vendor:       None
Version:      0.2.8

Arch:         x86_64
Buildhost:    x86-002.build.bos.redhat.com
Checksum:     b7bd103de7115516bb7e98e36b614bdb3bae6b00
Checksumtype: sha
Description:  thin-provisioning-tools contains check,dump,restore,repair,rmap
              and metadata_size tools to manage device-mapper thin provisioning
              target metadata devices; cache check,dump,restore and repair tools
              to manage device-mapper cache metadata devices are included.
Epoch:        0
Filename:     device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm
License:      GPLv3+
Name:         device-mapper-persistent-data
Provides:     device-mapper-persistent-data = 0.2.8-2.el6-0,
              device-mapper-persistent-data(x86-64) = 0.2.8-2.el6-0
Release:      2.el6
Requires:     expat, libc.so.6()(64bit), libc.so.6(GLIBC_2.2.5)(64bit),
              libc.so.6(GLIBC_2.3.4)(64bit), libc.so.6(GLIBC_2.4)(64bit),
              libexpat.so.1()(64bit), libgcc_s.so.1()(64bit),
              libgcc_s.so.1(GCC_3.0)(64bit), libm.so.6()(64bit),
              libm.so.6(GLIBC_2.2.5)(64bit), libstdc++.so.6()(64bit),
              libstdc++.so.6(CXXABI_1.3)(64bit),
              libstdc++.so.6(GLIBCXX_3.4)(64bit),
              libstdc++.so.6(GLIBCXX_3.4.11)(64bit),
              libstdc++.so.6(GLIBCXX_3.4.9)(64bit), rtld(GNU_HASH)
Vendor:       None
Version:      0.2.8

bash-4.1$ cd /opt/pulp/content/rpm/device-mapper-persistent-data/0.2.8/2.el6/x86_64
bash-4.1$ find . -type f -name \*.rpm -exec ls -l {} \;
-rw-r--r--. 1 apache apache 2444388 Nov 21 16:06 ./b7bd103de7115516bb7e98e36b614bdb3bae6b00/Packages/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm
-rw-r--r--. 1 apache apache 2444388 Nov 21 08:06 ./69c365f820d1d0a3504a27ea9b9acba444919463/Packages/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm



bash-4.1$ find . -type f -name \*.rpm -exec rpm -q --checksig -p {} \;
./b7bd103de7115516bb7e98e36b614bdb3bae6b00/Packages/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm: rsa sha1 (md5) pgp md5 OK
./69c365f820d1d0a3504a27ea9b9acba444919463/Packages/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING KEYS: (MD5) PGP#f21541eb)

Note: That GPG key is public key "Red Hat, Inc. (beta key 2) <security>" ??

The repo looks like this:
Id:                  rhel-x86_64-server-6
Display Name:        rhel-x86_64-server-6
Description:         None
Content Unit Counts:
  Distribution:           1
  Erratum:                2256
  Package Category:       10
  Package Group:          202
  Rpm:                    12507
  Yum Repo Metadata File: 1
Notes:
Importers:
  Config:
    Feed:                https://cdn.redhat.com/content/dist/rhel/server/6/6Serv
                         er/x86_64/os
    Feed SSL Configured: True
    Proxy Host:          https://ukproxy.bip.XXXX
    Proxy Port:          8000
    SSL CA Cert:         -----BEGIN CERTIFICATE-----
...
  Id:               yum_importer
  Importer Type Id: yum_importer
  Last Sync:        2014-01-06T08:09:30Z
  Repo Id:          rhel-x86_64-server-6
  Scheduled Syncs:  2012-12-15T00:00Z/PT4H
Distributors:
  Auto Publish:        True
  Config:
    Checksum Type: sha1
    Gpgkey:        The following public key can be used to verify RPM packages
                   built and signed by Red Hat, Inc.  This key is used for
                   packages in Red Hat products shipped after November 2009, and
                   for all updates to those products.  Questions about this key
                   should be sent to security.  pub  4096R/FD431D51
                   2009-10-22 Red Hat, Inc. (release key 2)
...
    Http:          True
    Https:         True
    Relative URL:  latest/rhel-x86_64-server-6
  Distributor Type Id: yum_distributor
  Id:                  yum_distributor
  Last Publish:        2014-01-06T08:13:34Z
  Repo Id:             rhel-x86_64-server-6
  Scheduled Publishes:

In pulp.log:
pulp.log:2014-01-06 08:09:34,360 pulp.plugins.pulp_rpm.yum_plugin.util:WARNING: Removing </var/lib/pulp/working/repos/rhel-x86_64-server-6/distributors/yum_distributor/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm> since it was pointing to </var/lib/pulp/content/rpm/device-mapper-persistent-data/0.2.8/2.el6/x86_64/b7bd103de7115516bb7e98e36b614bdb3bae6b00/Packages/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm> and not </var/lib/pulp/content/rpm/device-mapper-persistent-data/0.2.8/2.el6/x86_64/69c365f820d1d0a3504a27ea9b9acba444919463/Packages/device-mapper-persistent-data-0.2.8-2.el6.x86_64.rpm>


It won't let me remove the duplicates either:

pulp-admin rpm repo remove rpm --repo-id rhel-x86_64-server-6 --match name=device-mapper-persistent-data --match version=0.2.8 --match release=2 --match checksum=69c365f820d1d0a3504a27ea9b9acba444919463

Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux Server release 6.4 (Santiago)
# rpm -qa |grep ^pulp
pulp-nodes-parent-2.3.1-1.el6.noarch
pulp-puppet-admin-extensions-2.3.1-1.el6.noarch
pulp-admin-client-2.3.1-1.el6.noarch
pulp-server-2.3.1-1.el6.noarch
pulp-nodes-admin-extensions-2.3.1-1.el6.noarch
pulp-rpm-plugins-2.3.1-1.el6.noarch
pulp-rpm-admin-extensions-2.3.1-1.el6.noarch
pulp-rpm-handlers-2.3.1-1.el6.noarch
pulp-nodes-common-2.3.1-1.el6.noarch
pulp-puppet-plugins-2.3.1-1.el6.noarch


How reproducible:
We have two separate pulp servers, each downloading from cdn and they both have the same issue but with different packages.

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: