Bug 1985958

Summary: [pulp3] Sync fails for repositories with duplicated content by checksum or by path
Product: Red Hat Satellite Reporter: Tanya Tereshchenko <ttereshc>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Stephen Wadeley <swadeley>
Severity: high Docs Contact:
Priority: high    
Version: 6.10.0CC: ggainey, rchan, swadeley, ttereshc, zhunting
Target Milestone: 6.10.0Keywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1992274 (view as bug list) Environment:
Last Closed: 2021-11-16 14:12:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tanya Tereshchenko 2021-07-26 11:11:20 UTC
Description of problem:
Pulp 3 currently doesn't handle well the following situations:
 - 2 packages with the same NEVRA, same checksum but different filenames/location
      - e.g. https://packages.grafana.com/oss/rpm
          - grafana-2.6.0-1.x86_64.rpm
          - grafana-2.6.0.x86_64.rpm

      - it can be just 2 different repositories having the same package under different paths

 - 2 packages with the same NEVRA, different checksums, same filenames but in different location/directory
      - e.g. http://ftp.cs.stanford.edu/centos/7/opstools/x86_64/
          - fluentd/rubygem-multipart-post-doc-2.0.0-2.el7.noarch.rpm
hash 6a8148297b09c9bb7fa433e1559e20760b21c6d9cf10eb8569e053704b20d8c9
          - logging/rubygem-multipart-post-doc-2.0.0-2.el7.noarch.rpm
hash 48e054113e7bb6b4b52d9c34726f5a58ccdc8045bfe5e15f687306ef935f49d3


It's a regression in comparison to pulp2 which just picks one.

Version-Release number of selected component (if applicable):
pulp_rpm 3.14.0 and older


How reproducible:
Always


Steps to Reproduce:
Sync repositories from the description, and observe errors.


Actual results:
There are 2 different errors but they are related.
They can't be fixed/tested separately.

You get either
`Cannot create repository version. Path is duplicated: <duplicated RPM>.`
or
`ValueError: No declared artifact with relative path X for content "<Package: Y>"`

Expected results:
Pick one and log warnings about duplicated or overlapped paths.

Comment 1 pulp-infra@redhat.com 2021-07-26 18:10:04 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 2 pulp-infra@redhat.com 2021-07-26 18:10:06 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 3 pulp-infra@redhat.com 2021-07-26 18:10:27 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 4 pulp-infra@redhat.com 2021-07-26 18:10:29 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 5 pulp-infra@redhat.com 2021-07-28 20:09:03 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2021-08-09 17:17:21 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 8 pulp-infra@redhat.com 2021-08-09 17:17:41 UTC
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

Comment 16 pulp-infra@redhat.com 2021-08-26 13:16:54 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 17 pulp-infra@redhat.com 2021-08-27 16:12:40 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 18 pulp-infra@redhat.com 2021-10-12 14:08:30 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 19 pulp-infra@redhat.com 2021-10-12 14:08:31 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 22 errata-xmlrpc 2021-11-16 14:12:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4702