Bug 1452212 - File repo type does not handle updates to files on manifest during re-sync correctly
Summary: File repo type does not handle updates to files on manifest during re-sync co...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Lukáš Hellebrandt
URL:
Whiteboard:
Depends On:
Blocks: 1575193
TreeView+ depends on / blocked
 
Reported: 2017-05-18 14:59 UTC by Craig Donnelly
Modified: 2021-04-06 17:50 UTC (History)
16 users (show)

Fixed In Version: pulp-2.13.3
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1575193 (view as bug list)
Environment:
Last Closed: 2018-02-21 16:54:37 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Pulp Redmine 2773 0 High CLOSED - CURRENTRELEASE ISO repo does not handle updates to files on manifest during re-sync correctly 2017-07-20 18:02:04 UTC
Pulp Redmine 3047 0 Normal CLOSED - CURRENTRELEASE ISO repo doesn't correctly handle updates to files for content already in Pulp 2019-05-16 15:32:48 UTC

Description Craig Donnelly 2017-05-18 14:59:11 UTC
Description of problem:

Currently, if you generate a PULP_MANIFEST and sync from another location via URL, you will receive the files in the Manifest and they will become locally available on the Satellite, but if you UPDATE a file that has already been imported and regenerate the PULP_MANIFEST, then sync again, the Satellite's versions of the manifest will have a duplicate file name entry with separate sha256sums and sizes, but will not retrieve the new file (It will maintain the original in the repo).

Version-Release number of selected component (if applicable):
nightly

How reproducible:
100%

Steps to Reproduce:
1. Generate PULP_MANIFEST and sync a File repo from a http based URL
2. Make a change to a file (text file, etc..), but DO NOT change the file NAME.
3. Re-Generate the PULP_MANIFEST for the remote file repo, and then re-sync on Satellite.

Actual results:
File that was modified will have two entries on the PULP_MANIFEST inside the local Satellite copy of the repo with separate sha256sums and sizes, but identical name, and only the original copy of the file prior to editing will exist in the repo.

Expected results:
We should replace original file with the newest checksum. (Act like mirror-on-sync?)

Additional info:
Perhaps the File repo type should be able to determine the time of PULP_MANIFEST generation and make a determination about what should exist based off of that? (i.e. Include a date inside the PULP_MANIFEST on creation)

Comment 1 Michael Hrivnak 2017-05-19 19:24:53 UTC
Can you add the specific Pulp version that was in use? Thanks!

Comment 2 pulp-infra@redhat.com 2017-05-19 19:37:02 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 3 pulp-infra@redhat.com 2017-05-19 19:37:05 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 4 Craig Donnelly 2017-05-19 19:41:03 UTC
Michael,

During this test I had:

pulp-server-2.9.3-2.el7sat.noarch

Comment 5 pulp-infra@redhat.com 2017-05-23 15:04:05 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 6 pulp-infra@redhat.com 2017-06-19 19:31:57 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2017-06-22 19:32:06 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 8 pulp-infra@redhat.com 2017-07-03 15:04:53 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 9 pulp-infra@redhat.com 2017-07-03 15:33:45 UTC
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

Comment 10 pulp-infra@redhat.com 2017-07-20 17:32:20 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 11 pulp-infra@redhat.com 2017-07-20 18:02:05 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 12 Lukáš Hellebrandt 2017-09-26 12:49:59 UTC
FailedQA with Sat6.3, snap 17. Thanks for advice to rplevka.

As manipulating Pulp directly is not supported is Satellite, we can use the following reproducer:

1) Create Product
2) In Product, create Repo1 with empty URL
3) Upload File to Repo1
4) In Product, create Repo2 with URL=<content_url_of_repo_1> [1]
5) Sync Repo2
6) Change contents of File
7) In any of Repo1 or Repo2 (works both ways), upload File
8) Sync Repo2 again
9) In content URL of Repo2, view contents of PULP_MANIFEST file

Actual results:
Two entries with the same filename and different hash

Expected results:
Not sure... how do we decide which version of the file to keep?


[1] for example, http://<FQDN>/pulp/isos/e2f04116-1ce6-4390-a781-75d00299c57f/

Comment 13 pulp-infra@redhat.com 2017-09-26 13:01:59 UTC
Requesting needsinfo from upstream developer daviddavis because the 'FailedQA' flag is set.

Comment 14 David Davis 2017-09-28 19:31:10 UTC
Found the bug and opened a new upstream issue.

Comment 15 pulp-infra@redhat.com 2017-09-28 19:32:20 UTC
Requesting needsinfo from upstream developer daviddavis because the 'FailedQA' flag is set.

Comment 16 pulp-infra@redhat.com 2017-09-28 20:37:39 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 17 pulp-infra@redhat.com 2017-09-28 20:37:43 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 18 pulp-infra@redhat.com 2017-10-03 17:01:29 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 19 Tanya Tereshchenko 2017-10-03 21:21:08 UTC
Upstream bug is fixed and merged, moving to POST

Comment 20 Lukáš Hellebrandt 2017-10-17 08:45:21 UTC
Verified with Sat6.3 snap 20.

Used reproducer from comment 12. After that, there were only the ew file's metadata and only the new file in both repositories.

Comment 21 pulp-infra@redhat.com 2017-10-19 14:01:56 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 22 pulp-infra@redhat.com 2017-10-25 21:32:07 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 23 pulp-infra@redhat.com 2017-10-26 15:01:58 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 24 pulp-infra@redhat.com 2017-10-26 23:31:53 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 25 pulp-infra@redhat.com 2017-10-30 20:31:38 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 26 pulp-infra@redhat.com 2017-11-16 20:32:38 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 27 Satellite Program 2018-02-21 16:54:37 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA.
> > 
> > For information on the advisory, and where to find the updated files, follow the link below.
> > 
> > If the solution does not work for you, open a new bug report.
> > 
> > https://access.redhat.com/errata/RHSA-2018:0336

Comment 28 pulp-infra@redhat.com 2019-05-16 15:32:49 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.


Note You need to log in before you can comment on or make changes to this bug.