Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2212523 - Content-export incremental with syncable format based does not include productid file into repodata directory
Summary: Content-export incremental with syncable format based does not include produc...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.12.4
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: 6.14.0
Assignee: satellite6-bugs
QA Contact: Vladimír Sedmík
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-05 19:37 UTC by Joniel Pasqualetto
Modified: 2023-11-08 14:19 UTC (History)
11 users (show)

Fixed In Version: pulpcore-3.22.8-1, pulpcore-3.21.11-1, pulpcore-3.18.21-1, pulpcore-3.28.16-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2238369 (view as bug list)
Environment:
Last Closed: 2023-11-08 14:19:28 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pulp pulpcore issues 3941 0 None closed fs_exporter does not include productid certificate on incremental exports 2023-07-12 17:15:09 UTC
Github pulp pulpcore pull 3942 0 None closed Fixes #3941 2023-07-12 17:15:13 UTC
Red Hat Issue Tracker SAT-18275 0 None None None 2023-06-12 13:34:42 UTC
Red Hat Knowledge Base (Solution) 7022271 0 None None None 2023-08-03 15:54:43 UTC
Red Hat Product Errata RHSA-2023:6818 0 None None None 2023-11-08 14:19:41 UTC

Description Joniel Pasqualetto 2023-06-05 19:37:50 UTC 
Description of problem:

Generating an incremental export with syncable format, generates an export with a missing file on the repodata directory.


Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Create a CV with a repository that contains a productid (example repo: Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs )
2. Do a complete export of the CV:

 hammer content-export complete version --id 41 --format syncable

3. Create a new version of the CV
4. Export CV again, this time an incremental:

~~~
hammer content-export incremental version --id 42 --format syncable
~~~

Note the files on the repodata directory:

~~~
# ls |sort
2ad01aecc583cdede5a1489ee50efd97c48b923b586d24fb83b42f5c9103b85b-other.xml.gz
5ac9d773403beea431e1d08255b782b8bb38e61cd458254bb0cbde907f444a77-updateinfo.xml.gz
641203f1aa0962bbb1af40e20142c824e669323aaee862acd3eea80e046ccc92-primary.xml.gz
67120121d59a4351342ec001ccc4e7ad4520175029770bbda447c0186ed9f1e9-filelists.sqlite.bz2
8e91c89c9670575d4123bc37cd0cfb9e1f7a57a9b9b93c2491a67678fa7ec9d4-filelists.xml.gz
a27718cc28ec6d71432e0ef3e6da544b7f9d93f6bb7d0a55aacd592d03144b70-comps.xml
ee7bbcac9b13fa0a8fe3aa083e4200c5391c945b76feb59e887c4d2e36944371-other.sqlite.bz2
f8d2107d953c43ede24bf85033c8ca025f1b32194e99b3919231ab0ce35c217a-primary.sqlite.bz2
repomd.xml
~~~

Looking inside repomd.xml, there will be a reference to file of data type "productid" which is missing:

~~~
# grep productid repomd.xml -A2
  <data type="productid">
    <checksum type="sha256">55a5ce52b4fe56961ccab8f58d2ac36230d09491d627e09f685d4b2ddd50e55b</checksum>
    <location href="repodata/7034ae03-a219-4911-9337-2118fa5768e8"/>
~~~

Actual results:
productid file is missing from repodata

Expected results:
A complete export of the repodata. An incremental export should only affect artifacts and not metadata.


Additional info:
Comment 5 Vladimír Sedmík 2023-07-31 14:06:03 UTC 
Verified in 6.14.0 snap 9 (python39-pulpcore-3.22.9-1.1.el8pc.noarch)

Now the incremental syncable exports contain the productid file:

1) Create and sync (immediate policy) the ansible-2-for-rhel-8-x86_64-rpms repo, add it to a CV, publish.

2) Run the complete export of the CV:
[root@sat ~]# hammer content-export complete version --id 5 --format syncable
[..................................................................................] [100%]
Generated /var/lib/pulp/exports/BZ-2212523/CV-1/1.0/2023-07-31T07-40-44-04-00/metadata.json

3) Publish new CV version and run incremental export of it:
[root@sat ~]# hammer content-export incremental version --id 6 --format syncable
[..................................................................................] [100%]
Generated /var/lib/pulp/exports/BZ-2212523/CV-1/2.0/2023-07-31T07-43-16-04-00/metadata.json

4) Check the productid in present the incremental export:
[root@sat ~]# ll /var/lib/pulp/exports/BZ-2212523/CV-1/2.0/2023-07-31T07-43-16-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/
total 5048
-rw-r--r--. 3 pulp pulp   21562 Jul 31 05:50 2ad01aecc583cdede5a1489ee50efd97c48b923b586d24fb83b42f5c9103b85b-other.xml.gz
-rw-r--r--. 3 pulp pulp    9190 Jul 31 05:50 5ac9d773403beea431e1d08255b782b8bb38e61cd458254bb0cbde907f444a77-updateinfo.xml.gz
-rw-r--r--. 3 pulp pulp    8065 Jul 31 05:50 641203f1aa0962bbb1af40e20142c824e669323aaee862acd3eea80e046ccc92-primary.xml.gz
-rw-r--r--. 3 pulp pulp 2411759 Jul 31 05:50 67120121d59a4351342ec001ccc4e7ad4520175029770bbda447c0186ed9f1e9-filelists.sqlite.bz2
-rw-r--r--. 3 pulp pulp    2139 Jul 31 05:50 7034ae03-a219-4911-9337-2118fa5768e8                                                    <======= here
-rw-r--r--. 3 pulp pulp 2553565 Jul 31 05:50 8e91c89c9670575d4123bc37cd0cfb9e1f7a57a9b9b93c2491a67678fa7ec9d4-filelists.xml.gz
-rw-r--r--. 3 pulp pulp     124 Jul 31 05:50 a27718cc28ec6d71432e0ef3e6da544b7f9d93f6bb7d0a55aacd592d03144b70-comps.xml
-rw-r--r--. 3 pulp pulp  109732 Jul 31 05:50 ee7bbcac9b13fa0a8fe3aa083e4200c5391c945b76feb59e887c4d2e36944371-other.sqlite.bz2
-rw-r--r--. 3 pulp pulp   27208 Jul 31 05:50 f8d2107d953c43ede24bf85033c8ca025f1b32194e99b3919231ab0ce35c217a-primary.sqlite.bz2
-rw-r--r--. 3 pulp pulp    4134 Jul 31 05:50 repomd.xml

5) Check it matches the repomd:
[root@sat ~]# cat /var/lib/pulp/exports/BZ-2212523/CV-1/2.0/2023-07-31T07-43-16-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/repomd.xml | grep productid -A 5
  <data type="productid">
    <checksum type="sha256">55a5ce52b4fe56961ccab8f58d2ac36230d09491d627e09f685d4b2ddd50e55b</checksum>
    <location href="repodata/7034ae03-a219-4911-9337-2118fa5768e8"/>
    <timestamp>1602714155</timestamp>
    <size>2139</size>
  </data>
[root@sat ~]# sha256sum /var/lib/pulp/exports/BZ-2212523/CV-1/2.0/2023-07-31T07-43-16-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/7034ae03-a219-4911-9337-2118fa5768e8
55a5ce52b4fe56961ccab8f58d2ac36230d09491d627e09f685d4b2ddd50e55b  /var/lib/pulp/exports/BZ-2212523/CV-1/2.0/2023-07-31T07-43-16-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/7034ae03-a219-4911-9337-2118fa5768e8

6) Check it's the same as the complete export:
[root@sat ~]# diff -s \
> /var/lib/pulp/exports/BZ-2212523/CV-1/2.0/2023-07-31T07-43-16-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/7034ae03-a219-4911-9337-2118fa5768e8 \
> /var/lib/pulp/exports/BZ-2212523/CV-1/1.0/2023-07-31T07-40-44-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/7034ae03-a219-4911-9337-2118fa5768e8
Files /var/lib/pulp/exports/BZ-2212523/CV-1/2.0/2023-07-31T07-43-16-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/7034ae03-a219-4911-9337-2118fa5768e8 and /var/lib/pulp/exports/BZ-2212523/CV-1/1.0/2023-07-31T07-40-44-04-00/content/dist/layered/rhel8/x86_64/ansible/2/os/repodata/7034ae03-a219-4911-9337-2118fa5768e8 are identical
Comment 9 errata-xmlrpc 2023-11-08 14:19:28 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.14 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6818
Note You need to log in before you can comment on or make changes to this bug.