Bug 2226950

Summary: Size of a "Complete Library export" can be double the consumption of /var/lib/pulp/media/artifact/
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: POST --- QA Contact: Satellite QE Team <sat-qe-bz-list>
Severity: high Docs Contact:
Priority: high    
Version: 6.13.3CC: dalley, dkliban, ggainey, jpasqual, mjia, paji, pcreech, rchan, rlavi
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: pulpcore-3.28.8, pulpcore-3.23.13, pulpcore-3.22.12, pulpcore-3.21.15, pulpcore-3.18.25 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2227913 2231504 (view as bug list) Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sayan Das 2023-07-27 06:14:05 UTC
Description of problem:

Size of a "Complete Library export" can be double the consumption of /var/lib/pulp/media/artifact/  when the same base repositories are part of multiple CVs and CCVs ( i.e. multiple copies of the same repo and artifacts exist).

See https://github.com/pulp/pulpcore/issues/4159 for more info.

Version-Release number of selected component (if applicable):

Satellite 6.13 


How reproducible:

Undetermined at present but possible to reproduce in the customer's environment. 


Steps to Reproduce: 

Probably:

1. Enable and sync these two repos with immediate download policies

- Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8.8
- Red Hat Enterprise Linux 8 for x86_64 - BaseOS Kickstart 8.8

2. Add them to a CV and publish a new version. 

3. Make 20 - 50 copies of that CV and Publish a new version for all of them.

4. Check the disk space consumption of  /var/lib/pulp/media/artifact/ directory.

5. Perform a full library export for that organization.

6. Check the size of export tar in the exported directory. 


Actual results:

Size noted in Step 6 is much bigger ( almost double ) than the size noted in Step 4.


Expected results:

The exported archive should never exceed the total size of artifacts downloaded in pulp


Additional info:

It might not be 100% possible to reproduce the issue with just RPM repos and the reproducer may need to include a file type repo or a docker type repo.

Comment 7 Robin Chan 2023-07-29 16:04:45 UTC
The Pulp upstream bug status is at closed. Updating the external tracker on this bug.