Bug 2238915

Summary:	Very large content import fails with subtask error
Product:	Red Hat Satellite	Reporter:	Keith Williams <keiwilli>
Component:	Pulp	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED DUPLICATE	QA Contact:	Satellite QE Team <sat-qe-bz-list>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.11.5	CC:	ahumbe, dalley, ipanova, juqiao, rlavi
Target Milestone:	6.15.0	Keywords:	Performance, Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-10-12 00:06:32 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Keith Williams 2023-09-14 10:02:18 UTC

Description of problem:
During very large (2.1TB) content-export, the import fails after about 12 days with error "7 subtask(s) failed for task group /pulp/api/v3/task-groups/....."

This BZ is raised in relation to support case #03609607.

Version-Release number of selected component (if applicable):
6.11.5.4

How reproducible:
Everytime

Steps to Reproduce:
1. place 2.1TB tar.gz file in to /var/lib/pulp/imports
2. run command: time hammer content-import version --organization-id=1 --path/var/lib/pulp/imports/z1d-export/


Actual results:
import fails after about 12 days with error "7 subtask(s) failed for task group /pulp/api/v3/task-groups/....."

Expected results:
import does not fail

Additional info:
Please see the support case #03609607 which has a lot more contextual information.
Please see https://bugzilla.redhat.com/show_bug.cgi?id=2238914.
Test of importing a 2.1TB content export that contains:
Repositories: 151 (both Red Hat and 3rd party repos)
RPM Packages: >200,000
Size of export: 2.1TB
I did find a knowledgebase article https://access.redhat.com/solutions/6967697 which relates to a lack of memory. This was fixed in 6.11.2 and we are running 6.11.5.4. Also on this host it has 24cpu and 192GB of memory and no out of memory issues were observed when the import failed.

Comment 2 Ina Panova 2023-10-03 14:20:24 UTC

I went through the customer case - I did not find any attachments to it with any additional info as well went through the comments searching for the logs or tracebacks.
To facilitate the bugzilla investigation and invest the time into its resolution - can logs, tracebacks, task info output be shared directly on the BZ?

Comment 3 Daniel Alley 2023-10-12 00:06:32 UTC

The main issue here is 2226950

We're working on some additional memory improvements beyond that that will be covered by a new BZ

*** This bug has been marked as a duplicate of bug 2226950 ***