Bug 2150116 - Entitlement certificate is missing content section for a custom product
Summary: Entitlement certificate is missing content section for a custom product
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Candlepin
Version: 6.8.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: 6.12.2
Assignee: Chris Roberts
QA Contact: Vladimír Sedmík
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-12-01 21:03 UTC by Odilon Sousa
Modified: 2023-07-11 10:11 UTC (History)
53 users (show)

Fixed In Version: candlepin-4.1.19-1.el8sat
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1931027
Environment:
Last Closed: 2023-02-08 14:35:01 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker SAT-14174 0 None None None 2022-12-01 21:21:44 UTC
Red Hat Product Errata RHBA-2023:0668 0 None None None 2023-02-08 14:35:17 UTC

Comment 1 Pavel Moravec 2023-01-03 12:28:51 UTC 
Reproducer using whole Satellite (and not communicating with candlepin directly):

0) configure candlepin to accept API requests from cmdline, per https://access.redhat.com/solutions/2955931 - alternativelly, configure OrphanCleanup per https://bugzilla.redhat.com/show_bug.cgi?id=1931027#c110 to be fired every 5 minutes. But invoking the OrphanCleanup from cmdline is more reliable.


1) Create 1 product and 20 repos:

prodNames="CUSTOM__ZOO_"
repoNames="CUSTOM__ZOO_"
products=1
repos=20

for productId in $(seq 1 $products); do
	productName="${prodNames}${productId}"
	hammer product create --name $productName --label $productName --organization-id 1
	for repoId in $(seq 1 $repos); do
		repoName=${repoNames}${productId}-${repoId}
		hammer repository create --product $productName --organization-id 1 --name ${repoName} --content-type=yum --url=https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/ --download-policy on_demand &
	done
done

wait


2) Have a Content Host assigned to Default CV / Library - check that:

subscription-manager refresh; subscription-manager repos --list | grep RedHat_CUSTOM__ZOO | sort | nl

returns all 20 repos

3) *Repeatedly* run below to update repos concurrently & call OrphansCleanup afterwards:

prodNames="CUSTOM__ZOO_"
repoNames="CUSTOM__ZOO_"
products=1
repos=20

for arch in noarch "\"\"" x86_64; do
	echo "$(date): concurrently modifying repos"
	for productId in $(seq 1 $products); do
		productName="${prodNames}${productId}"
		for repoId in $(seq 1 $repos); do
			repoName=${repoNames}${productId}-${repoId}
			hammer repository update --product $productName --organization-id 1 --name ${repoName} --arch $arch &
		done
	done
	wait
	echo "$(date): running orphan cleanup"
	sleep 1
	curl -H "Content-Type:application/json" -X POST -u admin:admin -ks https://localhost:23443/candlepin/jobs/schedule/OrphanCleanupJob
	echo
	sleep 2
done


4) Time to time, check on the Content Host if it still sees all 20 custom repos - it takes random number of iterations (5-10) of step 3 to see a repo is missing.
Comment 2 Vladimír Sedmík 2023-01-04 21:31:04 UTC 
The issue was successfully reproduced in 6.12.0 (candlepin-4.1.15-1.el8sat.noarch) after ~30 runs of 3) from comment#1 - some of the repos disappeared on the content host side, as well as from the cp db:

candlepin=# SELECT product.uuid, product.name, content.uuid, content.name FROM cp2_content content LEFT JOIN (cp2_product_content pc JOIN cp2_products product ON product.uuid = pc.product_uuid) ON pc.content_uuid = content.uuid WHERE content.contenturl LIKE '%ZOO%';
               uuid               |     name      |               uuid               |       name       
----------------------------------+---------------+----------------------------------+------------------
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc6881d7419 | CUSTOM__ZOO_1-11
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc6886d7429 | CUSTOM__ZOO_1-17
                                  |               | 8a818310857d49fb01857dc688af7439 | CUSTOM__ZOO_1-12
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68a0c743a | CUSTOM__ZOO_1-15
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68b17744b | CUSTOM__ZOO_1-6
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68b72745b | CUSTOM__ZOO_1-7
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68bb6746b | CUSTOM__ZOO_1-10
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68bd6747b | CUSTOM__ZOO_1-13
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68c1c748b | CUSTOM__ZOO_1-14
                                  |               | 8a818310857d49fb01857dc68c7f749b | CUSTOM__ZOO_1-16
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68cc2749c | CUSTOM__ZOO_1-5
                                  |               | 8a818310857d49fb01857dc68cfe74ac | CUSTOM__ZOO_1-20
                                  |               | 8a818310857d49fb01857dc68acc744a | CUSTOM__ZOO_1-18
                                  |               | 8a818310857d49fb01857dc68de474ce | CUSTOM__ZOO_1-19
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68e0674cf | CUSTOM__ZOO_1-3
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68e3c74df | CUSTOM__ZOO_1-1
                                  |               | 8a818310857d49fb01857dc68d1c74ad | CUSTOM__ZOO_1-8
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68d5674ae | CUSTOM__ZOO_1-4
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc68dab74be | CUSTOM__ZOO_1-2
 8a818310857d49fb01857dc68e4374e0 | CUSTOM__ZOO_1 | 8a818310857d49fb01857dc3863e6186 | CUSTOM__ZOO_1-9
(20 rows)

On the contrary I was not able to reproduce the issue in 6.12.1 (candlepin-selinux-4.1.19-1.el8sat.noarch) even after 300 runs of the same - content host still shows all the repos, as well as the cp db:
candlepin=# SELECT product.uuid, product.name, content.uuid, content.name FROM cp2_content content LEFT JOIN (cp2_product_content pc JOIN cp2_products product ON product.uuid = pc.product_uuid) ON pc.content_uuid = content.uuid WHERE content.contenturl LIKE '%ZOO%';
               uuid               |     name      |               uuid               |       name       
----------------------------------+---------------+----------------------------------+------------------
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71ab9c26b0 | CUSTOM__ZOO_1-17
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71abdd26dc | CUSTOM__ZOO_1-5
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71b5472708 | CUSTOM__ZOO_1-2
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e719c9a2592 | CUSTOM__ZOO_1-7
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a05925ea | CUSTOM__ZOO_1-8
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a91e266e | CUSTOM__ZOO_1-18
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a2ce2616 | CUSTOM__ZOO_1-15
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e719cb325a8 | CUSTOM__ZOO_1-6
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e719dba25d4 | CUSTOM__ZOO_1-13
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71b9c4271e | CUSTOM__ZOO_1-1
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71b4f426f2 | CUSTOM__ZOO_1-14
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71ab10269a | CUSTOM__ZOO_1-9
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a4612642 | CUSTOM__ZOO_1-11
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e719b43257c | CUSTOM__ZOO_1-12
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a97b2684 | CUSTOM__ZOO_1-20
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a0d22600 | CUSTOM__ZOO_1-3
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a3fd262c | CUSTOM__ZOO_1-4
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e719cf625be | CUSTOM__ZOO_1-10
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71a7f02658 | CUSTOM__ZOO_1-16
 8a8183c3857c8cae01857e71b9c9271f | CUSTOM__ZOO_1 | 8a8183c3857c8cae01857e71abbb26c6 | CUSTOM__ZOO_1-19
(20 rows)

Given this result, I consider this particular reproducer as resolved in 6.12.1, although it may not be the only reproducer of the issue (see https://bugzilla.redhat.com/show_bug.cgi?id=1931027#c112).
Comment 11 Vladimír Sedmík 2023-02-01 13:47:11 UTC 
Re-verified in 6.12.2 snap 1.0 (candlepin-4.1.19-1.el8sat.noarch) - after 300 runs of 3) from comment#1 the content host still shows all the repos, as well as the cp db.
Comment 16 errata-xmlrpc 2023-02-08 14:35:01 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite 6.12.2 Async Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:0668
Note You need to log in before you can comment on or make changes to this bug.