Description of problem: If user performs a manifest refresh against a manifest having all expired subscriptions or an empty manifest, Candlepin will remove all the environment contents in the organization. After renewing all subscriptions and attach to the manifest, user performs a manifest refresh again. The job completed successfully but all clients are not able to subscribe to or enable any Redhat repositories due to no content listing. Steps to Reproduce: 1. Have a valid manifest imported to the Satellite. Enable and sync some Redhat repositories. Create some custom repositories. Create one or more content views and attach some repositories. 2. Go to Web UI -> Content -> Subscriptions -> Manage Manifest -> click the "Subscription Allocation" link to go to the Customer Portal page. 3. In the Customer Portal, delete all subscriptions allocated to the manifest. 4. Then refresh the manifest using the Satellite Web UI. Actual results: # You will see the import job remove all products and contents references like below in candlepin.log file. NOTE: If you don't see the "Removing owner-content references for owner" and "Removing owner-product references for owner", you will need to follow step (4) again. I refreshed the empty manifest twice to reproduce the issue. Not sure why. -------------------------------------------------------------------------- 2022-03-09 14:41:29,987 [thread=http-bio-127.0.0.1-23443-exec-3] [req=67b1d364-ddd4-4c72-a2f6-09291d658b8d, org=, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.common.filter.LoggingFilter - Request: verb=POST, uri=/candlepin/owners/redhat/imports/async 2022-03-09 14:41:30,000 [thread=http-bio-127.0.0.1-23443-exec-3] [req=67b1d364-ddd4-4c72-a2f6-09291d658b8d, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.resource.OwnerResource - Running async import of archive /var/cache/tomcat/temp/pfx11102612137104972728sfx for owner redhat 2022-03-09 14:41:30,036 [thread=http-bio-127.0.0.1-23443-exec-3] [req=67b1d364-ddd4-4c72-a2f6-09291d658b8d, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.async.JobManager - Job queued: AsyncJobStatus [id: 8ac8a52d7f673a55017f6cfb68ad01c0, name: Import Manifest, key: ImportJob, state: QUEUED] 2022-03-09 14:41:30,046 [thread=http-bio-127.0.0.1-23443-exec-3] [req=67b1d364-ddd4-4c72-a2f6-09291d658b8d, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=60 2022-03-09 14:41:30,063 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.async.JobManager - Starting job "Import Manifest" using class: org.candlepin.async.tasks.ImportJob 2022-03-09 14:41:30,101 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.sync.RulesImporter - Importing new rules from manifest, current version: 5.41 new version: 5.41 2022-03-09 14:41:30,129 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.policy.js.JsRunnerProvider - Recompiling rules with timestamp: 2022-03-09 14:41:30.103 2022-03-09 14:41:32,583 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] WARN org.candlepin.sync.Importer - No products found to import, skipping product import. 2022-03-09 14:41:32,585 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] WARN org.candlepin.sync.Importer - No entitlements in manifest, removing all subscriptions for owner. 2022-03-09 14:41:32,589 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.controller.CandlepinPoolManager - Refreshing pools for owner: Owner [id: 8ac8a52d7f1ba631017f1bb6f3ed0001, key: redhat] 2022-03-09 14:41:32,851 [thread=http-bio-127.0.0.1-23443-exec-7] [req=634aef21-933c-4cd6-adee-466ae61108c4, org=, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.common.filter.LoggingFilter - Request: verb=GET, uri=/candlepin/jobs/8ac8a52d7f673a55017f6cfb68ad01c0? 2022-03-09 14:41:32,856 [thread=http-bio-127.0.0.1-23443-exec-7] [req=634aef21-933c-4cd6-adee-466ae61108c4, org=, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=5 2022-03-09 14:41:33,641 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.model.OwnerContentCurator - Removing owner-content references for owner: Owner [id: 8ac8a52d7f1ba631017f1bb6f3ed0001, key: redhat], [8ac8a52d7f1ba631017f1bc30ef50dbf, 8ac8a52d7f1ba631017f1bc30ef50dbb, 8ac8a52d7f1ba631017f1bc30ef50dbc, 8ac8a52d7f1ba631017f1bc30ef50dbd, 8ac8a52d7f1ba631017f1bc30ef50dbe, 8ac8a52d7f1ba631017f1bc30ef50dca, 8ac8a52d7f1ba631017f1bc310653139, 8ac8a52d7f1ba631017f1bc310653138, 8ac8a52d7f1ba631017f1bc310653137, 8ac8a52d7f1ba631017f1bc310653136, 8ac8a52d7f673a55017f675abd700080, 8ac8a52d7f1ba631017f1bc310653135, 8ac8a52d7f1ba631017f1bc310653134, 8ac8a52d7f1ba631017f1bc30e77045f, 8ac8a52d7f1ba631017f1bc31065313b, 8ac8a52d7f1ba631017f1bc31065313a, 8ac8a52d7f1ba631017f1bc30ffc28a1, 8ac8a52d7f1ba631017f1bc30ffc28a0, 8ac8a52d7f1ba631017f1bc31065313f, 8ac8a52d7f1ba631017f1bc31065313e, 8ac8a52d7f1ba631017f1bc31065313d, 8ac8a52d7f1ba631017f1bc31065313c, 8ac8a52d7f1ba631017f1bc30e770465, 8ac8a52d7f1ba631017f1bc30e770466, 8ac8a52d7f1ba631017f1bc30ef50db9, 8ac8a52d7f1ba631017f1bc310653140, 8ac8a52d7f1ba631017f1bc30ef50dc0, 8ac8a52d7f1ba631017f1bc30ef50dc1, 8ac8a52d7f1ba631017f1bc3102d2ce7, 8ac8a52d7f1ba631017f1bc30ef50dc2, 8ac8a52d7f1ba631017f1bc3102d2ce5, 8ac8a52d7f1ba631017f1bc3102d2ce6, 8ac8a52d7f1ba631017f1bc3102d2ce3, 8ac8a52d7f1ba631017f1bc3102d2ce4, 8ac8a52d7f1ba631017f1bc30e770461, 8ac8a52d7f1ba631017f1bc3102d2ce1, 8ac8a52d7f1ba631017f1bc30e770462, 8ac8a52d7f1ba631017f1bc3102d2ce2, 8ac8a52d7f1ba631017f1bc30e770463, 8ac8a52d7f1ba631017f1bc3102d2ce0, 8ac8a52d7f1ba631017f1bc30e770464, 8ac8a52d7f1ba631017f1bc30e770460, 8ac8a52d7f673a55017f675abcd0000a, 8ac8a52d7f1ba631017f1bc30ef50dba, 8ac8a52d7f1ba631017f1bc3102d2cca, 8ac8a52d7f1ba631017f1bc30ef50dc7, 8ac8a52d7f1ba631017f1bc30ef50dc8, 8ac8a52d7f1ba631017f1bc30ef50dc9, 8ac8a52d7f1ba631017f1bc30ef50dc3, 8ac8a52d7f1ba631017f1bc30ef50dc4, 8ac8a52d7f1ba631017f1bc30ef50dc5, 8ac8a52d7f1ba631017f1bc30ef50dc6, 8ac8a52d7f1ba631017f1bc30eb80829, 8ac8a52d7f1ba631017f1bc30eb80827, 8ac8a52d7f1ba631017f1bc30eb80828, 8ac8a52d7f1ba631017f1bc30eb8081a, <snip> 2022-03-09 14:41:35,076 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.model.OwnerProductCurator - Removing owner-product references for owner: Owner [id: 8ac8a52d7f1ba631017f1bb6f3ed0001, key: redhat], [8ac8a52d7f1ba631017f1bc310402ef8, 8ac8a52d7f1ba631017f1bc3108f3782, 8ac8a52d7f1ba631017f1bc30ede0bb3, 8ac8a52d7f1ba631017f1bc30e730431, 8ac8a52d7f1ba631017f1bc3107e34eb, 8ac8a52d7f1ba631017f1bc310a039bd, 8ac8a52d7f1ba631017f1bc30ef80e14, 8ac8a52d7f1ba631017f1bc30eef0d29, 8ac8a52d7f1ba631017f1bc30ee40c42, 8ac8a52d7f1ba631017f1bc30f3a143f, 8ac8a52d7f1ba631017f1bc30ecd0a1a, 8ac8a52d7f1ba631017f1bc3102a2c79, 8ac8a52d7f1ba631017f1bc3108e3742, 8ac8a52d7f1ba631017f1bc310a83af3, 8ac8a52d7f1ba631017f1bc30ed00a67, 8ac8a52d7f1ba631017f1bc30f871be6, 8ac8a52d7f1ba631017f1bc30ff2274c, 8ac8a52d7f1ba631017f1bc3109037ad, 8ac8a52d7f1ba631017f1bc310a83ae5, 8ac8a52d7f1ba631017f1bc310172ab9, 8ac8a52d7f1ba631017f1bc30eb20783, 8ac8a52d7f1ba631017f1bc8da383c8c, 8ac8a52d7f1ba631017f1bc3109037b1, 8ac8a52d7f1ba631017f1bc3100629e9, 8ac8a52d7f1ba631017f1bc30f8f1caf, 8ac8a52d7f1ba631017f1bc310a73ac5, 8ac8a52d7f1ba631017f1bc3102a2c8c, 8ac8a52d7f1ba631017f1bc3108b36bf, 8ac8a52d7f1ba631017f1bc3100b2a44, 8ac8a52d7f1ba631017f1bc30f8f1cc9, 8ac8a52d7f1ba631017f1bc310ac3b65, 8ac8a52d7f1ba631017f1bc30f10100b, 8ac8a52d7f1ba631017f1bc30ef20d82, 8ac8a52d7f1ba631017f1bc310ad3b8e, 8ac8a52d7f1ba631017f1bc30f8f1cc8, 8ac8a52d7f1ba631017f1bc30f4c1577, 8ac8a52d7f1ba631017f1bc30e9b0650, 8ac8a52d7f1ba631017f1bc30ec50964, 8ac8a52d7f1ba631017f1bc30f8d1c94, 8ac8a52d7f1ba631017f1bc30eb507f6, 8ac8a52d7f1ba631017f1bc30f281251, 8ac8a52d7f1ba631017f1bc310a73ad5, 8ac8a52d7f1ba631017f1bc8da373c60, <snip> 2022-03-09 14:41:35,128 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.model.OwnerProductCurator - 293 owner-product relations removed 2022-03-09 14:41:35,175 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.model.OwnerProductCurator - 0 activation key product(s) removed 2022-03-09 14:41:35,454 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.controller.CandlepinPoolManager - Refresh pools for owner: redhat completed in: 2865ms 2022-03-09 14:41:35,615 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.controller.ManifestManager - Deleting stored manifest file: 8ac8a52d7f673a55017f6cfb689101bf 2022-03-09 14:41:35,714 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.async.tasks.ImportJob - Async import complete. 2022-03-09 14:41:35,794 [thread=Thread-75 (ActiveMQ-client-global-threads)] [job=8ac8a52d7f673a55017f6cfb68ad01c0, job_key=ImportJob, org=redhat, csid=90346f37-caa9-49d6-8cff-7a9ff49f85c3] INFO org.candlepin.async.JobManager - Job "Import Manifest" completed in 5673ms -------------------------------------------------------------------------- # Before refreshing the manifest: su - postgres -c "psql candlepin -c \"select e.name, c.contenturl, c.vendor, c.label from cp2_environment_content ec left join cp_environment e on e.id = ec.environment_id left join cp2_content c on c.uuid = content_uuid order by e.name, c.vendor, c.label;\"" name | contenturl | vendor | label ---------------------------------+-----------------------------------------------------------------+---------+----------------------------------------- Library | /custom/hao_product/no-feed-repo | Custom | redhat_hao_product_no-feed-repo Library | /content/dist/rhel/server/7/$releasever/$basearch/os | Red Hat | rhel-7-server-rpms Library | /content/dist/rhel/server/7/7Server/$basearch/sat-tools/6.10/os | Red Hat | rhel-7-server-satellite-tools-6.10-rpms Library/cv-local-repos | /custom/hao_product/no-feed-repo | Custom | redhat_hao_product_no-feed-repo Library/sat-tools-only-imported | /content/dist/rhel/server/7/7Server/$basearch/sat-tools/6.10/os | Red Hat | rhel-7-server-satellite-tools-6.10-rpms # After refreshing the manifest. Only the custom environment contents are left: candlepin=# select e.name, c.contenturl, c.vendor, c.label from cp2_environment_content ec left join cp_environment e on e.id = ec.environment_id left join cp2_content c on c.uuid = content_uuid; name | contenturl | vendor | label ------------------------+----------------------------------+--------+--------------------------------- Library | /custom/hao_product/no-feed-repo | Custom | redhat_hao_product_no-feed-repo Library/cv-local-repos | /custom/hao_product/no-feed-repo | Custom | redhat_hao_product_no-feed-repo Adding the subscriptions back and refresh the manifest doesn't solve the issue. Expected results: Anything that we can improve the avoid this issue.
The workaround of this issue is: Steps: 1) Complete resync all Redhat repositories. 2) Publish new version and promote all content views. 3) Publish new version and promote all composite content views if any.
Steps to Retest: 1. Have a valid manifest imported to the Satellite. Enable and sync some Redhat repositories. Create some custom repositories. Create one or more content views and attach some repositories. 2. Go to Web UI -> Content -> Subscriptions -> Manage Manifest -> click the "Subscription Allocation" link to go to the Customer Portal page. 3. In the Customer Portal, delete all subscriptions allocated to the manifest. 4. Then refresh the manifest using the Satellite Web UI. 5. Run grep for "Removing owner-content references for owner" on /var/log/candlepin/candlepin.log 6. Run su - postgres -c "psql candlepin -c \"select e.name, c.contenturl, c.vendor, c.label from cp2_environment_content ec left join cp_environment e on e.id = ec.environment_id left join cp2_content c on c.uuid = content_uuid order by e.name, c.vendor, c.label;\"" Expected result: 5. Should return empty 6. Should return only the number of repos that was synced and added to cv Actual: 5. Did return empty 6. Did return only the number of repos that was synced and added to cv I also did manifest refresh a few times to ensure the error didn't occur anymore. Everything seems to check out ok. Verified on 6.12 snap 4
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Satellite 6.12 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:8506