Red Hat Bugzilla – Bug 1161643
Content View Publish takes longer than anticipated
Last modified: 2017-08-21 06:35:59 EDT
Description of problem: A user reported an issue with publishing a content view taking ~13 minutes to complete. This user had about 4 repos with ~25,000 packages total, and ~50 puppet modules. The user had 4 CPUs and 16 GB ram Version-Release number of selected component (if applicable): 6.0.4 How reproducible: Always Steps to Reproduce: 1. Create a content view and add roughly what is above. 2. Make sure you are using a package include filter with 'original packages' checked (not sure if this makes a difference). 3. Publish the content view Actual results: Takes ~13 minutes Expected results: Should take less, given that rhel 6Server + a kickstart tree takes ~3-5 minutes, i'd expect this to take ~5-7 minutes, need to identify the cause Additional info: Dynflow actions output: Status: stopped Result: success Started at: 2014-11-07 12:01:51 UTC Ended at: 2014-11-07 12:14:10 UTC sequence concurrence 6: Actions::Pulp::Repository::CreateInPlan (success) [ 0.07s / 0.07s ] 9: Actions::Pulp::Repository::CopyRpm (success) [ 8.00s / 4.45s ] 11: Actions::Pulp::Repository::CopyErrata (success) [ 116.00s / 9.95s ] 13: Actions::Pulp::Repository::CopyPackageGroup (success) [ 0.73s / 0.73s ] 15: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.30s / 0.79s ] 17: Actions::Pulp::Repository::CopyDistribution (success) [ 1.21s / 0.71s ] 19: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 1.43s / 1.43s ] 21: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 1.36s / 1.36s ] 24: Actions::Pulp::Repository::DistributorPublish (success) [ 2.90s / 1.46s ] 26: Actions::ElasticSearch::Repository::IndexContent (success) [ 3.10s / 3.10s ] 30: Actions::Pulp::Repository::RemoveRpm (success) [ 5.59s / 1.86s ] 32: Actions::Pulp::Repository::RemoveErrata (success) [ 5.73s / 1.64s ] 34: Actions::Pulp::Repository::RemovePackageGroup (success) [ 5.20s / 1.43s ] 36: Actions::Pulp::Repository::RemoveDistribution (success) [ 5.72s / 1.98s ] 38: Actions::Pulp::Repository::RemovePuppetModule (success) [ 5.72s / 1.72s ] 41: Actions::Pulp::Repository::CopyRpm (success) [ 2.87s / 1.34s ] 43: Actions::Pulp::Repository::CopyErrata (success) [ 0.89s / 0.89s ] 45: Actions::Pulp::Repository::CopyPackageGroup (success) [ 0.80s / 0.80s ] 47: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.41s / 1.41s ] 49: Actions::Pulp::Repository::CopyDistribution (success) [ 0.83s / 0.83s ] 52: Actions::Pulp::Repository::DistributorPublish (success) [ 1.80s / 1.80s ] 54: Actions::ElasticSearch::Repository::IndexContent (success) [ 4.57s / 4.57s ] 56: Actions::Pulp::Repository::RegenerateApplicability (success) [ 7.81s / 3.15s ] 60: Actions::Pulp::Repository::CreateInPlan (success) [ 0.07s / 0.07s ] 63: Actions::Pulp::Repository::CopyRpm (success) [ 52.16s / 14.01s ] 65: Actions::Pulp::Repository::CopyErrata (success) [ 4.00s / 1.07s ] 67: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.24s / 0.74s ] 69: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.66s / 0.99s ] 71: Actions::Pulp::Repository::CopyDistribution (success) [ 1.25s / 1.25s ] 73: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 7.52s / 7.52s ] 75: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 7.63s / 7.12s ] 78: Actions::Pulp::Repository::DistributorPublish (success) [ 56.84s / 3.13s ] 80: Actions::ElasticSearch::Repository::IndexContent (success) [ 24.64s / 24.64s ] 84: Actions::Pulp::Repository::RemoveRpm (success) [ 18.64s / 6.25s ] 86: Actions::Pulp::Repository::RemoveErrata (success) [ 14.65s / 2.52s ] 88: Actions::Pulp::Repository::RemovePackageGroup (success) [ 14.52s / 2.10s ] 90: Actions::Pulp::Repository::RemoveDistribution (success) [ 14.50s / 2.28s ] 92: Actions::Pulp::Repository::RemovePuppetModule (success) [ 13.67s / 2.36s ] 95: Actions::Pulp::Repository::CopyRpm (success) [ 38.21s / 4.19s ] 97: Actions::Pulp::Repository::CopyErrata (success) [ 1.65s / 1.13s ] 99: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.53s / 0.90s ] 101: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.69s / 0.69s ] 103: Actions::Pulp::Repository::CopyDistribution (success) [ 0.81s / 0.81s ] 106: Actions::Pulp::Repository::DistributorPublish (success) [ 2.47s / 0.95s ] 108: Actions::ElasticSearch::Repository::IndexContent (success) [ 13.81s / 13.81s ] 110: Actions::Pulp::Repository::RegenerateApplicability (success) [ 17.30s / 1.63s ] 114: Actions::Pulp::Repository::CreateInPlan (success) [ 0.07s / 0.07s ] 117: Actions::Pulp::Repository::CopyRpm (success) [ 7.92s / 5.10s ] 119: Actions::Pulp::Repository::CopyErrata (success) [ 8.02s / 5.45s ] 121: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.97s / 1.97s ] 123: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.94s / 1.94s ] 125: Actions::Pulp::Repository::CopyDistribution (success) [ 1.82s / 1.82s ] 127: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 5.03s / 5.03s ] 129: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 4.13s / 2.06s ] 132: Actions::Pulp::Repository::DistributorPublish (success) [ 4.80s / 2.06s ] 134: Actions::ElasticSearch::Repository::IndexContent (success) [ 3.93s / 3.93s ] 138: Actions::Pulp::Repository::RemoveRpm (success) [ 20.53s / 3.99s ] 140: Actions::Pulp::Repository::RemoveErrata (success) [ 17.68s / 3.07s ] 142: Actions::Pulp::Repository::RemovePackageGroup (success) [ 20.57s / 4.48s ] 144: Actions::Pulp::Repository::RemoveDistribution (success) [ 19.42s / 3.41s ] 146: Actions::Pulp::Repository::RemovePuppetModule (success) [ 19.67s / 2.60s ] 149: Actions::Pulp::Repository::CopyRpm (success) [ 5.88s / 2.32s ] 151: Actions::Pulp::Repository::CopyErrata (success) [ 1.35s / 0.84s ] 153: Actions::Pulp::Repository::CopyPackageGroup (success) [ 0.81s / 0.81s ] 155: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 6.17s / 1.02s ] 157: Actions::Pulp::Repository::CopyDistribution (success) [ 1.47s / 0.97s ] 160: Actions::Pulp::Repository::DistributorPublish (success) [ 48.65s / 2.54s ] 162: Actions::ElasticSearch::Repository::IndexContent (success) [ 3.27s / 3.27s ] 164: Actions::Pulp::Repository::RegenerateApplicability (success) [ 8.88s / 1.94s ] 168: Actions::Pulp::Repository::CreateInPlan (success) [ 0.05s / 0.05s ] 171: Actions::Pulp::Repository::CopyRpm (success) [ 135.65s / 20.77s ] 173: Actions::Pulp::Repository::CopyErrata (success) [ 1.98s / 0.88s ] 175: Actions::Pulp::Repository::CopyPackageGroup (success) [ 5.26s / 1.23s ] 177: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 2.98s / 1.01s ] 179: Actions::Pulp::Repository::CopyDistribution (success) [ 0.74s / 0.74s ] 181: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 18.14s / 17.57s ] 183: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 12.52s / 11.33s ] 186: Actions::Pulp::Repository::DistributorPublish (success) [ 162.03s / 3.46s ] 188: Actions::ElasticSearch::Repository::IndexContent (success) [ 35.27s / 35.27s ] 192: Actions::Pulp::Repository::RemoveRpm (success) [ 36.17s / 6.46s ] 194: Actions::Pulp::Repository::RemoveErrata (success) [ 1.67s / 1.15s ] 196: Actions::Pulp::Repository::RemovePackageGroup (success) [ 1.70s / 1.19s ] 198: Actions::Pulp::Repository::RemoveDistribution (success) [ 32.86s / 3.19s ] 200: Actions::Pulp::Repository::RemovePuppetModule (success) [ 1.65s / 1.14s ] 203: Actions::Pulp::Repository::CopyRpm (success) [ 82.79s / 5.11s ] 205: Actions::Pulp::Repository::CopyErrata (success) [ 1.95s / 0.94s ] 207: Actions::Pulp::Repository::CopyPackageGroup (success) [ 4.77s / 1.23s ] 209: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.34s / 0.83s ] 211: Actions::Pulp::Repository::CopyDistribution (success) [ 0.76s / 0.76s ] 214: Actions::Pulp::Repository::DistributorPublish (success) [ 9.96s / 2.37s ] 216: Actions::ElasticSearch::Repository::IndexContent (success) [ 31.41s / 31.41s ] 218: Actions::Pulp::Repository::RegenerateApplicability (success) [ 27.60s / 1.98s ] 222: Actions::Pulp::Repository::CreateInPlan (success) [ 0.07s / 0.07s ] 225: Actions::Pulp::Repository::CopyRpm (success) [ 26.05s / 10.23s ] 227: Actions::Pulp::Repository::CopyErrata (success) [ 1.39s / 1.39s ] 229: Actions::Pulp::Repository::CopyPackageGroup (success) [ 8.76s / 2.27s ] 231: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.11s / 1.11s ] 233: Actions::Pulp::Repository::CopyDistribution (success) [ 3.52s / 1.42s ] 235: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 5.44s / 5.44s ] 237: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 5.16s / 5.16s ] 240: Actions::Pulp::Repository::DistributorPublish (success) [ 103.26s / 5.33s ] 242: Actions::ElasticSearch::Repository::IndexContent (success) [ 13.62s / 13.62s ] 246: Actions::Pulp::Repository::RemoveRpm (success) [ 9.43s / 2.95s ] 248: Actions::Pulp::Repository::RemoveErrata (success) [ 8.48s / 2.12s ] 250: Actions::Pulp::Repository::RemovePackageGroup (success) [ 8.56s / 2.36s ] 252: Actions::Pulp::Repository::RemoveDistribution (success) [ 8.29s / 2.56s ] 254: Actions::Pulp::Repository::RemovePuppetModule (success) [ 8.12s / 2.56s ] 257: Actions::Pulp::Repository::CopyRpm (success) [ 20.51s / 2.93s ] 259: Actions::Pulp::Repository::CopyErrata (success) [ 0.73s / 0.73s ] 261: Actions::Pulp::Repository::CopyPackageGroup (success) [ 3.66s / 1.13s ] 263: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.28s / 0.78s ] 265: Actions::Pulp::Repository::CopyDistribution (success) [ 0.71s / 0.71s ] 268: Actions::Pulp::Repository::DistributorPublish (success) [ 4.12s / 1.34s ] 270: Actions::ElasticSearch::Repository::IndexContent (success) [ 7.75s / 7.75s ] 272: Actions::Pulp::Repository::RegenerateApplicability (success) [ 0.66s / 0.66s ] 276: Actions::Pulp::Repository::CreateInPlan (success) [ 0.03s / 0.03s ] 279: Actions::Pulp::Repository::CopyRpm (success) [ 3.35s / 3.35s ] 281: Actions::Pulp::Repository::CopyErrata (success) [ 9.64s / 5.24s ] 283: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.84s / 1.84s ] 285: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 2.03s / 2.03s ] 287: Actions::Pulp::Repository::CopyDistribution (success) [ 1.91s / 1.91s ] 289: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 6.71s / 6.71s ] 291: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 2.29s / 2.29s ] 294: Actions::Pulp::Repository::DistributorPublish (success) [ 3.25s / 2.12s ] 296: Actions::ElasticSearch::Repository::IndexContent (success) [ 2.40s / 2.40s ] 300: Actions::Pulp::Repository::RemoveRpm (success) [ 3.43s / 1.37s ] 302: Actions::Pulp::Repository::RemoveErrata (success) [ 3.40s / 1.34s ] 304: Actions::Pulp::Repository::RemovePackageGroup (success) [ 3.11s / 1.24s ] 306: Actions::Pulp::Repository::RemoveDistribution (success) [ 5.35s / 2.72s ] 308: Actions::Pulp::Repository::RemovePuppetModule (success) [ 4.36s / 2.28s ] 311: Actions::Pulp::Repository::CopyRpm (success) [ 1.52s / 1.00s ] 313: Actions::Pulp::Repository::CopyErrata (success) [ 2.56s / 1.10s ] 315: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.40s / 0.89s ] 317: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.55s / 1.00s ] 319: Actions::Pulp::Repository::CopyDistribution (success) [ 0.85s / 0.85s ] 322: Actions::Pulp::Repository::DistributorPublish (success) [ 2.11s / 1.10s ] 324: Actions::ElasticSearch::Repository::IndexContent (success) [ 1.86s / 1.86s ] 326: Actions::Pulp::Repository::RegenerateApplicability (success) [ 10.30s / 2.83s ] 330: Actions::Pulp::Repository::CreateInPlan (success) [ 0.04s / 0.04s ] 333: Actions::Pulp::Repository::CopyRpm (success) [ 12.31s / 6.12s ] 335: Actions::Pulp::Repository::CopyErrata (success) [ 1.29s / 1.29s ] 337: Actions::Pulp::Repository::CopyPackageGroup (success) [ 2.11s / 2.11s ] 339: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.80s / 1.80s ] 341: Actions::Pulp::Repository::CopyDistribution (success) [ 1.38s / 1.38s ] 343: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 10.89s / 10.89s ] 345: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 2.44s / 2.44s ] 348: Actions::Pulp::Repository::DistributorPublish (success) [ 22.46s / 4.58s ] 350: Actions::ElasticSearch::Repository::IndexContent (success) [ 5.12s / 5.12s ] 354: Actions::Pulp::Repository::RemoveRpm (success) [ 5.75s / 2.79s ] 356: Actions::Pulp::Repository::RemoveErrata (success) [ 1.83s / 1.22s ] 358: Actions::Pulp::Repository::RemovePackageGroup (success) [ 2.91s / 1.42s ] 360: Actions::Pulp::Repository::RemoveDistribution (success) [ 2.72s / 1.92s ] 362: Actions::Pulp::Repository::RemovePuppetModule (success) [ 4.77s / 2.00s ] 365: Actions::Pulp::Repository::CopyRpm (success) [ 6.73s / 1.37s ] 367: Actions::Pulp::Repository::CopyErrata (success) [ 1.35s / 0.84s ] 369: Actions::Pulp::Repository::CopyPackageGroup (success) [ 0.80s / 0.80s ] 371: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.37s / 0.86s ] 373: Actions::Pulp::Repository::CopyDistribution (success) [ 0.74s / 0.74s ] 376: Actions::Pulp::Repository::DistributorPublish (success) [ 2.02s / 0.94s ] 378: Actions::ElasticSearch::Repository::IndexContent (success) [ 2.58s / 2.58s ] 380: Actions::Pulp::Repository::RegenerateApplicability (success) [ 9.00s / 1.45s ] 384: Actions::Pulp::Repository::Create (success) [ 2.24s / 2.24s ] 387: Actions::Pulp::Repository::CopyPuppetModule (success) [ 20.86s / 7.25s ] 391: Actions::Pulp::Repository::RemovePuppetModule (success) [ 1.02s / 1.02s ] 393: Actions::Pulp::Repository::CopyPuppetModule (success) [ 4.11s / 1.60s ] 396: Actions::Pulp::Repository::DistributorPublish (success) [ 3.71s / 1.79s ] 398: Actions::ElasticSearch::ContentViewPuppetEnvironment::IndexContent (success) [ 2.66s / 2.66s ] 401: Actions::Pulp::Repository::DistributorPublish (success) [ 3.04s / 1.39s ] 404: Actions::Candlepin::Environment::SetContent (success) [ 0.15s / 0.15s ] 408: Actions::Katello::ContentView::Publish (success) [ 0.82s / 0.82s ]
Assigned RedHat Repos: INFO: - Assigning repo 'Red Hat Enterprise Linux Server/Red Hat Enterprise Linux 6 Server Kickstart x86_64 6.5' INFO: - Assigning repo 'Red Hat Enterprise Linux Server - Extended Update Support/Red Hat Enterprise Linux 6 Server - Extended Update Support RPMs x86_64 6.5' INFO: - Assigning repo 'Red Hat Enterprise Linux Server - Extended Update Support/Red Hat Enterprise Linux 6 Server - Extended Update Support - Optional RPMs x86_64 6.5' INFO: - Assigning repo 'Red Hat Enterprise Linux Server/Red Hat Enterprise Linux 6 Server - RH Common RPMs x86_64 6.5' INFO: - Assigning repo 'Red Hat Software Collections for RHEL Server/Red Hat Software Collections RPMs for Red Hat Enterprise Linux 6 RHEL 6 Server EUS x86_64 6.5' INFO: - Assigning repo 'Oracle Java for RHEL Server - Extended Update Support/Red Hat Enterprise Linux 6 Server - Extended Update Support - Oracle Java RPMs x86_64 6.5'
This bug has been fixed in Katello per http://projects.theforeman.org/issues/8563; Needs ported to Satellite 6
The situation with repository syncs improved with Sat6.1.6 and Sat6.1.7. The IO reduced a lot. But the content view publishing did not really improve as much as expected compared to Sat6.1.5. Below are the timings using Sat6.1.7 for a composite content view with 7 repos and 33.000 RPMs it still takes 25 minutes, with Sat6.1.5 it was 28 minutes. Most of the time is spend in the PurgeEmptyErrata and PurgeEmptyPackageGroups. Regards, Peter Status: stopped Result: success Started at: 2016-03-06 20:11:19 UTC Ended at: 2016-03-06 20:34:34 UTC Plan Run Finalize sequence concurrence 6: Actions::Pulp::Repository::Create (success) [ 0.67s / 0.67s ] 7: Actions::Katello::Repository::Create (success) [ 1.23s / 1.23s ] 12: Actions::Pulp::Repository::CopyRpm (success) [ 39.95s / 6.76s ] 14: Actions::Pulp::Repository::CopyErrata (success) [ 1.18s / 0.60s ] 16: Actions::Pulp::Repository::CopyPackageGroup (success) [ 2.43s / 0.80s ] 18: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 3.61s / 0.45s ] 20: Actions::Pulp::Repository::CopyDistribution (success) [ 1.63s / 0.38s ] 22: Actions::Katello::Repository::IndexErrata (success) [ 0.22s / 0.22s ] 24: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 92.53s / 92.00s ] 26: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 6.39s / 5.89s ] 29: Actions::Pulp::Repository::DistributorPublish (success) [ 3.63s / 1.10s ] 31: Actions::ElasticSearch::Repository::IndexContent (success) [ 6.78s / 6.78s ] 35: Actions::Pulp::Repository::RemoveRpm (success) [ 7.70s / 2.08s ] 37: Actions::Pulp::Repository::RemoveErrata (success) [ 1.10s / 0.59s ] 39: Actions::Pulp::Repository::RemovePackageGroup (success) [ 1.37s / 0.83s ] 41: Actions::Pulp::Repository::RemoveDistribution (success) [ 1.93s / 0.89s ] 43: Actions::Pulp::Repository::RemovePuppetModule (success) [ 7.56s / 1.94s ] 46: Actions::Pulp::Repository::CopyRpm (success) [ 11.12s / 1.54s ] 48: Actions::Pulp::Repository::CopyErrata (success) [ 0.84s / 0.33s ] 50: Actions::Pulp::Repository::CopyPackageGroup (success) [ 0.84s / 0.34s ] 52: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.85s / 0.34s ] 54: Actions::Pulp::Repository::CopyDistribution (success) [ 0.81s / 0.30s ] 57: Actions::Pulp::Repository::DistributorPublish (success) [ 1.60s / 0.59s ] 59: Actions::ElasticSearch::Repository::IndexContent (success) [ 4.31s / 4.31s ] 63: Actions::Pulp::Repository::Create (success) [ 0.95s / 0.95s ] 64: Actions::Katello::Repository::Create (success) [ 1.69s / 1.69s ] 69: Actions::Pulp::Repository::CopyRpm (success) [ 20.89s / 4.69s ] 71: Actions::Pulp::Repository::CopyErrata (success) [ 13.67s / 3.19s ] 73: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.89s / 0.85s ] 75: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.02s / 0.51s ] 77: Actions::Pulp::Repository::CopyDistribution (success) [ 2.18s / 0.86s ] 79: Actions::Katello::Repository::IndexErrata (success) [ 0.31s / 0.31s ] 81: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 38.14s / 33.11s ] 83: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 6.42s / 3.82s ] 86: Actions::Pulp::Repository::DistributorPublish (success) [ 6.91s / 0.93s ] 88: Actions::ElasticSearch::Repository::IndexContent (success) [ 8.98s / 8.98s ] 92: Actions::Pulp::Repository::RemoveRpm (success) [ 6.89s / 0.69s ] 94: Actions::Pulp::Repository::RemoveErrata (success) [ 6.58s / 0.73s ] 96: Actions::Pulp::Repository::RemovePackageGroup (success) [ 6.67s / 0.68s ] 98: Actions::Pulp::Repository::RemoveDistribution (success) [ 7.20s / 1.06s ] 100: Actions::Pulp::Repository::RemovePuppetModule (success) [ 6.15s / 0.95s ] 103: Actions::Pulp::Repository::CopyRpm (success) [ 9.39s / 1.83s ] 105: Actions::Pulp::Repository::CopyErrata (success) [ 1.66s / 0.54s ] 107: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.59s / 0.52s ] 109: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.28s / 0.45s ] 111: Actions::Pulp::Repository::CopyDistribution (success) [ 2.53s / 0.75s ] 114: Actions::Pulp::Repository::DistributorPublish (success) [ 3.35s / 2.00s ] 116: Actions::ElasticSearch::Repository::IndexContent (success) [ 4.57s / 4.57s ] 120: Actions::Pulp::Repository::Create (success) [ 0.97s / 0.97s ] 121: Actions::Katello::Repository::Create (success) [ 1.76s / 1.76s ] 126: Actions::Pulp::Repository::CopyRpm (success) [ 4.46s / 1.70s ] 128: Actions::Pulp::Repository::CopyErrata (success) [ 26.40s / 5.36s ] 130: Actions::Pulp::Repository::CopyPackageGroup (success) [ 4.12s / 1.40s ] 132: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.09s / 0.56s ] 134: Actions::Pulp::Repository::CopyDistribution (success) [ 1.31s / 0.78s ] 136: Actions::Katello::Repository::IndexErrata (success) [ 0.42s / 0.42s ] 138: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 2.36s / 1.06s ] 140: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 2.25s / 0.78s ] 143: Actions::Pulp::Repository::DistributorPublish (success) [ 3.19s / 1.13s ] 145: Actions::ElasticSearch::Repository::IndexContent (success) [ 1.85s / 1.85s ] 149: Actions::Pulp::Repository::Create (success) [ 1.47s / 1.47s ] 150: Actions::Katello::Repository::Create (success) [ 1.34s / 1.34s ] 155: Actions::Pulp::Repository::CopyRpm (success) [ 11.01s / 0.75s ] 157: Actions::Pulp::Repository::CopyErrata (success) [ 4.91s / 0.34s ] 159: Actions::Pulp::Repository::CopyPackageGroup (success) [ 4.30s / 0.52s ] 161: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 5.97s / 0.98s ] 163: Actions::Pulp::Repository::CopyDistribution (success) [ 2.42s / 0.40s ] 166: Actions::Pulp::Repository::DistributorPublish (success) [ 3.94s / 0.74s ] 168: Actions::ElasticSearch::Repository::IndexContent (success) [ 1.46s / 1.46s ] 172: Actions::Pulp::Repository::Create (success) [ 0.90s / 0.90s ] 173: Actions::Katello::Repository::Create (success) [ 1.96s / 1.96s ] 178: Actions::Pulp::Repository::CopyRpm (success) [ 277.95s / 11.37s ] 180: Actions::Pulp::Repository::CopyErrata (success) [ 2.72s / 0.69s ] 182: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.21s / 0.71s ] 184: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.92s / 0.41s ] 186: Actions::Pulp::Repository::CopyDistribution (success) [ 0.91s / 0.40s ] 188: Actions::Katello::Repository::IndexErrata (success) [ 0.34s / 0.34s ] 190: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 324.43s / 324.43s ] 192: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 202.25s / 201.74s ] 195: Actions::Pulp::Repository::DistributorPublish (success) [ 4.01s / 1.48s ] 197: Actions::ElasticSearch::Repository::IndexContent (success) [ 37.81s / 37.81s ] 201: Actions::Pulp::Repository::RemoveRpm (success) [ 65.25s / 11.46s ] 203: Actions::Pulp::Repository::RemoveErrata (success) [ 60.79s / 6.88s ] 205: Actions::Pulp::Repository::RemovePackageGroup (success) [ 60.45s / 6.61s ] 207: Actions::Pulp::Repository::RemoveDistribution (success) [ 1.00s / 1.00s ] 209: Actions::Pulp::Repository::RemovePuppetModule (success) [ 59.58s / 5.49s ] 212: Actions::Pulp::Repository::CopyRpm (success) [ 85.32s / 7.61s ] 214: Actions::Pulp::Repository::CopyErrata (success) [ 1.39s / 0.38s ] 216: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.03s / 0.34s ] 218: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.96s / 0.38s ] 220: Actions::Pulp::Repository::CopyDistribution (success) [ 0.99s / 0.38s ] 223: Actions::Pulp::Repository::DistributorPublish (success) [ 6.76s / 1.14s ] 225: Actions::ElasticSearch::Repository::IndexContent (success) [ 29.47s / 29.47s ] 229: Actions::Pulp::Repository::Create (success) [ 1.02s / 1.02s ] 230: Actions::Katello::Repository::Create (success) [ 2.05s / 2.05s ] 235: Actions::Pulp::Repository::CopyRpm (success) [ 38.22s / 5.99s ] 237: Actions::Pulp::Repository::CopyErrata (success) [ 1.49s / 0.78s ] 239: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.45s / 0.67s ] 241: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 2.14s / 0.49s ] 243: Actions::Pulp::Repository::CopyDistribution (success) [ 4.03s / 0.74s ] 245: Actions::Katello::Repository::IndexErrata (success) [ 0.42s / 0.42s ] 247: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 50.38s / 47.70s ] 249: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 10.38s / 9.37s ] 252: Actions::Pulp::Repository::DistributorPublish (success) [ 1.46s / 0.84s ] 254: Actions::ElasticSearch::Repository::IndexContent (success) [ 9.83s / 9.83s ] 258: Actions::Pulp::Repository::RemoveRpm (success) [ 13.98s / 2.60s ] 260: Actions::Pulp::Repository::RemoveErrata (success) [ 15.13s / 2.73s ] 262: Actions::Pulp::Repository::RemovePackageGroup (success) [ 14.94s / 2.65s ] 264: Actions::Pulp::Repository::RemoveDistribution (success) [ 15.75s / 3.92s ] 266: Actions::Pulp::Repository::RemovePuppetModule (success) [ 13.88s / 2.05s ] 269: Actions::Pulp::Repository::CopyRpm (success) [ 13.76s / 2.07s ] 271: Actions::Pulp::Repository::CopyErrata (success) [ 2.62s / 1.10s ] 273: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.05s / 0.54s ] 275: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.29s / 0.63s ] 277: Actions::Pulp::Repository::CopyDistribution (success) [ 0.42s / 0.42s ] 280: Actions::Pulp::Repository::DistributorPublish (success) [ 0.96s / 0.45s ] 282: Actions::ElasticSearch::Repository::IndexContent (success) [ 6.38s / 6.38s ] 286: Actions::Pulp::Repository::Create (success) [ 0.50s / 0.50s ] 287: Actions::Katello::Repository::Create (success) [ 1.33s / 1.33s ] 292: Actions::Pulp::Repository::CopyRpm (success) [ 358.25s / 14.29s ] 294: Actions::Pulp::Repository::CopyErrata (success) [ 3.97s / 2.44s ] 296: Actions::Pulp::Repository::CopyPackageGroup (success) [ 8.80s / 2.14s ] 298: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.37s / 0.87s ] 300: Actions::Pulp::Repository::CopyDistribution (success) [ 0.81s / 0.81s ] 302: Actions::Katello::Repository::IndexErrata (success) [ 0.74s / 0.74s ] 304: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 435.58s / 435.07s ] 306: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 178.88s / 178.37s ] 309: Actions::Pulp::Repository::DistributorPublish (success) [ 5.55s / 1.00s ] 311: Actions::ElasticSearch::Repository::IndexContent (success) [ 58.30s / 58.30s ] 315: Actions::Pulp::Repository::RemoveRpm (success) [ 86.18s / 8.39s ] 317: Actions::Pulp::Repository::RemoveErrata (success) [ 6.19s / 1.62s ] 319: Actions::Pulp::Repository::RemovePackageGroup (success) [ 4.66s / 1.09s ] 321: Actions::Pulp::Repository::RemoveDistribution (success) [ 5.18s / 1.57s ] 323: Actions::Pulp::Repository::RemovePuppetModule (success) [ 5.04s / 1.47s ] 326: Actions::Pulp::Repository::CopyRpm (success) [ 99.66s / 5.94s ] 328: Actions::Pulp::Repository::CopyErrata (success) [ 1.99s / 0.47s ] 330: Actions::Pulp::Repository::CopyPackageGroup (success) [ 5.71s / 1.14s ] 332: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.77s / 0.27s ] 334: Actions::Pulp::Repository::CopyDistribution (success) [ 0.78s / 0.28s ] 337: Actions::Pulp::Repository::DistributorPublish (success) [ 7.72s / 1.16s ] 339: Actions::ElasticSearch::Repository::IndexContent (success) [ 42.09s / 42.09s ] 343: Actions::Pulp::Repository::Create (success) [ 0.41s / 0.41s ] 344: Actions::Katello::Repository::Create (success) [ 2.21s / 2.21s ] 349: Actions::Pulp::Repository::CopyRpm (success) [ 21.37s / 5.17s ] 351: Actions::Pulp::Repository::CopyErrata (success) [ 12.80s / 2.34s ] 353: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.06s / 0.55s ] 355: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.67s / 0.60s ] 357: Actions::Pulp::Repository::CopyDistribution (success) [ 1.58s / 0.61s ] 359: Actions::Katello::Repository::IndexErrata (success) [ 0.57s / 0.57s ] 361: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 25.41s / 16.32s ] 363: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 6.11s / 3.96s ] 366: Actions::Pulp::Repository::DistributorPublish (success) [ 3.07s / 0.64s ] 368: Actions::ElasticSearch::Repository::IndexContent (success) [ 5.62s / 5.62s ] 372: Actions::Pulp::Repository::RemoveRpm (success) [ 9.90s / 1.53s ] 374: Actions::Pulp::Repository::RemoveErrata (success) [ 12.26s / 1.81s ] 376: Actions::Pulp::Repository::RemovePackageGroup (success) [ 11.64s / 1.96s ] 378: Actions::Pulp::Repository::RemoveDistribution (success) [ 11.35s / 1.88s ] 380: Actions::Pulp::Repository::RemovePuppetModule (success) [ 10.10s / 1.46s ] 383: Actions::Pulp::Repository::CopyRpm (success) [ 7.67s / 1.45s ] 385: Actions::Pulp::Repository::CopyErrata (success) [ 2.58s / 0.62s ] 387: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.24s / 0.51s ] 389: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 2.45s / 1.08s ] 391: Actions::Pulp::Repository::CopyDistribution (success) [ 1.59s / 0.56s ] 394: Actions::Pulp::Repository::DistributorPublish (success) [ 3.29s / 0.85s ] 396: Actions::ElasticSearch::Repository::IndexContent (success) [ 10.66s / 10.66s ] 400: Actions::Pulp::Repository::Create (success) [ 0.39s / 0.39s ] 401: Actions::Katello::Repository::Create (success) [ 1.95s / 1.95s ] 406: Actions::Pulp::Repository::CopyRpm (success) [ 195.17s / 9.11s ] 408: Actions::Pulp::Repository::CopyErrata (success) [ 0.80s / 0.30s ] 410: Actions::Pulp::Repository::CopyPackageGroup (success) [ 11.03s / 1.46s ] 412: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.81s / 0.31s ] 414: Actions::Pulp::Repository::CopyDistribution (success) [ 0.83s / 0.32s ] 416: Actions::Katello::Repository::IndexErrata (success) [ 0.21s / 0.21s ] 418: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 193.41s / 192.89s ] 420: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 118.16s / 117.65s ] 423: Actions::Pulp::Repository::DistributorPublish (success) [ 6.67s / 3.62s ] 425: Actions::ElasticSearch::Repository::IndexContent (success) [ 38.23s / 38.23s ] 429: Actions::Pulp::Repository::RemoveRpm (success) [ 33.95s / 10.25s ] 431: Actions::Pulp::Repository::RemoveErrata (success) [ 33.32s / 9.57s ] 433: Actions::Pulp::Repository::RemovePackageGroup (success) [ 33.03s / 7.47s ] 435: Actions::Pulp::Repository::RemoveDistribution (success) [ 37.03s / 7.23s ] 437: Actions::Pulp::Repository::RemovePuppetModule (success) [ 35.93s / 6.51s ] 440: Actions::Pulp::Repository::CopyRpm (success) [ 65.25s / 10.23s ] 442: Actions::Pulp::Repository::CopyErrata (success) [ 0.36s / 0.36s ] 444: Actions::Pulp::Repository::CopyPackageGroup (success) [ 7.39s / 3.38s ] 446: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.96s / 0.45s ] 448: Actions::Pulp::Repository::CopyDistribution (success) [ 1.87s / 1.32s ] 451: Actions::Pulp::Repository::DistributorPublish (success) [ 7.49s / 2.92s ] 453: Actions::ElasticSearch::Repository::IndexContent (success) [ 29.87s / 29.87s ] 457: Actions::Pulp::Repository::Create (success) [ 0.84s / 0.84s ] 458: Actions::Katello::Repository::Create (success) [ 2.18s / 2.18s ] 463: Actions::Pulp::Repository::CopyRpm (success) [ 27.28s / 5.88s ] 465: Actions::Pulp::Repository::CopyErrata (success) [ 5.87s / 2.16s ] 467: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.12s / 0.58s ] 469: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.10s / 0.58s ] 471: Actions::Pulp::Repository::CopyDistribution (success) [ 1.34s / 0.67s ] 473: Actions::Katello::Repository::IndexErrata (success) [ 0.48s / 0.48s ] 475: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 15.32s / 12.75s ] 477: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 7.98s / 6.00s ] 480: Actions::Pulp::Repository::DistributorPublish (success) [ 5.83s / 0.74s ] 482: Actions::ElasticSearch::Repository::IndexContent (success) [ 5.76s / 5.76s ] 486: Actions::Pulp::Repository::RemoveRpm (success) [ 9.53s / 0.75s ] 488: Actions::Pulp::Repository::RemoveErrata (success) [ 9.27s / 0.77s ] 490: Actions::Pulp::Repository::RemovePackageGroup (success) [ 10.78s / 1.23s ] 492: Actions::Pulp::Repository::RemoveDistribution (success) [ 10.61s / 0.74s ] 494: Actions::Pulp::Repository::RemovePuppetModule (success) [ 10.36s / 0.68s ] 497: Actions::Pulp::Repository::CopyRpm (success) [ 9.77s / 1.08s ] 499: Actions::Pulp::Repository::CopyErrata (success) [ 1.70s / 0.66s ] 501: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.50s / 0.53s ] 503: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.17s / 0.47s ] 505: Actions::Pulp::Repository::CopyDistribution (success) [ 2.07s / 0.62s ] 508: Actions::Pulp::Repository::DistributorPublish (success) [ 2.02s / 1.20s ] 510: Actions::ElasticSearch::Repository::IndexContent (success) [ 6.55s / 6.55s ] 514: Actions::Pulp::Repository::Create (success) [ 0.96s / 0.96s ] 515: Actions::Katello::Repository::Create (success) [ 2.34s / 2.34s ] 520: Actions::Pulp::Repository::CopyRpm (success) [ 36.15s / 5.88s ] 522: Actions::Pulp::Repository::CopyErrata (success) [ 3.08s / 0.77s ] 524: Actions::Pulp::Repository::CopyPackageGroup (success) [ 2.46s / 0.63s ] 526: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 2.56s / 0.70s ] 528: Actions::Pulp::Repository::CopyDistribution (success) [ 4.42s / 0.62s ] 530: Actions::Katello::Repository::IndexErrata (success) [ 0.35s / 0.35s ] 532: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 19.80s / 18.30s ] 534: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 12.29s / 8.57s ] 537: Actions::Pulp::Repository::DistributorPublish (success) [ 3.32s / 1.05s ] 539: Actions::ElasticSearch::Repository::IndexContent (success) [ 8.45s / 8.45s ] 543: Actions::Pulp::Repository::RemoveRpm (success) [ 9.21s / 1.62s ] 545: Actions::Pulp::Repository::RemoveErrata (success) [ 9.48s / 1.80s ] 547: Actions::Pulp::Repository::RemovePackageGroup (success) [ 9.65s / 2.08s ] 549: Actions::Pulp::Repository::RemoveDistribution (success) [ 9.85s / 1.80s ] 551: Actions::Pulp::Repository::RemovePuppetModule (success) [ 10.04s / 2.00s ] 554: Actions::Pulp::Repository::CopyRpm (success) [ 15.75s / 3.29s ] 556: Actions::Pulp::Repository::CopyErrata (success) [ 1.00s / 0.48s ] 558: Actions::Pulp::Repository::CopyPackageGroup (success) [ 0.97s / 0.46s ] 560: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 1.21s / 0.70s ] 562: Actions::Pulp::Repository::CopyDistribution (success) [ 1.10s / 0.59s ] 565: Actions::Pulp::Repository::DistributorPublish (success) [ 1.04s / 0.54s ] 567: Actions::ElasticSearch::Repository::IndexContent (success) [ 5.28s / 5.28s ] 571: Actions::Pulp::Repository::Create (success) [ 0.64s / 0.64s ] 572: Actions::Katello::Repository::Create (success) [ 2.21s / 2.21s ] 577: Actions::Pulp::Repository::CopyRpm (success) [ 218.30s / 12.60s ] 579: Actions::Pulp::Repository::CopyErrata (success) [ 1.52s / 0.50s ] 581: Actions::Pulp::Repository::CopyPackageGroup (success) [ 0.82s / 0.31s ] 583: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.87s / 0.36s ] 585: Actions::Pulp::Repository::CopyDistribution (success) [ 1.04s / 0.53s ] 587: Actions::Katello::Repository::IndexErrata (success) [ 0.49s / 0.49s ] 589: Actions::Pulp::Repository::PurgeEmptyErrata (success) [ 202.54s / 202.02s ] 591: Actions::Pulp::Repository::PurgeEmptyPackageGroups (success) [ 146.46s / 145.92s ] 594: Actions::Pulp::Repository::DistributorPublish (success) [ 5.58s / 3.02s ] 596: Actions::ElasticSearch::Repository::IndexContent (success) [ 34.64s / 34.64s ] 600: Actions::Pulp::Repository::RemoveRpm (success) [ 46.38s / 15.70s ] 602: Actions::Pulp::Repository::RemoveErrata (success) [ 43.50s / 8.85s ] 604: Actions::Pulp::Repository::RemovePackageGroup (success) [ 43.20s / 9.21s ] 606: Actions::Pulp::Repository::RemoveDistribution (success) [ 42.65s / 8.53s ] 608: Actions::Pulp::Repository::RemovePuppetModule (success) [ 42.77s / 6.90s ] 611: Actions::Pulp::Repository::CopyRpm (success) [ 58.01s / 18.49s ] 613: Actions::Pulp::Repository::CopyErrata (success) [ 2.18s / 1.15s ] 615: Actions::Pulp::Repository::CopyPackageGroup (success) [ 1.26s / 0.75s ] 617: Actions::Pulp::Repository::CopyYumMetadataFile (success) [ 0.94s / 0.43s ] 619: Actions::Pulp::Repository::CopyDistribution (success) [ 0.79s / 0.79s ] 622: Actions::Pulp::Repository::DistributorPublish (success) [ 4.37s / 1.74s ] 624: Actions::ElasticSearch::Repository::IndexContent (success) [ 16.17s / 16.17s ] 628: Actions::Pulp::Repository::Create (success) [ 0.58s / 0.58s ] 631: Actions::Pulp::Repository::CopyPuppetModule (success) [ 9.02s / 1.57s ] 633: Actions::Pulp::Repository::CopyPuppetModule (success) [ 14.88s / 3.39s ] 635: Actions::Pulp::Repository::CopyPuppetModule (success) [ 15.98s / 3.39s ] 639: Actions::Pulp::Repository::RemovePuppetModule (success) [ 21.61s / 4.80s ] 641: Actions::Pulp::Repository::CopyPuppetModule (success) [ 5.55s / 1.42s ] 644: Actions::Pulp::Repository::DistributorPublish (success) [ 8.49s / 1.02s ] 646: Actions::ElasticSearch::ContentViewPuppetEnvironment::IndexContent (success) [ 122.96s / 122.96s ] 649: Actions::Pulp::Repository::Destroy (success) [ 38.57s / 7.33s ] 652: Actions::ElasticSearch::Repository::Destroy (success) [ 0.47s / 0.47s ] 656: Actions::Candlepin::Environment::SetContent (success) [ 0.77s / 0.77s ] 660: Actions::Candlepin::Environment::SetContent (success) [ 0.18s / 0.18s ] 664: Actions::Katello::ContentView::ErrataMail (success) [ 0.08s / 0.08s ] 666: Actions::Katello::ContentView::Publish (success) [ 2.48s / 2.48s ] Back
This should be resolved by https://bugzilla.redhat.com/show_bug.cgi?id=1323009 You might even consider them duplicates. I will move to on_qa and let qa verify it.
A bit of info from the Pull Request: On a lightly loaded satellite this only saved about 14 seconds for PurgeEmptyErrata (17s vs 3s) and 10 seconds for PurgeEmptyPackageGroups (11s vs 8s). However for heavily loaded systems the slowest part of each of the steps no longer exists (pulling all the package filenames and names for all rpms in the repo). Some servers have seen these steps take ~30 mintues or longer due to these calls.
Verified in Satellite 6.2.2 Snap 1.1 Published a content view with the following content. Total publish time was 7 minutes. 52028 Packages 7749 Errata ( 1671 5099 979 ) 50 Puppet Modules
*** Bug 1241261 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1885