Bug 860686 - Yum importer/distributor logging is too verbose in INFO
Yum importer/distributor logging is too verbose in INFO
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: rpm-support (Show other bugs)
Master
Unspecified Unspecified
unspecified Severity unspecified
: ---
: Sprint 40
Assigned To: Pradeep Kilambi
Preethi Thomas
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-09-26 09:34 EDT by Jay Dobies
Modified: 2013-01-09 12:07 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-01-09 12:07:45 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jay Dobies 2012-09-26 09:34:43 EDT
I think some debug logging was left in as INFO. After doing a sync/publish there is a lot of spam to the log files. For instance, the full metadata of each RPM is logged:

2012-09-26 09:29:38,433 1308:140595276277504: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:197 RPM object {'size': 86209, 'vendor': '', 'checksumtype': 'sha256', 'license': 'GPLv2', 'checksum': '1f3b15c15a7b7f0896860c006b298320fcb53ec1bae356391aa8cbf00c1cd51e', 'description': 'Provides a collection of platform plugins that extend the Pulp platform\nto provide RPM specific support.', 'pkgpath': '/var/lib/pulp/content/rpm/.//pulp-rpm-plugins/0.0.328/2.fc17/noarch/1f3b15c15a7b7f0896860c006b298320fcb53ec1bae356391aa8cbf00c1cd51e', 'savepath': u'/var/lib/pulp/working/repos/r1/importers/yum_importer/r1/', 'buildhost': 'localhost', 'downloadurl': 'http://repos.fedorapeople.org/repos/pulp/pulp/v2/testing/fedora-17/x86_64/pulp-rpm-plugins-0.0.328-2.fc17.noarch.rpm', 'item_type': 'rpm', 'epoch': '0', 'version': '0.0.328', 'arch': 'noarch', 'provides': [('pulp-rpm-plugins', 'EQ', ('0', '0.0.328', '2.fc17')), ('config(pulp-rpm-plugins)', 'EQ', ('0', '0.0.328', '2.fc17'))], 'relativepath': 'pulp-rpm-plugins-0.0.328-2.fc17.noarch.rpm', 'release': '2.fc17', 'fileName': 'pulp-rpm-plugins-0.0.328-2.fc17.noarch.rpm', 'filename': 'pulp-rpm-plugins-0.0.328-2.fc17.noarch.rpm', 'requires': [('python-pulp-rpm-common', 'EQ', ('0', '0.0.328', None)), ('pulp-server', 'EQ', ('0', '0.0.328', None))], 'name': 'pulp-rpm-plugins'} ; KEY : ('pulp-rpm-plugins', '0', '0.0.328', '2.fc17', 'noarch', 'sha256', '1f3b15c15a7b7f0896860c006b298320fcb53ec1bae356391aa8cbf00c1cd51e')


There's a message for each unit that doesn't need to be re-publish, which will be the common case after the initial publish:

2012-09-26 09:29:38,905 1308:140595276277504: pulp.plugins.yum_distributor.distributor:INFO: distributor:553 Unit exists at: /var/lib/pulp/content/rpm/.//grinder/0.1.6/1.fc17/noarch/8eb74f0f282a793b91613eba0fc67e5e731179e31dd4f63e2b49582ab4d6f8d0/grinder-0.1.6-1.fc17.noarch.rpm we need to symlink to: /var/lib/pulp/working/repos/r1/distributors/yum_distributor/grinder-0.1.6-1.fc17.noarch.rpm


We should remove or move these to DEBUG as they clutter the logs.
Comment 1 Pradeep Kilambi 2012-10-03 10:22:06 EDT
commit 2c286ab2bf8f75a2b30407bfe6cda6059ae8cfb8
Comment 2 Jeff Ortel 2012-10-05 17:43:08 EDT
build: 0.331
Comment 3 Preethi Thomas 2012-10-12 10:04:09 EDT
verified
[root@pulp-master ~]# rpm -q pulp-rpm-server
pulp-rpm-server-0.0.331-1.fc17.noarch
[root@pulp-master ~]# 

log during a sync

2012-10-12 09:46:25,903 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:477 Begin sync of repo <pulp> from feed_url <http://repos.fedorapeople.org/repos/pulp/pulp/v2/candidate/fedora-17/x86_64/>
2012-10-12 09:46:35,587 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:633 35 rpms are available in the source repo pulp, calculated in 7.00950622559e-05 seconds
2012-10-12 09:46:35,591 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:644 Repo <pulp> 0 existing rpm units, 0 have been orphaned, 35 new rpms, 0 missing rpms.
2012-10-12 09:46:35,592 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:656 Delta RPMs to sync 0
2012-10-12 09:46:35,595 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:662 0 drpms are available in the source repo pulp, calculated in 0.00260996818542 seconds
2012-10-12 09:46:35,595 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:668 Repo <pulp> 0 existing drpm units, 0 have been orphaned, 0 new drpms, 0 missing drpms.
2012-10-12 09:46:42,713 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:685 0 distributions are available in the source repo pulp, calculated in 7.11751699448 seconds
2012-10-12 09:46:42,713 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:689 Repo <pulp> 0 existing distro units, 0 have been orphaned, 0 new distro files, 0 missing distro.
2012-10-12 09:47:32,171 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:523 Finished download of pulp in 49.4571661949econds.  35 successes, 35 downloads, 0 errors
2012-10-12 09:47:32,175 24046:140061718845184: pulp.plugins.yum_importer.importer_rpm:INFO: importer_rpm:336 checksum type info [{u'checksum_type': u'sha256'}] set to repo scratchpad
2012-10-12 09:47:32,407 24046:140061718845184: pulp.plugins.yum_importer.drpm:INFO: drpm:117 purging orphaned drpm units
2012-10-12 09:47:32,449 24046:140061718845184: pulp.plugins.yum_importer.comps:INFO: comps:224 No package group/category data found in </var/lib/pulp/working/repos/pulp/importers/yum_importer/pulp>
2012-10-12 09:47:32,450 24046:140061718845184: pulp.plugins.yum_importer.comps:INFO: comps:330 Parsed comps data from </var/lib/pulp/working/repos/pulp/importers/yum_importer/pulp>: 0 groups and 0 categories are available in <pulp>
2012-10-12 09:47:32,455 24046:140061718845184: pulp.plugins.yum_importer.comps:INFO: comps:335 Existing package groups/categories from <pulp>: 0 groups, 0 categories
2012-10-12 09:47:32,455 24046:140061718845184: pulp.plugins.yum_importer.comps:INFO: comps:373 Comps Summary: {'time_total_sec': 0.00707697868347168, 'num_orphaned_groups': 0, 'num_available_groups': 0, 'num_new_groups': 0, 'num_new_categories': 0, 'num_orphaned_categories': 0, 'num_available_categories': 0} 
 Details: {}
2012-10-12 09:47:32,460 24046:140061718845184: pulp.server.dispatch.task:INFO: task:154 Task 37755568-1473-11e2-9ad4-54520075d18e: CallRequest: RepoSyncManager.sync(u'pulp', sync_config_override=None, importer_config={}, importer_instance=<yum_importer.importer.YumImporter object at 0x7f6284227fd0>) SUCCEEDED
2012-10-12 09:47:32,689 24046:140061718845184: pulp.plugins.pulp_rpm.yum_plugin.metadata:INFO: metadata:519 Performing per unit metadata merge on 35 units
2012-10-12 09:47:32,718 24046:140061718845184: pulp.plugins.pulp_rpm.yum_plugin.metadata:INFO: metadata:545 per unit metadata merge completed in 0.0278210639954 seconds
2012-10-12 09:47:32,962 24046:140061718845184: pulp.plugins.pulp_rpm.yum_plugin.metadata:INFO: metadata:583 comps xml path does not exist; skipping merge
2012-10-12 09:47:32,962 24046:140061718845184: pulp.plugins.pulp_rpm.yum_plugin.metadata:INFO: metadata:595 updateinfo xml path does not exist; skipping merge
2012-10-12 09:47:32,963 24046:140061718845184: pulp.plugins.pulp_rpm.yum_plugin.metadata:INFO: metadata:551 check scratchpad for any repodata
2012-10-12 09:47:32,963 24046:140061718845184: pulp.plugins.pulp_rpm.yum_plugin.metadata:INFO: metadata:722 Metadata generation finished in 0.273802042007 seconds
2012-10-12 09:47:32,963 24046:140061718845184: pulp.plugins.yum_distributor.distributor:INFO: distributor:462 HTTPS Publishing repo <pulp> to </var/lib/pulp/published/https/repos/repos/pulp/pulp/v2/candidate/fedora-17/x86_64>
2012-10-12 09:47:32,964 24046:140061718845184: pulp.plugins.yum_distributor.distributor:INFO: distributor:510 Publish complete:  summary = <{'num_package_units_errors': 0, 'https_publish_dir': u'/var/lib/pulp/published/https/repos/repos/pulp/pulp/v2/candidate/fedora-17/x86_64', 'num_package_categories_published': 0, 'num_distribution_units_errors': 0, 'skip_metadata_update': False, 'num_distribution_units_published': 0, 'num_package_groups_published': 0, 'num_package_units_attempted': 35, 'num_package_units_published': 35, 'num_distribution_units_attempted': 0, 'relative_path': u'repos/pulp/pulp/v2/candidate/fedora-17/x86_64/'}>, details = <{'errors': [], 'time_metadata_sec': 0.2748429775238037}>
2012-10-12 09:47:32,969 24046:140061718845184: pulp.server.dispatch.task:INFO: task:154 Task 37755c99-1473-11e2-9ad4-54520075d18e: CallRequest: RepoPublishManager.publish(u'pulp', u'yum_distributor', distributor_instance=<yum_distributor.distributor.YumDistributor object at 0x7f62840e50d0>, distributor_config={}) SUCCEEDED
Comment 4 Preethi Thomas 2013-01-09 12:07:45 EST
Pulp v2.0 released

Note You need to log in before you can comment on or make changes to this bug.