As a sysadmin syncing repositories is a long and time consuming task that often requires debugging and progress tracking outside the scope of an API call. While it is useful to be able to track task progress via the API and CLI there are times it would be highly convenient to be able to tail a logfile to see what Pulp was actually doing while CPU and bandwidth were being utilized. If there is a way to configure logging in Pulp such that we could watch task progress and see how far through a download it would make administering a Satellite 6 server much easier. Pulp 1.x had this tracked via grinder but we no longer have this in the 2.x series.
Is viewing the progress report in "pulp-admin tasks details --help" sufficient for this use case?
pulp-admin isn't installed by default and I don't think is shipped (pulp-admin-client package). The information that pulp-admin rpm repo sync status --repo-id provides is somewhat useful. pulp-admin tasks details would be useful, too.
nope, we don't ship with pulp-admin and relying on API calls as the only way to debug running processes can be pretty arduous for a sysadmin. we could add this function to our 'hammer' cli but the problem would really remain the same, you are forced to poll a CLI call poll an API to try and debug running processes. I'm generally trying to make the case for enhanced logging options for tasks within pulp. It doesn't have to be on by default but myself and other Sat6 users often want to just be able to enable debug logging tail a log file to figure out what is going on.
I'm not clear on why it's a problem to monitor task progress through an API. I think we can improve the quality and usefulness of pulp's logging, but it's worth knowing that all pulp logging is now (2.4) going through syslog. Since pulp is multi-process now, we can't share a single log file among processes. And since pulp can be run across multiple machines, it could be difficult to figure out which machine has the log file where your task's progress is being reported. I really think the right way to make task progress available is through the API. However, changes in task state would be valuable to have in pulp's regular logs. Would you be ok with changing the scope of this RFE to something along the lines of "ensure that pulp logs state changes for user-facing tasks"?
Having talked with McCune, we agreed that task state changes need to be logged. This bug should also include a fresh look at the rpm sync process, with log statements added as makes sense. Logging download progress may be complicated, but if we can log at debug level every 100 RPMs that are download (DEBUG: downloaded 200 of 1739 rpms), that would be useful. But we should avoid adding complexity to the workflow just for the sake of adding progress logging.
agreed with Comment #5. looking forward to the more detailed logging
https://github.com/pulp/pulp_rpm/pull/501
This was fixed in pulp-2.4.0-0.19.beta.
Created attachment 904744 [details] journalctl
tested in pulp-2.4.0-0.19.beta with DEBUG log level enabled: More info in the attachment. # pulp-admin -u admin -p admin rpm repo create --repo-id f20 --feed http://ftp.linux.cz/pub/linux/fedora/linux/releases/20/Fedora/x86_64/os/ --relative-url myurlf20 Successfully created repository [f20] [root@ec2-54-74-97-52 var]# pulp-admin -u admin -p admin rpm repo sync run --repo-id f20 +----------------------------------------------------------------------+ Synchronizing Repository [f20] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [/] ^C # pulp-admin -u admin -p admin tasks details --task-id 615652bd-82af-45e9-83e9-bbfef5570854 +----------------------------------------------------------------------+ Task Details +----------------------------------------------------------------------+ Operations: sync Resources: f20 (repository) State: Running Start Time: 2014-06-09T14:20:17Z Finish Time: Incomplete Result: Incomplete Task Id: 615652bd-82af-45e9-83e9-bbfef5570854 Progress Report: Yum Importer: Comps: State: NOT_STARTED Content: Details: Drpm Done: 0 Drpm Total: 0 Rpm Done: 1125 Rpm Total: 4071 Error Details: Items Left: 2946 Items Total: 4071 Size Left: 2799997884 Size Total: 4244826490 State: IN_PROGRESS Distribution: Error Details: Items Left: 0 Items Total: 0 State: NOT_STARTED Errata: State: NOT_STARTED Metadata: State: FINISHED from logs: # journalctl SYSLOG_IDENTIFIER=pulp | grep "items left to download"| tail -10 Jun 09 14:26:50 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 900 items left to download. Jun 09 14:26:59 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 800 items left to download. Jun 09 14:27:06 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 700 items left to download. Jun 09 14:27:10 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 600 items left to download. Jun 09 14:27:15 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 500 items left to download. Jun 09 14:27:22 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 400 items left to download. Jun 09 14:27:32 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 300 items left to download. Jun 09 14:27:43 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 200 items left to download. Jun 09 14:27:49 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 100 items left to download. Jun 09 14:27:56 ec2-54-74-97-52.eu-west-1.compute.amazonaws.com pulp[30709]: pulp_rpm.plugins.importers.yum.report:DEBUG: 0 items left to download.
This has been fixed in Pulp 2.4.0-1.