This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 799512 - katello CLI - sync progress starts at 100.0%, then changes to 1.1%
katello CLI - sync progress starts at 100.0%, then changes to 1.1%
Status: CLOSED CURRENTRELEASE
Product: Red Hat Satellite 6
Classification: Red Hat
Component: katello-agent (Show other bugs)
6.0.1
Unspecified Unspecified
low Severity low (vote)
: Unspecified
: --
Assigned To: Ivan Necas
Og Maciel
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-02 15:35 EST by James Laska
Modified: 2014-09-18 11:32 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-08-22 14:30:07 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description James Laska 2012-03-02 15:35:21 EST
Description of problem:

When performing a provider synchronization using the katello CLI, the progress meeting starts out at 100% ... then jumps back down to the real % complete, and works it's way back to 100%.

I've seen this before, I'm not certain, but it's possible this error only happens when syncing previously synced provider.

Version-Release number of selected component (if applicable):

 * candlepin-0.5.23-1.el6.src.rpm
 * katello-0.1.301-2.el6.src.rpm
 * katello-candlepin-cert-key-pair-1.0-1.src.rpm
 * katello-certs-tools-1.0.3-1.el6.src.rpm
 * katello-cli-0.1.100-2.el6.src.rpm
 * katello-configure-0.1.101-1.el6.src.rpm
 * katello-qpid-broker-key-pair-1.0-1.src.rpm
 * katello-qpid-client-key-pair-1.0-1.src.rpm
 * katello-selinux-0.1.8-1.el6.src.rpm
 * pulp-1.0.0-4.el6.src.rpm

Steps to Reproduce:
1. katello -u admin -p admin provider synchronize --name "Red Hat"
  
Actual results:

Output starts at ...

Progress: [##################################################] 100.0%

Then changes to ...
Progress: [                                                  ] 1.1%

Expected results:

 * I would expect the synchronization progress to start at 0, and increase, up to 100%.

Additional info:
Comment 1 James Laska 2012-03-02 15:36:26 EST
Seems like a few more jumps were needed before the % complete was *increasing*

== Actual results ==

Output starts at ...

Progress: [##################################################] 100.0%
Progress: [                                                  ] 1.1%
Progress: [                                                  ] 0.5%
Progress: [                                                  ] 0.7%
Progress: [                                                  ] 1.4%
Progress: [#                                                 ] 3.3%
...
Comment 4 Ivan Necas 2012-03-14 10:27:35 EDT
Fixed in commit 81f3eac56e17521a27cbd63380e8f5929cf04a16

In case that the task consist of more subtasks (usually product or provider
synchronization) the problem was we calculated the progress from
finished/unfinished files reported by pulp but we didn't know the
total number of files at the beginning. So the total number of files was
changing every time a new sync task changed from pending to running.

After this change the progress for tasks consisting from more subtasks is
calculated from this number of finished/unfinished tasks. For single repo
synchronization it stays the same. For product or provider synchronization the
progress bar won't be theoretically so accurate (the progress bar will grow the
same amount regardless small or large repo). But since we can't get the size of
repo before the synchronization started it's better. For more details the user
can to to UI (sync management).
Comment 7 Og Maciel 2012-03-15 13:09:46 EDT
Verified:

* candlepin-0.5.26-1.el6.noarch
* candlepin-tomcat6-0.5.26-1.el6.noarch
* katello-0.1.304-1.el6.noarch
* katello-all-0.1.304-1.el6.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.0.4-1.el6.noarch
* katello-cli-0.1.105-1.el6.noarch
* katello-cli-common-0.1.105-1.el6.noarch
* katello-common-0.1.304-1.el6.noarch
* katello-configure-0.1.106-1.el6.noarch
* katello-glue-candlepin-0.1.304-1.el6.noarch
* katello-glue-foreman-0.1.304-1.el6.noarch
* katello-glue-pulp-0.1.304-1.el6.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-0.1.9-1.el6.noarch
* pulp-1.0.0-5.el6.noarch
* pulp-common-1.0.0-5.el6.noarch
* pulp-selinux-server-1.0.0-5.el6.noarch
Comment 8 James Laska 2012-03-23 15:48:29 EDT
On 3 different systems, located in 2 sites (BOS and RDU), I've observed a strange behavior with the katello CLI progress meter during a 'provider sync' from cdn.rcm-qa.redhat.com.

On all 3 different systems, using the following packages:
 * katello-0.1.306-1.el6.src.rpm
 * katello-candlepin-cert-key-pair-1.0-1.src.rpm
 * katello-certs-tools-1.0.6-1.el6.src.rpm
 * katello-cli-0.1.107-1.el6.src.rpm
 * katello-configure-0.1.106-1.el6.src.rpm
 * katello-qpid-broker-key-pair-1.0-1.src.rpm
 * katello-qpid-client-key-pair-1.0-1.src.rpm
 * katello-selinux-0.1.9-1.el6.src.rpm
 * pulp-1.0.0-7.el6.src.rpm

I've observed that during a provider sync, the progress meter starts with a consistent rate of increase.  However, after a short time, the progress meter stops.

Two examples are shown below:

> # Stuck at 25% for 2 hours and 30 minutes
> katello -u admin -p admin provider synchronize --name "Red Hat"
> Progress: [############                                      ] 25.0%

> # Stuck at 12% for 3 hours and 50 minutes
> katello -u admin -p admin provider synchronize --name "Red Hat"
> Progress: [######                                            ] 12.5%

I can confirm that content *is* syncing by `tail`ing /var/log/pulp/{grinder,pulp}.log.  However, for some reason the progress meter stops.

I am attempting to sync the following repositories from the CDN:

> # katello -u admin -p admin repo list -v --environment Library | grep Name
> Name:          Red Hat CloudForms Cloud Engine Beta RPMs x86_64 6Server
> Name:          Red Hat CloudForms System Engine RPMs x86_64 6Server
> Name:          Red Hat CloudForms System Engine Beta RPMs x86_64 6Server
> Name:          Red Hat CloudForms Cloud Engine RPMs x86_64 6Server
> Name:          Red Hat Enterprise Linux 5 Server RPMs x86_64 5Server
> Name:          Red Hat CloudForms Tools for RHEL 6 Beta RPMs i386 6Server
> Name:          Red Hat CloudForms Tools for RHEL 6 Beta RPMs x86_64 6Server
> Name:          Red Hat Enterprise Linux 6 Server RPMs x86_64 6Server
> Name:          Red Hat CloudForms Tools for RHEL 6 RPMs x86_64 6Server
> Name:          Red Hat CloudForms Tools for RHEL 5 RPMs x86_64 5Server
> Name:          Red Hat CloudForms Tools for RHEL 5 Beta RPMs x86_64 5Server
> Name:          Red Hat Enterprise Linux 5 Server RPMs i386 5Server
> Name:          Red Hat CloudForms Tools for RHEL 5 RPMs i386 5Server
> Name:          Red Hat CloudForms Tools for RHEL 5 Beta RPMs i386 5Server
> Name:          Red Hat Enterprise Linux 6 Server RPMs i386 6Server
> Name:          Red Hat CloudForms Tools for RHEL 6 RPMs i386 6Server

Could there be a new problem related to the progress meter, introduced by this change?
Comment 9 James Laska 2012-03-23 16:03:49 EDT
inecas: Do you have any thoughts on comment#8, could this be a problem introduced by this bug fix?
Comment 11 Ivan Necas 2012-03-26 04:25:33 EDT
Yeah. I've tried to say that this might be the issue in comment https://bugzilla.redhat.com/show_bug.cgi?id=799512#c4. So the issue you're seeing is when you synchronize 4 repositories: one small and 3 large. After the first is synchronized you see the 25% very soon. But then another repositories start syncing that take much more time.

The only way how to deal with that came to my mind would be to create multiple progress-bars for each repo sync. However I'm afraid it won't be possible without depending on ncurses. This doesn't have to be wrong but having less dependencies possible for CLI is not a bad idea.
Comment 12 James Laska 2012-03-26 09:19:49 EDT
(In reply to comment #11)
> The only way how to deal with that came to my mind would be to create multiple
> progress-bars for each repo sync. However I'm afraid it won't be possible
> without depending on ncurses. This doesn't have to be wrong but having less
> dependencies possible for CLI is not a bad idea.

Hmm, good thinking.  I think multiple progress bars is a really good idea (probably post v1).  Additionally, I don't think this provides too much detail.  In fact, I think I'm more familiar with this approach from other tools, and I think it would be much more informative for the user.  If it's possible, I definitely support the idea :)

I could envision this working much the same way 'yum' emits download progress when syncing repodata or downloading packages.  Should I open this up as a new bug against post-v1?
Comment 14 Mike McCune 2013-08-16 14:07:44 EDT
getting rid of 6.0.0 version since that doesn't exist

Note You need to log in before you can comment on or make changes to this bug.