Bug 472691

Summary: /etc/cron.daily/rpm can be 10x faster while keeping same results
Product: [Fedora] Fedora Reporter: Stéphane Lesimple <speed47_redhat>
Component: rpmAssignee: Panu Matilainen <pmatilai>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 10CC: ffesti, jnovy, n3npq, pmatilai, redhat-bugzilla
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-11-30 20:19:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stéphane Lesimple 2008-11-23 15:12:03 UTC
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.4) Gecko/2008111217 Fedora/3.0.4-1.fc9 Firefox/3.0.4

Currently, the daily report is generated using :

/bin/rpm -qa --qf '%{name}-%{version}-%{release}.%{arch}.rpm\n'

Which is horribly slow. If we add those two switches :

/bin/rpm -qa --nodigest --nosignature --qf '%{name}-%{version}-%{release}.%{arch}.rpm\n'

We get a 10x speedup (really measured with /usr/bin/time !), as rpm won't compute digests or check signatures, which we don't need for our output anyway.

Reproducible: Always

Steps to Reproduce:
1. time /etc/cron.daily/rpm
Actual Results:  
real    0m4.980s
user    0m4.808s
sys     0m0.097s

Expected Results:  
real    0m0.409s
user    0m0.263s
sys     0m0.082s

Comment 1 Jeff Johnson 2008-11-24 16:27:40 UTC
The difference between 4.980 seconds and 0.409 seconds for a job run once a day Really Does Not Matter.

Comment 2 Bug Zapper 2008-11-26 05:49:14 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 3 Stéphane Lesimple 2008-11-27 12:05:19 UTC
Well, when I can get a 10x performance improvement at no additional cost, with minimal effort, minor modifications, and without any possible adverse effect, I usually do it. It seemed like common sense to me until I read your reply.
I use Fedora on laptops, and I prefer not having a 100% CPU peak for several seconds every day, with the CPU fan speeding up suddenly, especially when it can be avoided that easily.
With this kind of tiny improvements all over the system, we get a more CPU-efficient OS, more battery lifetime, whatever.
It may Not Really Matter to you, but it may for your users. :)

Comment 4 Jeff Johnson 2008-11-27 13:49:00 UTC
The 4.571 seconds of "additional cost" confirms that rpmdb headers
are intact.

But you might juts as well turn off the cron job if you don't want to
see the 100% CPU spike. The output from the cron job will cause your
disk to spin-up no matter whether signatures/digests are checked (or not).
Avoiding disk I/O ia  tiny improvement that will help your battery life, and
preserve your disk life for many times longer than 4.571 seconds.

Comment 5 Robert Scheck 2008-11-30 19:36:43 UTC
This report seems to be nonsens. For a cron job it shouldn't really matter,
whether it runs a bit longer or not. I don't which CPU you've got, but the
very slow CPU of my EeePC does not have 100% CPU peak for several seconds...

Without your "speedup" rpm is logging and reporting if something does wrong
with the RPM database. But if you're really focussed to save less than one 
second (!) of CPU time per day *shrug*, then please save also the last few
hundredth seconds:

$ time rpm -qa --qf '%{name}-%{version}-%{release}.%{arch}.rpm\n' \
    > /dev/null 2>&1

real 0m0.594s
user 0m0.411s
sys  0m0.123s
$

$ time rpm -qa --nodigest --nosignature --qf \
    '%{name}-%{version}-%{release}.%{arch}.rpm\n' > /dev/null 2>&1

real 0m0.500s
user 0m0.272s
sys  0m0.159s
$

$ time rpm -qa --nodigest --nosignature --nohdrchk --qf \
    '%{name}-%{version}-%{release}.%{arch}.rpm\n' > /dev/null 2>&1

real 0m0.403s
user 0m0.205s
sys  0m0.137s
$

BTW, ever thought just to kill the cron job if it wastes CPU time for you?

Comment 6 Stéphane Lesimple 2008-11-30 20:19:02 UTC
This bug report was not about me: I know how to deactivate a cron job.

I thought adding these switches was leading *exactly* to the same result. But as you both pointed out, it seems that not specifying nosignature/nodigest allows rpm to check if the rpm database is not corrupted. Now that makes sense to use this extra CPU time.

Robert : weird that the difference between the 2 first commands is so tiny on your Eeepc, here it really takes 5 seconds vs 0.5 seconds. But, hey, please don't get ironic that fast. It's all about having better software, right?

Anyway, I think we can close this bug now.

Comment 7 Jeff Johnson 2008-11-30 20:43:59 UTC
FYI: Robert has likely disabled --nosignature --nodigest persistently form rpmdb queries using a macro.

No its all about user friendliness, saving battery life, not spinning fans nor eating cpu cycles
and saving ~4 seconds a day. The software is already well enough engineered that any
or all of those goals can be achieved without changing the software at all.