Bug 140207 - makewhatis -u (update) does not function, including in cron.daily.
Summary: makewhatis -u (update) does not function, including in cron.daily.
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: man
Version: 3
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Ivana Varekova
QA Contact: Ben Levenson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2004-11-21 03:08 UTC by Dale Stimson
Modified: 2007-11-30 22:10 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2005-04-13 08:56:59 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
patch against makewhatis from man-1.5o1 to restore -u functionality (1.01 KB, patch)
2005-05-10 18:48 UTC, Gilles Detillieux
no flags Details | Diff

Description Dale Stimson 2004-11-21 03:08:21 UTC
Description of problem:
                                                                     
                                                                      
"makewhatis -u" should update the whatis database with all changes
since the last makewhatis run , but does not.
                                                                     
                                                                      
Synopsis:
                                                                     
                                                                      
Command "makewhatis -u" finds no files to be updated because find arg
"-cmin 0" matches only files that are exactly 0 minutes old (in other
words, no files).  From /usr/bin/makewhatis:
                                                                     
                                                                      
    -u) findarg="-cmin 0"
        update=1
        continue;;
                                                                     
                                                                      
Compare against how it was under RHL 9 with man-1.5k-6 on a RHL 9 system:
                                                                     
                                                                      
    -u) if [ -e /var/cache/man/whatis ]; then
          findarg="-newer /var/cache/man/whatis"
          update=1
        fi
        continue;;
                                                                     
                                                                      
A minor problem with using file /var/cache/man/whatis as the timestamp
is that its modification time will be the end (instead of the
beginning) of the previous makewhatis run.
                                                                     
                                                                      
I think the right solution would be to have makewhatis create a file
explicitly to be used with "-newer".  That file could be created
during makewhatis initialization as
/var/cache/man/whatis_timestamp.tmp and renamed it to
/var/cache/man/whatis_timestamp when the run is completed.  That way
the time would be correct and an incomplete run would not leave a
timestamp.
                                                                     
                                                                      
As an aside, perhaps the same file could be used as a lock file to
help avoid the simultaneous running of multiple instances of
makewhatis, allowing the lock file logic to be removed from files
/etc/cron.daily/00-makewhatis.cron and
/etc/cron.weekly/00-makewhatis.cron .
                                                                     
                                                                      
By the way, /etc/cron.weekly/00-makewhatis executes "makewhatis -w",
which should work fine.  That would have helped to mask the problem
with the update because the whatis database is properly re-generated
once a week.
                                                                     
                                                                      
                                                                     
                                                                      
Version-Release number of selected component (if applicable):
                                                                     
                                                                      
man-1.5o1-7 from most recent FC3 errata.
Also, man-1.5m2-9 (as it originally shipped with FC3).
                                                                     
                                                                      
                                                                     
                                                                     
                                                                     
                                                                      
How reproducible:
Always.
                                                                     
                                                                      
                                                                     
                                                                      
Steps to Reproduce:
1.  Delete the existing whatis database
        rm /var/cache/man/whatis
                                                                     
                                                                      
2.  Touch one of the man pages so there is legitimately something to
update.
        touch /usr/share/man/man8/makewhatis.8.gz
                                                                     
                                                                      
3.  Wait a few seconds that the touched file is more than 0 minutes old.
    I only had to wait a second or so.
                                                                     
                                                                      
4.  Execute a makewhatis update, as in /etc/cron.daily/00-makewhatis.cron
        makewhatis -w -u
                                                                     
                                                                      
                                                                     
                                                                      
5.  ls -l /var/cache/man/whatis
    gives:
    -rw-r--r--  1 root root 0 Nov 20 18:44 /var/cache/man/whatis
                                                                     
                                                                      
                                                                     
                                                                      
Actual results:
                                                                     
                                                                      
No files are found for the update, even if they should be.
                                                                     
                                                                      
                                                                     
                                                                      
                                                                     
                                                                      
Expected results:
                                                                     
                                                                      
All man page files changed since the previous run should be included
in the update.

Comment 1 Ivana Varekova 2005-03-17 15:46:20 UTC
Thank you for your notice.
The problem is fixed in man-1.5p-2. 
There is changed the findarg to "-ctime 0" again.
I did noot use the version with time stamps becouse of there should be problem
with using makewhatis with parametres. 
The man pages must be changed yet.
Ivana Varekova

Comment 2 Dale Stimson 2005-03-17 16:10:54 UTC
I see what you mean about the problem if directory parameters are given on the
makewhatis command line and have to agree that the "-ctime 0" solution is an
acceptable compromise.  It's too bad it adds a "one day" window to "-u", however. 

I wonder if it would be better to use "-ctime 7" or "-ctime 8".  That wouldn't
require significantly more processing time and it would work better if the daily
cron job was missed (e.g., laptop powered off for a few days).

Comment 3 Ivana Varekova 2005-03-18 07:17:30 UTC
I think this change is sufficient becouse of there is the weekly cron job which
make makewhatis without the parametre -u. So if the daily cron job was missed,
the weekly cron job repaired this problem. 
I think this solution is quite good becouse of there can't be man pages which
are not file in whatis.
 
Ivana Varekova

Comment 4 Ivana Varekova 2005-04-08 11:16:22 UTC
Man man-pages was be fixed (in man-1.5p-4).
Ivana Varekova 

Comment 5 Gilles Detillieux 2005-05-10 18:48:48 UTC
Created attachment 114217 [details]
patch against makewhatis from man-1.5o1 to restore -u functionality

Ivana, I'm not sure why the use of command line parameters would make the use
of the database's timestamp a problem in deciding what files need to be
updated.  Presumably, if you're specifying non-standard man directories on the
command line, you'll also be specifying a non-standard database location as
well, otherwise you'd end up messing up the standard (or default) database
anyway, so the concept of updating it wouldn't make much sense anyway.

However, when it comes time to updating the database, whether the default one
or a user-specified one, it doesn't make sense to assume that only changes in
the last second, minute, day or week need to be recorded.  These are all
arbitrary quantities that assume a specific schedule for makewhatis runs.  That
assumption doesn't hold when running makewhatis with command line arguments
that specify alternate man page and database locations, and it doesn't hold
when the system isn't kept running in the wee hours when the daily and weekly
cron jobs are run.

It seems to me that using the database's timestamp (i.e. mtime) and comparing
against the man page files' change times (i.e. ctimes), which is what the find
-cnewer option does, is the only approach that really makes sense.  Am I
missing something here?

Attached is my proposed fix, which sets the findarg variable after the final
database file name is decided, and makes sure the database isn't modified until
the very end, so that the find command is comparing against the unmodified
database for all man directories.  As far as I can tell, this patch solves the
problem.  Can anyone point out what's wrong with doing it this way, because I
don't see it?

Comment 6 Ivana Varekova 2005-05-11 09:06:58 UTC
Gilles,
I think -u is quite problematic option. 
In your patch you use whatisdb timestamp, but it should be problem becouse of
there is some delay between begin and end of makewhatis run. (See comment 0.)
This option is used in cron.daily so I think this version is quite acceptable.
There should be problem with timestamps becouse of you expect some relations
between whatis databases and used directories, it could not be true.
But there is another problem if you install/update any rpm package then new
manpages have quite old timestamp - it is not the time when they are
installed/updated, so it does not metter that you use whatisdb timestamp to
identify the strict time when whatisdb was changed. There could be man pages
which will not be found if you use -u.
Ivana Varekova

Comment 7 Gilles Detillieux 2005-05-11 16:53:33 UTC
As the original comment says, the delay between the start and end of the makewhatis run is a 
minor problem.  In virtually all cases, a system will have only one administrator who's installing 
packages and or man pages, and likely won't be doing this during the makewhatis run. I'll take that 
over all the other compromises involved in using -cmin or some other find option that doesn't take 
any account of the time the database was last updated.

As for the older timestamps in man pages installed by rpm, I think you're confusing the ctime and 
mtime timestamps.  The older makewhatis script used the -newer option, which compared mtime 
to mtime, so it would have missed rpm-installed man pages under -u.  What I'm proposing is to 
use -cnewer, which will compare the ctime of the man page files to the mtime of the database. The 
ctime is updated when the man pages are installed from the rpm package, so any rpm-installed 
man pages that show up since the last database update will be found.  Try an "ls -lc" of some man 
page files to see their ctime.


Note You need to log in before you can comment on or make changes to this bug.